GSim · simulator engineering report · 12 September 2026

Gaussian scenes.
Parallel GPU simulation.

A self-contained simulator for the reconstructed UR5, RG2 gripper and wrist camera. Multiple approaches tested on one RTX A6000, with visual correctness checked against the original environment.

Full benchmark report ↓Raw measurements ↓Methods
3,613aggregate env steps / second
128×128 RGB · 16,384 worlds
27.29 GiBprocess GPU memory at that rate
NVML sampling · includes Warp + Torch
16,384 worldslargest completed batch
Independent physics state · shared scene

Each observation includes 50 physics steps at 1 ms plus fresh RGB. Aggregate throughput is not per-world frame rate: the fastest configuration takes 4.53 seconds per batch. No RL policy is included.

The configuration to build on

MJLab / MuJoCo Warp physics + calibrated, batched gsplat + a reference-compatible foreground renderer. Keep full degree-three appearance, enable shadows, and use chunks of 128 cameras. Simplified room/base visual meshes offer a measured speed–accuracy tradeoff; physical collision geometry is preserved.

18× lower Gaussian renderer memory

At 4,096 views, chunking reduced Torch allocation from about 38,962 to 2,152 MiB while retaining roughly 6.4k images/s. These are renderer-only measurements.

Camera and lighting repaired

Near-plane clipping and textured lighting needed explicit compatibility fixes. The faster raw Warp images failed the visual gate; the corrected renderer passes.

Stock MuGS did not run correctly

Its mask expansion is incompatible with current two-channel segmentation. Camera and SH-loading assumptions also differ from this scene.

Rasterization prototype failed quality

nvdiffrast ran on CUDA, but vertex-baked table textures lost detail without improving useful throughput. Lossy SH and radius shortcuts were also rejected.

Qualified physics + RGB measurements

All rows use shadows, calibrated cameras, SH degree 3, packed rendering, 128-camera chunks and 50 physics substeps. LOD means simplified room/base visibility meshes. “Skip background shading” omits lighting for mesh colors subsequently replaced by Gaussians.

RendererRGB sizeWorldsEnv steps/sBatch msProcess GiB
LOD128×128641,24551.42.18
LOD + skip background shading128×128641,28349.92.18
LOD128×1282562,241114.24.40
LOD + skip background shading128×1282562,424105.64.40
LOD128×1281,0242,896353.65.18
LOD + skip background shading128×1281,0243,256314.55.18
LOD + skip background shading128×1284,0963,5731,146.59.04
LOD + skip background shading128×1288,1923,6112,268.615.17
LOD + skip background shading128×12816,3843,6134,534.827.29
LOD256×2561,0241,357754.45.75
Full meshes320×2006472987.82.58
LOD320×2006490570.72.30
LOD + skip background shading320×2006498265.22.30
LOD320×2002561,326193.14.63
Full meshes320×2001,0241,102928.86.55
LOD320×2001,0241,527670.56.21
LOD + skip background shading320×2001,0241,788572.66.21
LOD320×2004,0961,5742,602.515.24
LOD + skip background shading320×2004,0961,8672,194.415.24

Two warmups and eight synchronized samples per callable. Medians shown; raw data includes p95, samples, initialization time and runtime checks. Shared host conditions apply.

Qualified throughput and process GPU memory versus parallel world count
Same scene and physics workload. Download vector figure.

Visual match, measured before recommending

Eight wrist poses compared with the original renderer at 320×200. Full meshes exceed 99.99% silhouette overlap; simplified meshes exceed 99.5%. Shared Gaussian background error stays below 0.0003/255.

CandidateForeground IoUWhole-image mean absolute error
Full meshes99.9943–99.9977%0.984–1.508 / 255
Simplified meshes99.5256–99.6782%1.105–1.757 / 255
Four wrist poses: original on left, full-mesh GPU in middle, amplified difference on right
Full meshes: original → GPU → absolute RGB difference amplified 6×.
Inspect the simplified-mesh comparison
Original versus simplified-mesh GPU render and amplified difference
LOD introduces small occlusion differences; collision shapes are unchanged.

At 500 matched physics steps, arm error was 1.94e-6 radians and bottle position error was 0.233 mm. Changing one world leaves other images unchanged; resetting it restores its image exactly.

64 worlds in motion

8×8 grid · 128×128 RGB per world · scripted joint targets. Playback follows 20 Hz simulated time, not measured wall-clock throughput. Readback and encoding are excluded from benchmarks.

Independent simulator copy

Assets, Gaussians, calibration, simulator code, packages and the CUDA build toolchain live in ~/GSim, beside ~/rl_training. Runtime does not read the old project folders.

Six reference frames match pixel for pixel; qpos/qvel are identical. All 163 XML asset paths resolve locally. MJLab's tracked source is unchanged.

Source hashes and package versions
Original calibrated wrist-camera reference frame
Original hybrid reference · 155,449 composed Gaussians.

What remains different

GPU-resident bulk data, with small host synchronizations. State, transforms and images stay on CUDA during stepping, but gsplat performs three scalar device-to-host reads per render chunk. Initialization is CPU-assisted.

Contact equivalence is incomplete. The short trajectory passes, but CPU MuJoCo produces five bottle contacts where this Warp path produces one per world. Long grasps and task success still need qualification.

A benchmark adapter, not an RL task yet. It accepts joint actuator targets. Cartesian IK, rewards, termination, policy inference, heterogeneous scenes and large-scale randomization are outside the measured scope.

ManiSkill / SAPIEN: published reference only

ManiSkill/SAPIEN was not installed or run. These official published results use one 128×128 RGB camera and an RTX 4090.

Published taskWorldsEnv steps/sReported GPU GiB
PickCube-v1646,6731.90
PickCube-v125612,9042.20
PickCube-v11,02417,4723.08
CartpoleBalanceBenchmark-v16421,8891.77
CartpoleBalanceBenchmark-v125630,3442.01
CartpoleBalanceBenchmark-v11,02432,1382.88

These tasks use two physics steps per image, versus GSim's 50. Hardware, scene, robot, contacts and renderer differ. Equal-compute parity has not been demonstrated. See the official methodology and full report.

All pipeline attempts and timing ablations

Historical no-shadow, one-substep and experimental runs are not equivalent to the qualified rows. Completed timing means runtime checks passed; visual quality is assessed separately.

Run identifierRecorded outcome
graph_128x128_n10243,219 env steps/s
graph_128x128_n2562,394 env steps/s
graph_128x128_n40963,526 env steps/s
graph_128x128_n641,262 env steps/s
graph_256x256_n10241,606 env steps/s
graph_256x256_n2561,377 env steps/s
graph_256x256_n40961,672 env steps/s
graph_256x256_n64913 env steps/s
graph_320x200_n10241,534 env steps/s
graph_320x200_n2561,322 env steps/s
graph_320x200_n40961,594 env steps/s
graph_320x200_n64890 env steps/s
hybrid_n136 env steps/s
hybrid_n10241,544 env steps/s
hybrid_n2561,327 env steps/s
hybrid_n40961,599 env steps/s
hybrid_n64905 env steps/s
lod_128x128_n10244,139 env steps/s
lod_128x128_n40964,705 env steps/s
lod_128x128_n81924,764 env steps/s
lod_320x200_n10242,159 env steps/s
lod_320x200_n40962,272 env steps/s
lod_320x200_n641,089 env steps/s
mugs_n1Error: The expanded size of the tensor (3) must match the existing size (2) at non-singleton dimension 3. Target sizes: [-1, -1, -1, 3]. Tensor sizes: [1, 200, 320, 2]
native_n138 env steps/s
native_n10242,080 env steps/s
native_n2561,703 env steps/s
native_n40962,172 env steps/s
native_n641,045 env steps/s
one_step_128_n10243,727 env steps/s
one_step_128_n40963,772 env steps/s
qualified_full_320x200_n10241,102 env steps/s
qualified_full_320x200_n64729 env steps/s
qualified_lod_128x128_n10242,896 env steps/s
qualified_lod_128x128_n2562,241 env steps/s
qualified_lod_128x128_n641,245 env steps/s
qualified_lod_256x256_n10241,357 env steps/s
qualified_lod_320x200_n10241,527 env steps/s
qualified_lod_320x200_n2561,326 env steps/s
qualified_lod_320x200_n40961,574 env steps/s
qualified_lod_320x200_n64905 env steps/s
qualified_skipbg_128x128_n10243,256 env steps/s
qualified_skipbg_128x128_n163843,613 env steps/s
qualified_skipbg_128x128_n2562,424 env steps/s
qualified_skipbg_128x128_n40963,573 env steps/s
qualified_skipbg_128x128_n641,283 env steps/s
qualified_skipbg_128x128_n81923,611 env steps/s
qualified_skipbg_320x200_n10241,788 env steps/s
qualified_skipbg_320x200_n40961,867 env steps/s
qualified_skipbg_320x200_n64982 env steps/s
raster_lod_n134 env steps/s
raster_lod_n64915 env steps/s
rejected_aspect_qualified_lod_128x128_n10243,225 env steps/s
rejected_aspect_qualified_lod_128x128_n163843,575 env steps/s
rejected_aspect_qualified_lod_128x128_n2562,406 env steps/s
rejected_aspect_qualified_lod_128x128_n40963,547 env steps/s
rejected_aspect_qualified_lod_128x128_n641,283 env steps/s
rejected_aspect_qualified_lod_128x128_n81923,576 env steps/s
rejected_aspect_qualified_lod_256x256_n10241,583 env steps/s