# GSim simulator benchmark report

Measured 12 September 2026 on one NVIDIA RTX A6000 (48 GiB), using the reconstructed UR5 / RG2 / D455 bottle scene. No RL training was performed.

## Result and recommendation

Use the external MJLab / MuJoCo Warp adapter, calibrated degree-three gsplat rendering, packed Gaussian-camera intermediates, and chunks of 128 cameras. Enable the local reference-rendering compatibility module and shadows. Use simplified room/base visual meshes when the measured silhouette tolerance is acceptable; retain full meshes for the closest match. Collision geometry and physical parameters are preserved.

The highest measured qualified throughput was **3,613 aggregate environment steps/s** at **16,384 worlds, 128×128 RGB**, using **27.29 GiB sampled process GPU memory**. Batch latency was **4,534.76 ms**. At the original 320×200 resolution, the best measured result was **1,867 steps/s**, 4,096 worlds and 15.24 GiB. These rates include **50 physics steps at 1 ms plus fresh RGB** per observation.

This is a working, independently copied simulator benchmark platform with a visually qualified GPU renderer. It does **not establish performance parity with ManiSkill on equal compute**. It also does not yet establish full grasp/contact equivalence or eliminate every CPU scalar synchronization.

For the next RL integration, **1,024 worlds at 128×128** is a practical starting point: 3,256 steps/s, 5.18 GiB and 314.5 ms batch latency, retaining 90.1% of the highest measured throughput. Larger batches consume substantially more memory for only a small rate gain, leaving less space for observations and policy training.

## Independent workspace

`~/GSim` is a sibling of `~/rl_training`. It contains copied simulator code, MJCF, meshes, textures, camera calibration, Gaussian checkpoints, Python packages, and a local CUDA compilation toolchain. Runtime code does not read `~/real2sim` or `~/policy_training`. All 163 XML asset paths resolve inside GSim. Six original snapshot frames, including a randomized reset and short action sequence, match **pixel for pixel**, with identical CPU qpos/qvel. A file-access trace and imported-module audit passed. Historical source paths appear only in import/provenance records, not runtime dependencies.

The uv-managed Python interpreter, operating-system libraries, and NVIDIA driver remain ordinary system dependencies. Third-party source checkouts are local; no tracked files in MJLab, MuGS, or nvdiffrast were modified. The compatibility renderer is an explicitly patched external copy of the Apache-licensed MuJoCo Warp renderer.

## Qualified pipeline measurements

Every row advances 50 × 1 ms physics steps and renders one RGB camera per world. Shadows, SH degree 3, calibrated intrinsics/extrinsics, and fresh images are enabled. World count is independent state count. Rates are aggregate throughput, **not per-world frame rate**. Table memory includes this process's Warp and Torch allocations, sampled by NVML every 50 ms; short peaks may be missed. Initialization/JIT time is recorded separately in raw JSON.

| Renderer | RGB size | Worlds | Env steps/s | Median batch ms | p95 batch ms | Process GiB |
|---|---:|---:|---:|---:|---:|---:|
| LOD | 128×128 | 64 | 1,245 | 51.40 | 51.58 | 2.18 |
| LOD + background shading skipped | 128×128 | 64 | 1,283 | 49.90 | 50.03 | 2.18 |
| LOD | 128×128 | 256 | 2,241 | 114.23 | 114.42 | 4.40 |
| LOD + background shading skipped | 128×128 | 256 | 2,424 | 105.62 | 105.95 | 4.40 |
| LOD | 128×128 | 1,024 | 2,896 | 353.62 | 353.94 | 5.18 |
| LOD + background shading skipped | 128×128 | 1,024 | 3,256 | 314.49 | 314.91 | 5.18 |
| LOD + background shading skipped | 128×128 | 4,096 | 3,573 | 1,146.50 | 1,147.02 | 9.04 |
| LOD + background shading skipped | 128×128 | 8,192 | 3,611 | 2,268.58 | 2,274.40 | 15.17 |
| LOD + background shading skipped | 128×128 | 16,384 | 3,613 | 4,534.76 | 4,537.93 | 27.29 |
| LOD | 256×256 | 1,024 | 1,357 | 754.37 | 754.64 | 5.75 |
| Full meshes | 320×200 | 64 | 729 | 87.84 | 87.93 | 2.58 |
| LOD | 320×200 | 64 | 905 | 70.74 | 70.83 | 2.30 |
| LOD + background shading skipped | 320×200 | 64 | 982 | 65.20 | 65.30 | 2.30 |
| LOD | 320×200 | 256 | 1,326 | 193.10 | 193.79 | 4.63 |
| Full meshes | 320×200 | 1,024 | 1,102 | 928.82 | 936.76 | 6.55 |
| LOD | 320×200 | 1,024 | 1,527 | 670.54 | 672.98 | 6.21 |
| LOD + background shading skipped | 320×200 | 1,024 | 1,788 | 572.60 | 573.27 | 6.21 |
| LOD | 320×200 | 4,096 | 1,574 | 2,602.50 | 2,604.19 | 15.24 |
| LOD + background shading skipped | 320×200 | 4,096 | 1,867 | 2,194.43 | 2,195.14 | 15.24 |

![Qualified throughput and memory](throughput.png)

The matched single-world copied baseline (CPU MuJoCo + EGL foreground + Gaussian rendering, held joint targets, 50 substeps) measured **10.35 steps/s**, with **96.60 ms** batch latency. It returns CPU RGB as in the original interface; GPU candidates return CUDA RGB. This comparison excludes Cartesian IK, rewards and policy inference on both sides.

## What improved performance, and what failed

1. **Packed Gaussian-camera intermediates and chunking:** at 4,096 views / 320×200, packed unchunked Gaussian rendering measured about 6,488 images/s and 38,962 MiB Torch allocation. Chunks of 128 measured about 6,437 images/s and 2,152 MiB: approximately **18× lower renderer memory** for similar throughput. This is renderer-only memory, not full-simulator memory. Packed vs dense maximum floating RGB difference was 0.000714/255 across the checked views, consistent with numerical roundoff.
2. **Simplified background visibility meshes:** room triangles decreased from 1,038,287 to 46,954, and base triangles from 180,000 to 11,886. These meshes still determine visibility/shadows, while Gaussians supply their displayed colors. Collision geometry, masses, inertia, joints, and actuators remain unchanged. This introduces small measured occlusion errors; full geometry remains available.
3. **Reference-compatible ray rendering:** raw Warp rendering did not match the original near-plane clipping or textured lighting. The external compatibility module restores primary-ray near clipping, clamps lighting before texture modulation, and separates the shadow-light additive pass. Square-image tests also exposed upstream sensor cropping on aspect-ratio changes. The adapter now constructs rays from the full calibrated intrinsics, matching the reference GL frustum and Gaussian camera. Earlier square measurements were rejected and replaced. Without these fixes, faster raw measurements failed visual qualification. The lighting fix is scoped to this scene's single shadow-casting light and active headlight; additional lighting setups need validation.
4. **Skip discarded background shading:** retain primary-ray visibility and shadow occluders, but stop shading a hit whose mesh color is replaced by Gaussians. This is a separately measured flag, `--skip-background`; qualification is recorded at 320×200, 128×128 and 256×256. The table shows its measured benefit rather than assuming one.
5. **Physics graph over the observation interval:** capture all 50 steps in one Warp graph. Large-batch throughput changed little in initial tests because rendering dominated. It remains a reproducible option rather than a claimed major speedup.
6. **Stock MuGS:** tested the copied integration with the same Gaussian asset in its expected in-memory format. It failed on current typed segmentation: `[N,H,W,2]` was expanded as if it were a one-channel mask. Its default camera/SH-loading assumptions also differ from this calibrated scene. No valid stock MuGS throughput is claimed.
7. **nvdiffrast foreground prototype:** implemented and tested CUDA rasterization with vertex-baked textures. At 64 worlds it measured about 915 steps/s, but the coarse table geometry lost texture detail and failed the visual gate. This prototype is retained for inspection, not recommended.
8. **Lossy Gaussian shortcuts:** SH degree 0 gave only a small speed gain, with MAE about 7.38/255 and PSNR 26.85 dB in the Gaussian-only check. Radius clipping at 2 pixels produced localized error up to 93.94/255 without useful speedup. Both are rejected for the recommended configuration.

The Gaussian field alone reaches roughly 6.4k 320×200 images/s on this GPU; foreground visibility/shadows add substantial work. The measured improvements preserve full SH appearance rather than trading it away. Initial no-shadow and one-substep results remain in raw data but are not advertised as equivalent qualified throughput.

## Visual and dynamics evidence

Eight distinct wrist poses were compared against the copied original renderer at 320×200:

| Candidate | Foreground IoU range | Whole RGB MAE, 0–255 scale | Shared background MAE |
|---|---:|---:|---:|
| Full geometry, compatibility + shadows | 99.9943–99.9977% | 0.984–1.508 | <0.0003/255 |
| Simplified visual meshes, compatibility + shadows | 99.5256–99.6782% | 1.105–1.757 | <0.0003/255 |

![Original / full-mesh GPU / absolute difference amplified 6×](gpu_visual_comparison_shadows_compat.png)

![Original / simplified-mesh GPU / absolute difference amplified 6×](gpu_visual_comparison_lod_shadows_compat.png)

The gates were set before the renderer fixes: IoU ≥0.97, background MAE <0.1/255, whole RGB MAE <8/255. The corrected configurations pass. This verifies the tested views and resolutions, not arbitrary unseen scenes. The 64-world video shows independently phased scripted joint motion at 20 Hz simulated-time playback; it is not a learned policy or wall-clock performance demonstration.

After 500 matched physics steps including gripper actuation, maximum arm error was **1.94e-6 rad** and bottle position error **0.233 mm**. Single-world motion changes only its image; subset reset restores its image exactly and leaves other worlds unchanged. All successful benchmark runs checked finite state and endpoint contact/constraint capacity.

There is a material contact-model difference: the reference has five bottle contacts where the Warp path has one per world, owing to its supported contact generation for these shapes. The short trajectory still passes the numerical gate. Long grasp trajectories, contact-rich rewards and task success must be validated before treating it as a drop-in RL task environment.

## GPU residency and remaining limits

Physics state, camera transforms, Gaussian parameters, foreground images and compositing remain on CUDA while stepping. Initialization uses CPU compilation and asset loading. A profiled step found **three device-to-host scalar reads inside gsplat per render chunk**; there are no full image/state readbacks in the simulator step. Chunking repeats those scalar reads, so this is not a completely CPU-free or fully capturable visual pipeline. Screenshots, validation and video encoding deliberately read images back outside timing runs.

The current benchmark exposes held joint actuator targets, not the original Cartesian IK action interface, rewards, termination or a training loop. It uses one shared Gaussian scene and small pose variations. Multiple heterogeneous scenes, RGB-D, domain randomization at large batch sizes, broad camera coverage and long-horizon contact stability are outside the measured scope.

## ManiSkill / SAPIEN published reference

ManiSkill/SAPIEN was **not installed or benchmarked**. The [official benchmark documentation](https://maniskill.readthedocs.io/en/latest/user_guide/additional_resources/performance_benchmarking.html) describes SAPIEN rasterization and notes the sensitivity to renderer and scene settings. Its [published RTX 4090 CSV](https://github.com/mani-skill/ManiSkill/blob/main/docs/source/user_guide/additional_resources/benchmarking_results/rtx_4090/maniskill.csv) contains these one-camera, 128×128 RGB results:

| Published task | Worlds | Env steps/s | Reported GPU GiB |
|---|---:|---:|---:|
| PickCube-v1 | 64 | 6,673 | 1.90 |
| PickCube-v1 | 256 | 12,904 | 2.20 |
| PickCube-v1 | 1,024 | 17,472 | 3.08 |
| CartpoleBalanceBenchmark-v1 | 64 | 21,889 | 1.77 |
| CartpoleBalanceBenchmark-v1 | 256 | 30,344 | 2.01 |
| CartpoleBalanceBenchmark-v1 | 1,024 | 32,138 | 2.88 |

The [published script](https://github.com/mani-skill/ManiSkill/blob/main/mani_skill/examples/benchmarking/scripts/maniskill.sh) sets PickCube simulation/control rates to 100/50 Hz; Cartpole uses 120/60 Hz defaults. Both imply two physics steps per image, versus GSim's 50. The GPU, robot, scene, collision model, appearance, and renderer differ. We do not normalize by GPU specification or claim an apples-to-apples speed ratio. The published rates remain higher reference targets; equal-compute parity has not been demonstrated.

## Reproduce and inspect

```bash
cd ~/GSim
source env.sh
CUDA_VISIBLE_DEVICES=0 python benchmarks/verify_copy.py
CUDA_VISIBLE_DEVICES=0 python benchmarks/validate_gpu.py --lod --shadows --compat --skip-background
CUDA_VISIBLE_DEVICES=0 python benchmarks/pipeline.py --mode hybrid --lod --shadows --compat \
  --skip-background --block-graph --chunk 128 --n 1024 --width 128 --height 128 \
  --output results/my_run.json
```

Remove `--lod` to retain full visibility meshes. For the closest validated match use full meshes with `--compat --shadows`; lossy SH/radius changes are off. `benchmarks/run_qualified.py` and `benchmarks/run_final_checks.py` record the final sweeps sequentially. `scripts/build_report.py` rebuilds this report from JSON. Timing uses two warmups and eight synchronized samples per pipeline callable; median and p95 are descriptive statistics, not confidence intervals. GPU 0 also had an unrelated resident allocation of about 4.5 GiB during the run; reported process memory excludes it. GPU 1 was initially busy and was not used. No clocks or power settings were changed.

Package versions: torch 2.8.0+cu128, mujoco 3.11.0, mujoco-warp 3.11.0, warp-lang 1.14.0, mjlab 1.6.0, gsplat 1.5.3, nvdiffrast 0.4.0. Complete package pins are in `requirements-lock.txt`.

Source commits:

- mjlab: `8ee51fbcf806a7419189f706d9e394cbeb7790fa`
- MuGS: `df1203176cf34c55aa2bb7269411f83484ce3905`
- nvdiffrast: `253ac4fcea7de5f396371124af597e6cc957bfae`

Raw data: `results/*.json`; validation: `reports/gpu_validation*.json`; provenance: `reports/import_manifest.json`, `reports/render_compat_source.json`, and `reports/reproducibility.json`. Methods: `reports/METHODS.md`. Published dashboard: https://real2sim.sudhirpratapyadav.com/gsim/ . Publication uses croc through `untu_vps`, SHA-256 verification, backups, and browser checks on the dedicated page.