# Benchmark methods

## Workload

The scene has 18 generalized velocities, 172 geoms, 148 mesh assets, and 155,449
Gaussians after composing the interaction scene. The original visible geometry
has 1,412,157 triangles; 1,218,287 belong to the Gaussian-replaced room and base.
Each GPU world has independent physics state and a wrist camera. The room field
is shared across worlds. These experiments use one scene/layout and small
variations of robot pose, not heterogeneous Gaussian scenes or a trained policy.

Main pipeline measurements advance 50 physics steps with a 1 ms timestep and
produce one fresh RGB observation per world. Joint actuator targets are held
through the observation interval. CPU initialization, asset loading, JIT
compilation, Cartesian IK, policy inference, reward computation, and image
encoding are excluded from steady-state timing and identified separately.

## Timing and memory

Use a single benchmark process on physical GPU 0. GPU 1 has an unrelated
workload. Warm up each callable, synchronize CUDA, time with `perf_counter`,
and synchronize at the end of every sample. Pipeline rows retain eight timing
samples; Gaussian ablations retain five samples summarized by median and p95.
These are short engineering measurements, not statistical confidence intervals.
The large batch sweeps take longer in wall time because each sample renders
thousands of complete images.

Final qualified runs enable shadows and the external compatibility renderer.
Its primary-ray near plane and fixed-function texture/lighting order match the
reference scene's single shadow-casting light and active headlight. The optional
background-shading early return preserves primary visibility and shadow
occluders. Simplified meshes are rendering assets only; model extent is held
fixed so near clipping does not shift when their bounds change.

At square resolutions, upstream Warp crops the sensor to the new aspect ratio.
The original renderer and Gaussian camera scale the full calibrated sensor.
The final adapter overrides the precomputed rays from the same intrinsics;
earlier square results are retained with a `rejected_aspect_` filename prefix.
Qualified square runs record `camera_rays=full_calibrated_sensor` and pass a
separate visual check at their resolution.

Aggregate env-FPS is worlds divided by batch latency. It is not the frame rate
of each individual world. Memory is reported as Torch allocated/reserved memory,
NVML process memory sampled at 50 ms intervals, and separately whole-device
occupancy. NVML samples can miss very short peaks. Original early microbenchmarks
have Torch-only memory measurements; those are not whole-simulator memory.

## Correctness

1. Copy verification compares six frames from nominal/randomized resets and
   short action sequences against an original-state snapshot. Images and
   double-precision CPU states must match exactly. XML assets are checked for
   local resolution; a file-access trace checks source-folder independence.
2. GPU checks compare eight different FK wrist views with the copied original
   renderer, separate background error from foreground IoU/color error, and
   save contact sheets. Provisional visual gates: foreground IoU ≥0.97,
   background MAE <0.1/255, and whole-image MAE <8/255. Failures remain reported.
3. Compare a 500-step GPU/CPU trajectory with identical actuator inputs,
   including gripper actuation. Gates: arm error <0.001 rad and bottle position
   error <5 mm. This is a short numerical check, not full grasp validation.
4. Check that changing one world's camera state changes its image, leaves other
   images unchanged, and a subset reset restores the affected image. Check
   finite state and contact/constraint capacity at benchmark endpoints.
5. Profile a GPU step separately from timing. Validation readbacks, screenshots,
   and encoding are outside the profiled simulator path.

## External comparison

Read-only reference: [ManiSkill benchmark documentation](https://maniskill.readthedocs.io/en/latest/user_guide/additional_resources/performance_benchmarking.html)
and [published RTX 4090 CSV](https://github.com/mani-skill/ManiSkill/blob/main/docs/source/user_guide/additional_resources/benchmarking_results/rtx_4090/maniskill.csv).
Only reference documents/data were downloaded. ManiSkill/SAPIEN was not installed
or executed.

At 1×128×128 RGB and 1,024 worlds, the CSV reports 17,472 env-FPS for PickCube and
32,138 for Cartpole. The supplied script uses 100 Hz simulation / 50 Hz control
for PickCube, and defaults to 120/60 for Cartpole: two physics steps per visual
observation. GSim's main workload uses 50. Robot/scene complexity, collision
model, renderer, action distribution, and GPU differ. These are reference
numbers, not evidence of equal-compute parity or a justified GPU-normalized
speed ratio.

MJLab source: https://github.com/mujocolab/mjlab

MuGS source: https://github.com/Renforce-Dynamics/MuGS

gsplat source: https://github.com/nerfstudio-project/gsplat

nvdiffrast source: https://github.com/NVlabs/nvdiffrast