# superdex_gs

Standalone simulator for the UR5 CB3 + OnRobot RG2 + **RealSense D415** bottle-picking setup:
**SuperDex** contact-first physics (the engine in which state-only PPO learned this pick) with the
calibrated **Gaussian-splat** wrist RGB pipeline of mjlab_gs. It is to SuperDex what mjlab_gs is to
MJLab: one environment serves privileged-state teacher RL and image-based student training, with all
domain randomization from policy_training, mjlab_gs and rl_training.

```bash
cd ~/superdex_gs
source env.sh
superdex-gs contract --task visual                 # full simulator contract (json)
superdex-gs smoke --task visual --envs 4 --dr visual --scale 1
superdex-gs pick --envs 48 --preset teacher --scale 0.6 --output reports/pick/mine.json
superdex-gs benchmark --task state --envs 64 256 --output reports/benchmark/mine.json
superdex-gs record --envs 4 --preset visual --scale 1 --seconds 12 --output reports/videos/mine
superdex-gs physics-checks
python -m pytest -q tests
```

```python
from superdex_gs.env import SuperDexGSEnv, EnvConfig
from superdex_gs.randomization import DRConfig
import numpy as np

env = SuperDexGSEnv(EnvConfig(num_envs=64, task="visual", dr=DRConfig.preset("visual", scale=0.3)))
obs, _ = env.reset()             # obs: state [64,43], proprio [64,16], goal [64,3], critic [64,61], rgb CUDA uint8 [64,180,320,3]
action = np.zeros((64, 7)); action[:, 6] = 1          # [dx,dy,dz,rx,ry,rz,gripper] in [-1,1], base frame
obs, reward, terminated, truncated, info = env.step(action)   # auto-resets finished episodes; info["final"] has episode results
env.close()
```

## Architecture

| Layer | Implementation |
|---|---|
| Scene | `scene/scene.xml` (MuJoCo MJCF, single source of truth): mjlab_gs calibrated room/table/pedestal/UR5/RG2/bottle with the D455 replaced by the D415 on mount v4 (`scripts/build_scene.py`) |
| Physics | `model.py` converts the MJCF robot into a SuperDex articulation (FK parity 3.4e-7 m); `world.py` builds one scene (RG2 parallelograms closed with cycle joints, SDF bottle, static tessellated table slab, cabinet, pedestal, floor) |
| Parallelism | `sim_core.ChunkSim` runs complete 20 Hz ticks (Cartesian IK + gravity compensation at 125 Hz + 25 physics substeps) for a block of worlds; `backends.py` runs blocks in worker **processes** with shared-memory state (default) or threads |
| Task | `env.py`: DR sampling, resets, 43/16/61-D observations, rewards, strict grasp-and-hold success, auto-reset |
| Rendering | `render/camera.py`: render-only mujoco_warp model fed with SuperDex poses; mjlab_gs's calibrated Warp foreground (`compat.py`, shadows, textures) composited over the gsplat room (155k Gaussians, pedestal field moves with the base); captured as a CUDA graph; per-world camera/light/material/texture/background/image randomization |
| Randomization | `randomization.py`: 11 families with independent random streams, presets `nominal`/`teacher`/`visual` — see [docs/RANDOMIZATION.md](docs/RANDOMIZATION.md) |
| Validation | `scripts/physics_checks.py`, `scripts/check_fk.py`, `scripts/validate_pick.py`, `tests/` |
| Measurement | `scripts/benchmark.py`; full evidence pipeline `scripts/run_all.sh` (~2.5 h) then `site_publish/publish.py` + `verify.py` |

Task/contract details: [docs/TASK.md](docs/TASK.md). Measured results: [reports/REPORT.md](reports/REPORT.md) and https://real2sim.sudhirpratapyadav.com/superdex-gs/.

## Workspace

- `assets/`: hard-linked copies of the mjlab_gs Gaussian fields, CAD and scans, the D415 mesh, calibration
  and mount v4 files from `~/extra/real2sim`, and the SuperDex bottle collision surface. SHA-256 in
  `assets-manifest.json` (`python scripts/assets_manifest.py --verify`). No runtime dependency on other projects.
- `.venv/` (Python 3.12, `requirements.lock.txt`): SuperDex 1.0.0, torch 2.8 cu128, warp 1.14, mujoco / mujoco_warp 3.11, gsplat 1.5.3.
- `toolchains/cuda` (hard-linked CUDA toolchain) and `torch_extensions/` (gsplat built for this Python).
- `site_publish/`: website builder and publisher.

Recreate: `uv venv --python 3.12 .venv && uv pip sync --python .venv/bin/python requirements.lock.txt --extra-index-url https://download.pytorch.org/whl/cu128 --index-strategy unsafe-best-match`, then `source env.sh`. First visual run compiles the Warp render kernel (~2.5 min) and gsplat (~2.5 min) once.