# SuperDex: YAM bottle picking

Independent, physics-only experiment using [Project SuperDex](https://github.com/facebookresearch/project_superdex), the YAM arm, and the actual bottle triangle mesh. No 3D Gaussian splatting or dependency on another simulator workspace at runtime.

**Learned result:** PPO checkpoint 70 (286,720 environment steps) passed 8/8 development episodes and then 32/32 fresh-seed episodes. The actor uses 43 state values and no images, choosing Cartesian motion and gripper opening. See [the RL instructions](rl/README.md) and [published videos](https://real2sim.sudhirpratapyadav.com/sims/#superdex-rl). This result covers upright bottles with ±1 cm XY randomization and initial gripper XY variation of ±5 mm in this simulator.

**Scripted reference:** approach, grasp, lift, and two-second hold succeed at the nominal bottle position. The bottle bottom reaches 15.5 cm above the floor and remains about 15.3 cm high at the end of the hold. Opening the gripper then releases the bottle; it falls and settles on the ground. Learned-policy results are documented in the RL section below.

## Run

```bash
cd sims/superdex
.venv/bin/python scripts/run_pick.py --mode rest --seconds 2 --output results/my_rest
.venv/bin/python scripts/run_pick.py --output results/my_pick
.venv/bin/python scripts/run_pick.py --release --seconds 15 --output results/my_release
EGL_DEVICE_ID=1 .venv/bin/python scripts/render_replay.py results/my_release
```

Each output directory must be new. Physics runs single-threaded on the CPU; the mesh replay uses EGL for rendering. `EGL_DEVICE_ID` selects a graphics device and can be changed. No GPU training is started. `--viewer` optionally connects the SuperDex desktop debugger and requires a working desktop display. `--record` optionally saves a native HDF5 physics recording; the tested replay path uses `trajectory.json`.

## Evidence

- `results/rest_004/summary.json`: upright rest with the final script, roughly 0.27 mm contact penetration; measured support 0.7841 N versus 0.7848 N weight.
- `results/pick_002/replay.mp4`: successful approach, grasp, lift, and hold, using recorded physics transforms and the copied visual meshes.
- `results/pick_release_003/`: successful lift/hold and release to ground; includes source snapshot, asset audit, configuration, trajectory, summary, and `replay.mp4`. `validation.json` confirms both fingers support the airborne bottle, there is no ground support during hold, and release restores ground support with zero finger force. No solver warnings or errors occurred.
- `results/pick_001/`: retained failed development probe with the finger opening direction reversed. It tipped the bottle during descent.
- `results/rest_001/`: explicitly invalidated development probe with an incorrectly packed inertia tensor; not evidence of stable simulation.

These scripted probes test the nominal position `[0.28, 0, 0.05144146]` m. The RL evaluations below randomize the initial position. Scripted-probe wall time includes robot collider baking and initialization.

## Asset and controller details

`assets/source/` contains independent copies of the YAM MJCF/STLs, bottle OBJ/MTL/textures, original bottle scene definition, and baseline configuration. `scripts/prepare_assets.py` recreates `assets/generated/` from these copies; it never reads a neighboring project. SHA-256 provenance is in `assets/generated/asset_audit.json`.

The 12 bottle material parts are joined at duplicated texture-seam vertices at 1e-8 m precision. All 768 original triangles remain, producing a closed 386-vertex surface. No convex hull, decomposition, or replacement bottle geometry is used. SuperDex bakes an SDF from this surface. The bottle keeps the source 80 g mass, local COM `[0, 0, -0.006]` m, and principal inertias `[7.2034708e-5, 7.2034708e-5, 1.3698342e-5]` kg m².

The [runtime URDF loader](upstream/superdex_robotics/examples/basic/example_urdf_loading.py) supports mesh colliders, so the YAM primitive colliders are tessellated and combined per moving link. Fixed child geometry is transformed into its parent link. Wrist/finger collision is enabled, matching the stock YAM task's gripper-only collision choice. Robot mesh textures are simplified in the replay; the bottle uses its original textures. No room scene is loaded.

There are six arm joints and two finger joints. The anchored base is 1 cm above the ground. Independent forward kinematics is checked against every imported link before stepping. Positive left-finger displacement opens the gripper: `(+0.0375, -0.0375)` is open and `(0, 0)` is closed.

The script uses bounded Cartesian IK and torque-limited implicit joint PD. Gravity remains enabled for the arm and bottle. It never attaches the bottle, teleports it after reset, or applies a lifting force directly to it. The final release checks ground support and zero finger force.

**Differences from the MJLab baseline:** PD gains are recorded explicitly and differ from MJLab; two symmetric finger drives replace the equality-coupled single actuator, splitting its force limit and reflected inertia. Contact-law parameters and collision discretization differ between engines. Source fingertip spheres share a link-level friction coefficient in SuperDex. These differences mean this is an initial simulator feasibility test, not an isolated engine benchmark.

## Recreate the environment

```bash
uv venv --python 3.12 .venv
uv pip sync --python .venv/bin/python requirements.lock.txt
.venv/bin/python scripts/prepare_assets.py
```

The lock installs the exact tested packages. Rendering uses PyOpenGL 3.1.10 because pyrender's old exact dependency on 3.1.0 fails with the installed NumPy; `uv pip sync` preserves the tested override. Physics uses the published SuperDex 1.0.0 wheels. `upstream/` is an unmodified shallow checkout of `stable`, pinned by commit in `provenance.json`, for examples and source inspection; it is not imported at runtime.

## RL training

The [state-only RL task](rl/README.md) uses SuperDex Lab's native Gymnasium
environment and Ray/RLlib PPO. It has 43 state observations and four Cartesian/
gripper actions; no images or scripted training actions. The Gymnasium checks,
action-interface pick test, optimizer smoke run, and checkpoint reload passed.

The active run is recorded in `results/rl/active_run.json`. Revision 1 was stopped
after two 0/8 development evaluations. Revision 2 uses scaled relative state,
explicit premature-closure feedback, and sixteen CPU physics workers.
Read the selected run's `status.json`, `progress.json`, and `latest_evaluation.json` for current
results. Training progress does not establish learned-policy pick success; use
the saved deterministic evaluations and separately labeled videos.

`results/rl/ppo_v2_seed42/qualification.json` selects checkpoint 70, which was
chosen on development performance before its 32-seed evaluation. The complete
evaluation is `heldout_00070/summary.json` under that run; the first two seeds have
recorded trajectories and rendered MP4 videos. Success requires the bottle bottom
above 10 cm for one second, both-finger support, and no floor support.