# superdex_gs task specification

Task `superdex-gs-bottle-pick-v1`, controller contract `superdex-gs-cartesian-v1`
(`superdex-gs contract` prints the full contract, including scene and asset hashes).

## Scene

Source of truth: `scene/scene.xml`, built by `scripts/build_scene.py` from the mjlab_gs
calibrated scene (`scene/source_mjlab_gs_scene_lod.xml`):

- UR5 CB3 (shiny CAD visuals, CAD collision meshes), mount pose from the calibrated pedestal.
- Legacy OnRobot RG2: six linkage hinges. SuperDex closes each parallelogram with a cycle joint
  (moment arm pin on the fingertip); two symmetric drives replace MuJoCo's single actuator plus
  five equality constraints (kp 12, kv 0.4, 3 N·m total, split equally).
- **RealSense D415 on mount v4** (from `~/extra/real2sim`): housing front centre (32.5, −50, 40) mm
  in the RG2 frame, 15° tilt; colour stream factory intrinsics (1280×720, fx 913.94, fy 912.14,
  cx 644.21, cy 347.87; serial 047422061197). The D455 body and the blue symmetric mount were removed.
  Checked: optical centre in the gripper frame equals `camera_metadata.json` to 2.5e-15 m.
- Bottle: 80 g, COM (0, 0, −6 mm), scanned closed surface (386 vertices, 768 triangles) as a SuperDex
  SDF collider; 12 textured scan parts for appearance.
- Table: 4 cm tabletop slab tessellated to ≤3 cm triangles (SuperDex samples contact on the colliding surface; the 12-triangle box let bottles sink or fall through away from its vertices) + cabinet (≤5 cm), pedestal column/flange hull, floor plane at −2 cm.

Articulation conversion (`model.py`): jointless MuJoCo bodies merge into their jointed ancestor
(inertias combined), every collision geom becomes a welded sub-link with MuJoCo's convex hull, and
hinge frames are rotated so the axis is +z (SuperDex limits are z components). FK parity against
MuJoCo: 3.4e-7 m over 20 random configurations (`reports/fk_parity.json`).

Friction: SuperDex mixes Coulomb coefficients by geometric mean (MuJoCo: max). Per-body values
(bottle 0.6, pads 1.667, table/floor 0.704) reproduce the MuJoCo pair values pad–bottle 1.0 and
bottle–table 0.65.

## Timing and control

| Item | Value |
|---|---|
| Physics | SuperDex 1.0.0 implicit solver, dt 2 ms (1 ms supported) |
| Policy / observation / camera | 20 Hz (25 substeps) |
| Joint-target (IK) updates | every 4 substeps = 8 ms (125 Hz, UR5 servo rate); 1–8 configurable |
| Arm drives | effective kp 10000/2500 N·m/rad, kd 400/100, effort ±150/±28 N·m (mjlab_gs runtime ×5 gains) |
| Gravity compensation | analytic gravity torque / kp added to targets (mjlab_gs bias compensation) |

Action (7, normalized to [−1, 1], robot-base frame): `[dx, dy, dz, rx, ry, rz, gripper]`.
Translation and rotation vectors are each norm-clamped to 1 and scaled to 5 mm and 1° per tick.
The desired TCP pose is latched once per tick from the measured TCP. Every IK period:
damped least squares (λ 0.005, orientation weight 0.2), per-joint step clipped to 1 rad/s × period,
plus gravity torque / kp. Gripper linkage target `(1 − g) · 0.6075` rad (−1 closed, +1 open, 0 half).

SuperDex drive convention (measured, `scripts/probe_gains.py`): tracking force = 2·stiffness·error
+ damping·velocity error, so stiffness is set to kp/2 (verified effective kp 2499.9 for kp 2500).

## Observations

| Group | Dim | Contents |
|---|---|---|
| `state` (teacher) | 43 | joint positions relative to nominal (12: arm + RG2 linkage) · joint velocities ×0.1 (12) · TCP→bottle in base frame ×5 (3) · bottle→goal ×5 (3) · previous action (7) · bottle up-axis in base frame (3) · pad contact forces ×0.1 (2) · hold progress (1) |
| `proprio` (student) | 16 | arm joints (6) · pad separation / 0.1008 m · TCP position (base) · first two TCP rotation columns |
| `goal` | 3 | commanded position in the base frame (nominal TCP ± 5/5/3 cm × DR scale) |
| `rgb` | [N,180,320,3] uint8 CUDA | D415 wrist image: Warp mesh foreground (shadows, textures) over the gsplat Gaussian room |
| `critic` | 61 | state (43) · bottle position/quaternion/linear/angular velocity (13) · table XY (2) · DR scale, friction factor, bottle-mass factor (3) |

The teacher's noise (training DR family `observation_noise`): joints ±0.01 rad, velocities ±0.05 rad/s, relative positions ±2 mm.

## Success, rewards, termination

Strict predicate (mjlab_gs): lowest bottle mesh point ≥ 10 cm above the tabletop, both pads in contact
with ≥ 0.05 N, bottle |v| < 0.15 m/s and |ω| < 1 rad/s, finite state; success = qualifying hold ≥ 2 s.
The monitor runs at 20 Hz (mjlab_gs runs it every 1 ms substep).

Reward (×dt): native MJLab `lift` (staged reach→bring) and `lift_precise`, bilateral contact 0.5,
clearance up to 2.0 while bilaterally grasped, action-rate −0.01, arm joint-speed hinge −0.01;
+10 once on success. Episodes last 15 s; terminate on non-finite/diverged state or the bottle
falling 19 cm below the tabletop (success does not terminate by default).

## Memory

Each physics worker process holds about 1.1 GiB (SuperDex scene with baked link colliders, MuJoCo model, kinematics caches):
a 56-worker environment uses ≈60 GiB of unique memory. Budget concurrent environments (training + validation + tools) against
system RAM; on the 187 GiB development machine, three concurrent 56-worker environments were OOM-killed.