# Bottle lift tasks, version 5 `Mjlab-GS-Bottle-UR5` (visual) and `Mjlab-GS-Bottle-UR5-State` (state teacher) share physics, actions, rewards and goals for BC/RL, recording, viewer and evaluation. This change configures the task; it does **not** train a policy or establish real grasp success. ## Policy interface and initial state The visual actor receives 16 proprioceptive float32 values, a separate 3-value `goal` position in robot-base coordinates, plus wrist RGB at **320×200** (`[N,3,200,320]`) float32 in `[0,1]`, all on CUDA. State order is six arm angles in radians, pad separation divided by 0.1008 m, TCP position in robot-base frame, and the first two TCP rotation columns in that frame, column-major. Seven normalized actions are `[dx,dy,dz,rx,ry,rz,gripper]`. Translation and rotation vectors are separately norm-clamped to one, then scaled to 5 mm and 1 degree per control step in the robot-base frame. Gripper -1 closes and +1 opens. Physics is 1 kHz; control and RGB are 20 Hz. This shares the legacy IO units, but the GPU differential IK controller is not trajectory-identical to the old CPU iterative IK controller. The servo now adds `qfrc_bias / actuator_gain` to its arm position targets, matching the old bias-compensation mechanism while retaining XML actuator force limits. This prevents neutral-action gravity sag. The simulator contract is **mjlab-gs-cartesian-v3**: goal conditioning is now explicit. The controller and action units are unchanged from v2, but old actor checkpoints cannot silently load because their inputs lack the new goal. The visual privileged critic has **62** values, in order: actor state (16), arm velocity (6), bottle world position/quaternion/linear/angular velocity (13), table world position/quaternion (7), robot-base world position/quaternion (7), camera local position/quaternion (7), bottle mass (1), episode DR scale (1), current qualifying hold seconds (1), bottle-to-goal world displacement (3). Quaternions are wxyz. An old 22- or 59-value critic checkpoint cannot be used for a full training resume. Actor-only loading within the current simulator contract allows evaluation under different task settings. State-teacher checkpoints use a distinct observation contract and cannot be loaded directly into a visual actor. Reset starts from copied `scene/initial_state.json`, with zero joint/root velocities and a nominally open gripper. The bottle starts upright with its bottom on the tabletop (tabletop z=0.788531178 m plus table translation). The nominal bottle center is table-local x=0.18, y=-0.13 m. Reset then applies the coupled distributions below. There is no hidden IK search or settling trajectory. ## Randomization and curriculum Settings are dataclasses in `src/mjlab_gs/tasks/settings.py`. JSON presets live in `configs/`. Every random draw is uniform unless stated otherwise. Parameters are sampled independently per world at episode reset and held for that episode; image noise is freshly sampled per rendered frame. | Family | Full-scale endpoint | |---|---| | Bottle XY | Table-local x ±0.339932 m, y ±0.182290 m; 5 cm edge margin | | Bottle yaw | ±180° relative to randomized table | | Table XY / yaw | ±30 cm per axis / ±30° | | Robot + pedestal XY / yaw | ±20 cm per axis / ±30°, rigidly coupled | | Six initial arm angles | Nominal ±[45,20,30,45,30,90]°, clipped to joint limits | | Initial gripper openness | 0.85–1.0, coupled passive linkage angles | | Camera mount XYZ / RPY | ±1 cm per axis / ±5° per Euler component | | Camera focal lengths | ±2% independently | | Camera principal point | ±6.4 / ±4 pixels in the 1280×800 calibration sensor | | Tangential friction | 0.7–1.3 × nominal | | Bottle mass and inertia | Shared 0.8–1.2 multiplier | | Joint damping | 0.8–1.2 × nominal | | Native light diffuse / material RGB | 0.7–1.3 / 0.8–1.2 × nominal, RGB clipped | | Composite exposure / white balance | 0.85–1.15 / 0.95–1.05 | | Image noise | Gaussian σ up to 2/255 | At scale `s`, zero-centered offsets are multiplied by `s`, multiplicative parameters interpolate from 1, and bottle XY interpolates from its nominal location toward a full-table proposal. Ten percent of training episodes remain exactly nominal. Fixed-scale matrix evaluation uses zero nominal mixing, so nominal and randomized strata are explicit. | Vector control steps | DR scale | |---:|---:| | 0 | 0.05 | | 20,000 | 0.15 | | 50,000 | 0.35 | | 100,000 | 0.65 | | 200,000 | 1.00 | A vector step advances all worlds once; transitions equal vector steps × number of worlds. The curriculum is step-based, not success-adaptive. A boundary affects a world at its next reset. Full training checkpoints save and restore MJLab's `common_step_counter` and validate task settings; actor-only evaluation does not restore the training counter. The broad position/camera bounds come from the copied policy-training study in `docs/provenance/policy_dr_findings.md` and `policy_dr_ranges.json`. Its combined geometry/IK/view checks accepted 10/512 independent samples and 23/256 samples with a conditional start-pose sampler. Those are proposal bounds, **not a proven feasible envelope**. The old study sampled Cartesian start poses and solved IK; this task uses direct bounded joint perturbations instead. The initial-joint, physics and image-effect endpoints are explicit starting design choices. No object placement is rejected for IK reachability or visibility. Unreachable, off-camera and physically poor starts remain in evaluation denominators. Contact sensors report penetration and invalid motion; a separate conservative table/pedestal XY-footprint overlap diagnostic flags potential static-layout problems (MuJoCo does not detect static-static contacts). That footprint test has false positives because the pedestal foot has holes; it does not reject samples or gate success. Wall/layout validity beyond actual robot/bottle contacts is not fully certified by this diagnostic. ## Rewards, success, terminations The reward dictionary and velocity-weight curriculum are taken directly from the pinned MJLab `make_lift_cube_env_cfg()` at commit `8ee51fbcf806a7419189f706d9e394cbeb7790fa`. Only the object name (`bottle`) and robot site (`tcp`) are changed. The reward functions are imported upstream functions, not reimplementations. MJLab integrates reward rates by 0.05 s per control step. Let `d_reach` be TCP-to-bottle-root distance and `d_goal` bottle-root-to-target distance, both in meters: | Upstream term | Definition | Weight | |---|---|---:| | `lift` | `exp(-d_reach²/0.2²) × (1 + exp(-d_goal²/0.3²))` | 1 | | `lift_precise` | `exp(-d_goal²/0.05²)` | 1 | | `action_rate_l2` | Squared normalized action difference | -0.01 | | `joint_pos_limits` | Sum of violations outside soft joint position limits | -10 | | `joint_vel_hinge` | Sum of squared speed excess above 0.5 rad/s | -0.01 → -0.1 → -1 | The last weight changes at vector control steps **0 / 12,000 / 24,000**, using MJLab's native reward curriculum on reset. As upstream, joint penalties cover all robot joints, including gripper linkage joints. The positive shaping terms have maximum combined rate 3; there is no separate success bonus or custom contact/lift/hold reward in this recipe. `BottleLiftingCommand` samples an episode-fixed goal around the **nominal TCP**, independently of bottle placement and initial arm-angle randomization. The nominal center is FK of `goal.nominal_joint_pos`, copied from the initial arm configuration: robot-base coordinates approximately `[0.656032, -0.138629, 0.265982]` m. At nominal base placement that is world `[0.147758, -0.171370, 0.988588]` m. The region follows the randomized robot base, not the table or object. At full scale the position must lie within **±5 cm X/Y and ±3 cm Z** of that center in robot-base axes. Runtime sampling draws 16 bounded joint proposals (nominal ±5° per arm joint), computes FK on CUDA, and selects the first point inside the Cartesian region. If none qualify it uses the nominal FK point. Every returned goal has a stored joint-limit-valid FK witness. This guarantees kinematic TCP reachability; arbitrary randomized obstacles may still obstruct an arm configuration or its approach path. The distribution is the accepted FK proposal distribution, **not uniform Cartesian sampling**. The same per-episode DR scale shrinks both joint proposals and the Cartesian region through the curriculum; nominal episodes use the center. The target is fixed until reset (inherited resampling interval 1e9 seconds). The sampler never moves the bottle and never rejects unreachable object starts. Settings live in JSON group `goal`; `high_lift_height_m` now only controls the high-success metric. The inherited MJViser target sphere is available in the browser. The visual actor receives the commanded base-frame goal in observation group `goal`; its critic receives bottle-to-goal displacement. The state teacher gets bottle-to-goal directly in its low-dimensional observation. BC recordings include `goal` and `next_goal`, and the visual BC network consumes the goal explicitly. Upstream command diagnostics (`Metrics/lift_height/at_goal` and `episode_success`) mean being within 5 cm of the goal, with no grasp/hold guard. **They are not our task success rate.** The authoritative task success metrics below remain contact- and hold-qualified. Dense proximity rewards can be earned without a valid grasp; their effectiveness needs an actual training experiment. Task spec **bottle-lift-v5** records reward functions, parameters, goal command and penalty schedule for strict resume checking. The observation/action simulator contract is **mjlab-gs-cartesian-v3**, with explicit visual/state modes. Older non-goal-conditioned actors require deliberate migration; strict resume also checks observation definitions/noise and the goal distribution. Success requires the bottle's **collision-geometry bottom** to be at least 10 cm above the randomized tabletop, both gripper pads contacting the bottle with normal force ≥0.05 N, linear speed <0.15 m/s and angular speed <1 rad/s, continuously for 2 seconds. The monitor runs at every physics substep; integer hold counters avoid floating-point errors at the two-second boundary. Episode validity additionally requires finite state, peak penetration ≤3 mm, peak arm speed ≤1.57 rad/s, actuator forces within configured limits, and no robot-room collision after a 0.2-second contact grace period. Penetration and speed peaks include startup. A separately logged high-success metric requires 15 cm with the same hold/validity criteria. This is a versioned task definition, not an exact recreation of the old study's success predicate. Default training continues after success and terminates at 15-second timeout, nonfinite state, penetration >2 cm, or bottle center below tabletop minus ~19 cm. Unreachable objects can simply time out. Continuing after success lets dense rewards reward continued holding rather than making successful termination forfeit future return. `objective.terminate_on_success=true` is an explicit optional override. Evaluation matrix runs also keep episodes running to observe high holds and subsequent drops. The main success flag remembers a completed qualifying hold, subject to episode-wide validity; drops are additionally reported. ## Evaluation and observability MJLab logs `Episode_Metrics/` success, high success, peak clearance, best hold, lift/drop flags, bilateral-contact fraction, collision, penetration, joint speed, force ratio, nonfinite state, elapsed time, DR scale and layout footprint overlap. `Episode_Reward/` exposes all reward components. Curriculum logs expose the active scale and vector step count. The evaluator assigns fixed episode quotas to worlds. Faster failure worlds cannot replace the quotas of slower worlds. It records every counted terminal episode before resetting, including failures, and produces success/lift tables by DR scale and a 3×3 table-local bottle grid. Lift rates at 2, 5, 10 and 15 cm are geometric peak-clearance rates, distinct from qualified lift-and-hold success. Unpopulated bins are absent, not zero-success claims. If the step budget expires, final success rate is null and unfinished quotas plus completed-only diagnostics are explicit. ```bash cd ~/mjlab_gs source env.sh # Inspect early curriculum and full randomization in the standard browser viewer. mjlab-gs view --randomize --task-config configs/curriculum.json mjlab-gs view --dr-scale 1 --num-envs 4 # No optimizer: default action holds the arm and keeps the gripper open. mjlab-gs evaluate --num-envs 8 --episodes 8 --steps 350 --dr-scale 1 \ --output reports/evaluation_full.json python scripts/evaluate_matrix.py --checkpoint path/to/canonical_policy.pt \ --num-envs 8 --episodes 32 --output reports/evaluation_matrix.json # Change endpoints, hold thresholds or schedule via a JSON preset. # Training is configured but intentionally not started in this change. ``` ## Rendering, memory and limitations Native mesh geometry and Gaussian camera projection use the same per-world camera extrinsics and intrinsics, including non-native image aspect ratios. The table moves as native geometry. Reset explicitly refreshes world-welded geom transforms after GPU kinematics; Warp otherwise caches these at startup. The collision broadphase reads current geom poses and the renderer refits its BVH before sensing. The pedestal Gaussian means, rotations and SH viewing directions follow its randomized rigid transform; room Gaussians remain shared. Fully randomized scenes render in bounded chunks of eight worlds (configurable) to avoid duplicating all Gaussian parameters across every world. Nominal scenes retain the shared-camera packed path with chunks of 128. The Gaussian room contains baked illumination: native light/material variation and final RGB exposure/white balance do not constitute physical Gaussian relighting. Table textures/patterns, lens distortion, latency, action delay, external pushes and scene-wide physical relighting are not implemented here. State, reset samples, physics parameters, IK, contact metrics and rendering calculations stay on CUDA. Startup, MJLab orchestration/reset bookkeeping, packed-rasterizer allocation synchronization, browser output and evaluation JSON still involve the CPU. Full host-free execution is not claimed. Derived contact and geometry data have MJLab's normal one-physics-substep lag (1 ms). Tests cover objective logic and simulator/render/reset integration. Zero-action evaluation is an infrastructure check, not a feasibility survey or a trained policy result. Historical integration and domain-v2 measurements retain their original task specifications. Current goal/teacher checks and throughput are in `reports/teacher/`. ## State teacher task `Mjlab-GS-Bottle-UR5-State` uses the native MJLab lift-cube observation pattern: 12 relative joint angles, 12 joint velocities, TCP-to-bottle (3), bottle-to-goal (3), previous action (7): **37 floats**. All joint/linkage states are included. Vectors are in robot-base coordinates. The actor has uniform observation noise (angles ±0.01 rad, velocities ±0.05 rad/s, relative positions ±2 mm); the critic receives the same 37 quantities without noise. Play/evaluation disables actor noise. Both networks are MLPs with hidden layers **256, 128, 64**, ELU activation, and observation normalization. No CNN or pixel rollout buffer is constructed. No camera sensor or render callback is attached. Gaussian assets are not loaded by a sensor, and camera/intrinsic/light/material model fields are not expanded for DR. The browser can still display native 3D geometry on demand. Physics, contact tracking, reset handling, rewards and the 50 physics substeps remain. Teacher defaults use table XY ±15 cm/yaw ±15°, robot/pedestal XY ±10 cm/yaw ±15°, and initial arm angles ±[15,10,10,15,15,30]°. Full-table bottle sampling, friction, mass/inertia and damping DR remain. Camera and appearance families are disabled. `configs/teacher.json` records the endpoints and shared curriculum. These are initial defaults; learning performance has not been established. See [teacher workflow](teacher.md) for commands and the planned student phase. ## Visual network default The PPO visual actor now uses an ACT-style architecture resembling the older policy_training export: **ResNet18**, a **256-wide transformer with 8 heads, 3 encoder layers and 2 decoder layers**, and a 256-unit PPO output head. It uses one learned action query to produce one 7D action per 20-Hz tick. Goal and proprioceptive tokens accompany the image tokens. Visual BC shares the same feature architecture with its own deterministic action head. Unlike the older ACT training recipe, there is no CVAE or 20-action chunk in this PPO actor. Transformer dropout is zero and BatchNorm statistics are frozen so the same observation has consistent outputs in rollout and minibatch modes. Weights start untrained, without automatic pretrained-weight downloads. The input is now 320×200, but causal proprioception history is still one frame; the old 16-frame history is not silently claimed to be implemented. The critic and rendering-free teacher retain their existing MLPs. `src/mjlab_gs/models/visual.py` owns this standalone implementation and its profile. Old small-CNN actor/BC weights require explicit migration; the observation contract also detects image-size mismatches. Task spec v5 records the actor profile. Three-input PyTorch/TorchScript export is supported; ONNX attention export remains to be qualified. See `reports/visual_policy/` for current checks; older throughput tables retain the resolution/architecture under which measured.