One simulator for BC, RL
and evaluation.

Shared environment, shared contract

Mjlab-GS-Bottle-UR5 uses 7 normalized actions: base-frame Cartesian translation/rotation and gripper openness. Control is 20 Hz with 50 physics substeps. The visual policy receives 16 proprioceptive values, a 3D commanded goal and Gaussian wrist RGB; the state teacher receives 37 floats. Checkpoints carry a simulator contract.

cd ~/mjlab_gs
source env.sh
mjlab-gs view --num-envs 4
mjlab-gs train --num-envs 64
mjlab-gs evaluate --checkpoint logs/rl/bottle/final.pt

Setup, APIs and workflow commands · Recorded checks and measurements · Integration report

320×200 visual policy · ACT-style network

The visual task, CLI and evaluation defaults now use 320×200 wrist RGB, matching the earlier policy-training image size. The actor uses ResNet18 with a 256-wide, 8-head transformer: 3 encoder layers and 2 decoder layers, followed by a 256-unit action head. Visual BC shares the same features. The state teacher retains its 256–128–64 MLP and does no rendering.

This is a single-action 20-Hz PPO adaptation: explicit commanded goal, one current proprioceptive frame, no CVAE or action chunk. Dropout is zero and BatchNorm statistics are frozen. Weights start untrained. The older ACT’s 16-frame temporal history remains a separate design to qualify.

320×200 GPU observations, finite action-distribution gradients, batch/train-eval consistency and scripted deterministic export equivalence passed. No optimization or policy training was run.

Current visual-policy validation · Task v5 specification

RL experiment guidance now accepts overall success below 90% when unreachable starts explain it. Use ordinary success curves, videos and labeled restricted-DR tests to establish reliable reachable behavior; no separate reachable-only success metric is requested.

Upstream MJLab lift-cube rewards

The current bottle task directly uses MJLab’s five lift-cube reward terms and defaults: staged reaching/lifting (+1), precise positioning (+1), action smoothness (−0.01), soft joint limits (−10), and excess joint speed (−0.01 → −0.1 → −1 at 0 / 12k / 24k vector steps).

The goal is randomized around the nominal TCP in the robot-base frame, independently of bottle placement. Valid arm configurations supply reachable FK candidates inside ±5 cm X/Y and ±3 cm Z; the curriculum scales the region. The reward functions are unchanged. The previous custom grasp/hold rewards and 200-point success bonus are removed. Episodes continue after success so sustained holding keeps earning reward. Strict two-pad, 10-cm, 2-second success metrics remain independent of proximity reward and MJLab’s loose goal diagnostics.

Current goal/teacher checks and measurements · Task v5 specification. No RL training was run.

Domain randomization and curriculum

The task now randomizes bottle/table/robot poses, initial joints and gripper, camera mount and calibration, friction, bottle mass/inertia, damping, native lighting/material color and image appearance. Table, robot and Gaussian pedestal placement stay coherent per world.

Training scales the ranges through 5%, 15%, 35%, 65%, 100% at 0, 20k, 50k, 100k and 200k vector control steps, applied on the next episode reset. Ten percent of training episodes remain nominal. Broad proposal ranges are intentionally not filtered by IK or visibility.

Success: a valid two-pad grasp, bottle bottom at least 10 cm above the table, held stably for 2 seconds. Metrics include 15-cm holds, peak lift, drops, collisions, penetration, motion limits and a table-location/DR-scale evaluation matrix. The visual actor receives 16 state values + 3 commanded-goal values + RGB; the privileged critic is now 62 values, including displacement to the lift goal. Bias compensation fixes neutral-action gravity sag. Observation contract v3 adds explicit goal conditioning and separates state/visual modes.

Complete ranges, rewards, curriculum and evaluation contract · Current goal/teacher report · Historical reward integration report · Historical DR validation report

Reset checks cover model/state isolation, curriculum boundaries, world-welded collision geometry and randomized visual agreement. The baked Gaussian room is not physically relit; exposure and white balance are image effects.

Historical reward-v3 throughput · before reachable goals

40 timed vector steps after warmup, including IK, 50 physics substeps, all task metrics and RGB. Different randomized views and failure/reset frequencies affect these short measurements.

Evaluation plumbing check · no trained policy

Zero Cartesian action, open gripper; fixed per-world quotas. These historical task-v2 results validate accounting and failure handling, not reachability or learned skill; they predate the upstream reward switch. No RL training was run for this update.

Rendering-free state teacher

Mjlab-GS-Bottle-UR5-State receives 37 state values: joint positions/velocities, TCP-to-bottle, bottle-to-goal and previous action. Its actor and critic are 256–128–64 MLPs. Camera sensors, Gaussian loading by sensors, pixel buffers and visual DR are absent. Physical randomization uses smaller initial pose ranges while retaining broad bottle placement and the same rewards, actions and success metrics.

mjlab-gs view --task Mjlab-GS-Bottle-UR5-State --num-envs 4
mjlab-gs benchmark --task Mjlab-GS-Bottle-UR5-State --num-envs 256 --task-config configs/teacher.json

The visual actor now sees the commanded goal explicitly; the reachable region follows robot-base randomization, even when the bottle starts beyond reach. FK reachability does not guarantee a collision-free route through every randomized layout.

Historical task-v4 measurements: A6000, matched teacher physical preset at its initial 5% scale, 40 timed steps after warmup; visual comparator uses 128×128 RGB. Includes physics, IK, rewards, metrics and observations; excludes policy inference/optimization and browser output. No training has been run.

Goal FK, partial resets, explicit goal observations and both untrained network types pass GPU checks. State initialization also passes with camera initialization forbidden. Goal/teacher baseline report · Teacher-to-student plan and commands

Browser visualization

The interactive 3D scene uses MJLab's mjviser integration. The camera-feed panel displays the actual Gaussian observation used by policies. Play, pause, step, reset and environment selection use MJLab's controls. The live viewer runs on the GPU host; access it through SSH port forwarding. This public page is a results page.

Visual parity

Original nominal reference on the left, canonical MJLab sensor on the right. Six poses are compared with matching camera and material settings. These six nominal comparisons are historical integration checks; the randomized comparison above is a historical DR/rendering check. Source and composed CPU models match geometry positions, mass matrix, actuator gains and collision parameters.

Historical task throughput · before curriculum and contact metrics

RTX A6000. Includes Cartesian IK, 50 physics steps, manager bookkeeping, Gaussian/mesh compositing and observation conversion. Excludes policy optimization, browser streaming and recording. NVML memory is sampled process memory. The older held-control GSim benchmark measures a different, cheaper workload.

Readiness and remaining work

Task construction, copied assets, GPU camera data, BC recording/loading, PPO integration, checkpoint validation and remote browser controls share one simulator. Earlier optimizer smoke checks validated infrastructure. The current update adds reachable randomized TCP goals and a rendering-free state teacher, and runs no RL training; it does not establish learned grasp success or real-robot readiness.

Bulk state and image computation remains on CUDA. Startup, browser/disk output and some small allocation/reset synchronizations involve the CPU. Packed gsplat is not completely host-free. PPO image rollout storage consumes substantially more memory than simulation alone.

MuJoCo Warp's contact-count differences remain: full grasp robustness needs separate qualification. The room Gaussian field remains fixed while the pedestal moves per world. Multi-camera observations, physical Gaussian relighting and production-scale training qualification remain future work.