Updated 2026-09-18. Objects are Google Scanned Objects (1030 scans, V-HACD convex parts from kevinzakka/mujoco_scanned_objects). A tote is filled to the brim by pouring random objects in one at a time, the heap is audited for stability with nothing actuated, and then a scripted gripper picks from it. Picks are allowed to fail; what is judged is whether the physics stays correct while the pile is disturbed. Every number here is read from the run files.
Development results, not a solved expert. The offset-grasp isolated-bottle benchmark cleared 4/4 bins, with retries. The latest mixed-pile benchmark delivered 4/11 objects and cleared 0/4 bins. Four isolated episodes do not establish near-100% reliability. New video rollouts below are separately scored.
UR5 + OnRobot RG2 + D415 with the existing mount-v4 assembly. The test bin is 400 × 140 × 60 mm internally; the historical bin was 400 × 128 × 150 mm. Shallower walls improve access but also reduce the number of objects poured. The arm position controller and gripper actuator effort limits are unchanged. The expert uses privileged collision meshes, multiple grasp directions and depths, tilted and offset grasp candidates, collision checks, slip detection and retries.
Only supported, slow-moving objects deposited outside the bin near the destination count as successes. Merely reaching the drop pose does not count. Empty lifts, motion timeouts and slips remain. A conservative convex hull represents mount collisions; geometric pad overlap does not prove a stable grasp.
Full rollouts at real-time playback, sampled at 4 fps. Both views now use a 3D Gaussian background. External bin walls are opaque; the old textured-room external recordings have been replaced. Rendering changes do not change collision geometry or control. The original recordings did not save frame poses, so these replacements are fresh rollouts with updated scores. Wrist video is 320 × 180; external video is 640 × 480. These are fresh runs, not replays of the physics-only benchmark. GPU pile dynamics can differ between runs.
1/1 verified deposits; 2 attempts; fully cleared; 37.7 simulated seconds. Failure log.
1/2 verified deposits; 6 attempts; not fully cleared; 80.0 simulated seconds. Failure log.
2/4 verified deposits; 6 attempts; not fully cleared; 80.0 simulated seconds. Failure log.
| Run / raw results | Mode | Poured | Attempts | Verified deposits | Fully cleared bins |
|---|---|---|---|---|---|
| shallow_single_60mm | Physics only | 4 | 4 | 3 | 3/4 |
| shallow_single_offset | Physics only | 4 | 9 | 4 | 4/4 |
| shallow_pile_60mm | Physics only | 11 | 11 | 4 | 0/4 |
| shallow_offset_60mm | Physics only | 11 | 30 | 4 | 0/4 |
| shallow_exact_60mm | Physics only | 11 | 31 | 4 | 0/4 |
| site_d415_gs_single | Video rollout | 1 | 2 | 1 | 1/1 |
| site_d415_gs_mixed | Video rollout | 6 | 12 | 3 | 0/2 |
These are development versions, not controlled ablations. Attempts can include
unfinished actions at the horizon. The legacy JSON field ejected
means outside the bin without verified delivery, including objects still being
carried; finished also includes exhausted candidates. Use
bin_cleared for complete task success.
Development report · Simulator and expert contract. 16 regression tests passed, including external background compositing and pose preservation. No RL policy has been trained for this update.
Historical work below uses earlier geometry and policies. The old integrated pick counter did not verify successful deposits and can count objects that slipped before release; do not compare its “picks out” directly with the verified D415 results above.
Settled means: with nothing actuated for 8 s, in the last second no object leaves the footprint, none moves faster than 20 mm/s or 0.5 rad/s, none drifts more than 3 mm, no object contact penetrates more than 5 mm, all state finite. 24 random fills per row, about 21 objects each in a 40×28×20 cm tote.
| physics | settled | objects | ejected | tail speed max | tail penetration max | peak penetration |
|---|---|---|---|---|---|---|
| default: dt 1 ms, implicitfast, Newton, solref 0.01 s, margin 0 | 23/24 | 20.5 | 0 | 135 mm/s | 1.0 mm | 3.3 mm |
| solref 0.005 s | 21/24 | 21.0 | 1 | 111 mm/s | 1.7 mm | 2.4 mm |
| timestep 2 ms | 20/24 | 20.4 | 2 | 224 mm/s | 4.2 mm | 9.2 mm |
| solref 0.02 s (MuJoCo default) | 20/24 | 20.2 | 1 | 45 mm/s | 9.2 mm | 23.7 mm |
| noslip 5 iterations | 20/24 | 21.5 | 4 | 14 mm/s | 2.1 mm | 2.8 mm |
| Euler integrator | 21/24 | 20.6 | 1 | 42 mm/s | 1.1 mm | 4.9 mm |
What mattered: mass from hull volume (the default density on overlapping parts made objects 3 to 6× too heavy), dropping degenerate V-HACD slivers (7.5% of parts; a 0.5 mm wedge sank 9 mm into the floor), zero collision margin, no free-joint damping or armature, and pouring with clearance under the object's true bounding radius (the scanned meshes have their origin at the base, not the centre). Full report: standalone README.
The gripper is welded softly to its target (20 ms time constant), so the pile can push it back, and squeezes with 40 N. Across 16 bins and 96 picks: 27 successes (28%), object-object penetration at most 13.6 mm, 2 samples where a neighbour moved faster than falling from its recent height could explain, 1 object ejected, no non-finite state.
| pick | target | lifted out | peak neighbour speed | object-object penetration | gripper squeeze penetration | anomalous samples | ejected |
|---|---|---|---|---|---|---|---|
| 1 | Twinlab_Premium_Creatine_Fuel_Powder | yes | 0.35 m/s | 12.3 mm | 18.7 mm | 0 | 0 |
| 2 | Perricone_MD_AcylGlutathione_Deep_Crea | yes | 1.18 m/s | 13.6 mm | 4.4 mm | 0 | 0 |
| 3 | Cole_Hardware_Bowl_Scirocco_YellowBlue | no | 0.24 m/s | 1.9 mm | 2.8 mm | 0 | 0 |
| 4 | Cole_Hardware_Bowl_Scirocco_YellowBlue | no | 0.37 m/s | 1.7 mm | 2.0 mm | 0 | 0 |
| 5 | RESCUE_CREW | no | 0.08 m/s | 0.6 mm | 0.9 mm | 0 | 0 |
| 6 | RESCUE_CREW | no | 0.09 m/s | 1.0 mm | 1.1 mm | 0 | 0 |
| 7 | Nintendo_Yoshi_Action_Figure | no | 2.12 m/s | 7.1 mm | 4.7 mm | 7 | 0 |
| 8 | Saccharomyces_Boulardii_MOS_Value_Size | no | 0.30 m/s | 5.6 mm | 11.8 mm | 0 | 0 |
| 9 | Saccharomyces_Boulardii_MOS_Value_Size | no | 0.33 m/s | 5.4 mm | 5.4 mm | 0 | 0 |
| 10 | Beetle_Adventure_Racing_Nintendo_64 | no | 0.00 m/s | 0.3 mm | 5.3 mm | 0 | 0 |
Same objects and physics in the Gaussian-splat MuJoCo Warp scene used for the tube task. The on-table tote is 40×12.8×15 cm (the reachable band of the table is only 14 cm deep), objects are limited to 12 cm. The arm parks aside, the bin is poured full, the heap settles, then the scripted picker takes the highest object whose narrow width fits the RG2 jaws, closes with the measured pad calibration, lifts, and releases beside the bin. No object is ever pinned or teleported.
| world | poured | picks out / attempts | left in bin | ejected | object-object penetration | robot-object penetration | peak neighbour speed | anomalous samples | finite | episode |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 10 | 0/15 | 10 | 0 | 6.2 mm | 1.3 mm | 0.67 m/s | 0 | yes | 200 s |
| 1 | 10 | 3/14 | 7 | 0 | 5.2 mm | 1.1 mm | 0.34 m/s | 0 | yes | 200 s |
Failed grasps are expected with a 12.8 cm wide bin and a parallel gripper; the columns to read are the penetration, ejection, anomalous-speed and finite ones.
Standalone README · fill audit (default physics) · pick audit · mjlab episode results · tube-sorting page