Tube sorting with free Hazel boxes and hollow PLA tubes
Updated 22 September 2026. A scripted UR5 + RG2 expert picks from the centre box and sorts red tubes left and blue tubes right.
New physical assets: three free 16 g Hazel 500 ml boxes, each 175 × 120 × 36 mm; 10 g hollow PLA tubes, 20 mm outer diameter, 16 mm inner diameter, 80 mm long. Boxes can slide or tip when contacted. Both tubes and boxes move through simulated contact forces.
Contact-aware Hermite expert
Development status: the four v4 regression worlds passed the physical
validity checks, but crowded-pile sorting remains incomplete. The updated policy is not yet ready for
large-scale demonstration collection.
The updated demonstrator uses smooth travel, descent and retreat paths, eased wrist rotation and gripper
motion, and live destination poses. It lowers before release, opens enough to separate from the tube,
checks separation, and lifts clear before opening fully.
Grasp search now checks three locations along a tube. Retry scoring keeps each grasp's own wall clearance;
timeouts and unintended box motion trigger recovery. A tube still touching a gripper pad does not count as sorted.
Motion recording
Actual control-frame recordings at 20 fps with expert v5, which also
budgets travel duration for eased wrist rotation within the controller's angular limit. 0/4 sorted over 30.0 s, physical gate passed.
These development clips show current behaviour; they are not a reliability benchmark.
Wrist view · recorded at 20 fpsExternal view · recorded at 20 fps
Task-focused regression trials · expert v4
Boxes remain free at 16 g and tubes at 10 g. Nominal box/tube sliding friction is 0.6/0.5, with the friction
variation below. Nested boxes are excluded from task acceptance. The eight task-focused cases include piles,
low drops, rim contact, a tilted tube, an occupied destination, a box push and gentle extraction.
The 0.5 ms profile remained finite in all eight cases, with peak active-object penetration 0.784 mm.
The 0.25 ms variant produced a nonfinite box-push case and was rejected. The recordings use 0.5 ms steps,
2 ms contact time constants and 0.99 impedance, with a 20 Hz policy.
Seed / world
Tubes
Friction
Sorted
Horizon
Box–tube peak
Physical gate
22 / 0
4
1.0×
1/4
80 s
0.414 mm
pass
7 / 0
1
0.7×
1/1
40 s
0.297 mm
pass
7 / 1
4
1.0×
1/4
40 s
0.339 mm
pass
7 / 2
10
1.3×
0/10
40 s
0.404 mm
pass
The physical gate includes every-substep active-object penetration: 0.8 mm for box contacts and 1.5 mm
for tube contacts, alongside robot-contact, joint-speed and force checks. A completed isolated pickup does
not establish crowded-pile success. Fixed horizons and different tube counts should not be combined into a success rate.
The collision-piece audit found no degenerate or non-convex parts. The centre section follows the visual
wall, rim and floor profile; this does not prove every internal seam is harmless. The supplied geometry was retained.
Collection now rejects invalid initial scenes without removing tubes, masks failed attempts from BC labels,
and scores failures before resetting. Attempt budgets bound failed collection runs. The CLI defaults to fully
sorted, physically valid episodes. Large-scale collection and student retraining have not been run for this update.
Fresh nominal evaluation: 2/8 tubes sorted across 2 four-tube episodes; 0/2 fully sorted, 0/2 fully sorted and physically valid. These are development recordings, not a reliability benchmark.
Remaining failure: boxes are dragged and lifted during picking and placement. The original robot-focused validity gate did not measure all object–object penetration. The contact investigation below reproduces coupling without the robot; physics validation is required before interpreting expert success.
Episode 1 (world 1): 0/4 sorted
Wrist viewExternal view
Episode 2 (world 0): 2/4 sorted
Wrist viewExternal view
Recorded outcomes
World
Sorted
Duration
Contact/speed gate
Max robot penetration
Box/room contact
1
0/4
33.9 s
yes
0.32 mm
no
0
2/4
45.0 s
yes
2.46 mm
no
Validity retains the 3 mm robot-contact penetration limit, 1.57 rad/s arm-speed limit and 0.05 N unintended box/room-contact gate. A completed sort can fail physical validity. Episode results · Run and simulator contract
What changed
The supplied visual meshes and convex shell parts replace procedural trays and capsule tubes. Explicit source inertia preserves the 16 g and 10 g masses without counting overlapping collision pieces multiple times. The boxes now have free joints, and the task tracks their current poses. The expert's gripper lookup was extended using the bin-picking calibration method so it can close below the new tubes' 20 mm diameter.
The box-force check applied 0.2 N for 0.1 s to an empty box; it moved 35.0 mm along the push direction after release. All eight tubes remained in the centre boxes in the separate two-world settling check. Force-response audit · Reset/settling check
Tube size stays fixed at the measured geometry. Colours use red/blue RGBA on the source hollow mesh; the packaged PLA textures are not applied in these recordings. External and wrist views combine Gaussian room appearance with the simulated foreground.
Further expert improvements
Bin picking already provides sampled IK/collision checks, multiple grasp orientations, slip recovery and verified placement. Those are useful next steps for tight-box tube sorting, especially when a tube rests against a wall or a light box moves. The current tube expert still uses its side/end grasp search and bounded nudge recovery.
Not yet validated: controlled tests reproduce box coupling without a robot grasp.
Harder contacts improve several interactions, but do not resolve every case. The settings below are experimental;
the production contact defaults have not been switched.
Backend: MJLab + MuJoCo Warp in mjlab_gs, with MuJoCo 3.11.0 / Warp backend 3.11.0.
The boxes remain free at 16 g and the tubes at 10 g. Forty controlled cases cover piles, drops, rim impacts,
nested and offset boxes, pushes and a high-drop stress test. Nine extraction cases and four 25-second expert
windows provide separate checks. These are diagnostic trials, not a task-success benchmark.
Profile
Ordinary peak penetration
High-drop stress peak
Baseline
2.825 mm
1.661 mm
Half timestep
3.318 mm
1.756 mm
4 ms contacts
1.417 mm
1.770 mm
2 ms contacts
0.614 mm
1.759 mm
2 ms contacts, rotational friction removed
Nonfinite: rejected
1.757 mm
Depth is measured every physics substep across active box–box, box–tube, box–room and tube–tube contacts.
The 2 ms profile uses a 0.5 ms step and 0.99 impedance on both contact participants, while keeping the policy at 20 Hz.
The model has 0.9 mm walls. The stress pulse lofts a tube roughly 1.2 m above the table; it is a separate high-speed limit test.
In baseline extraction, pulling only the upper box lifts the lower box by 68–78 mm. Stiffer contacts remove
the measured offset-box coupling, but aligned nested boxes still rise together. At the sampled aligned pose,
rim–wall penetration is only about 0.002 mm; frictional wedging must be considered separately from excessive overlap.
Reducing box friction releases the aligned pair, supporting frictional wedging, but the later return impact
launches an offset lower box by 238 mm. These coefficients are uncalibrated and are not promoted.
Extraction rise uses the first 0.5 s; the separate one-second window includes return impacts.
CPU and GPU collision depths agree within about 0.0001 mm at six saved failure poses. That comparison does
not validate full dynamic equivalence. NoSlip, suggested in some MuJoCo discussions, is unsupported by this installed Warp backend.
Unchanged expert comparison
Same seed, four- and ten-tube scenes, 25 simulated seconds, no automatic resets.
These clips replay the measured poses at 4 frames/s. Incomplete windows are not complete-episode success rates.
Profile
Tubes
Sorted at 25 s
Peak box–tube penetration
baseline
4
1
3.369 mm
baseline
10
2
4.750 mm
hard2
4
1
0.999 mm
hard2
10
1
1.050 mm
baseline: 4 tubes, first 25 secondsbaseline: 10 tubes, first 25 secondshard2: 4 tubes, first 25 secondshard2: 10 tubes, first 25 seconds
