Tube bin picking: scripted expert evaluation

Updated 24 September 2026. A privileged scripted UR5 + RG2 expert empties a free 16 g Hazel box of 15 hollow PLA tubes (20 mm × 80 mm, 10 g, piled three layers deep), dropping each tube into either side box. Tubes are only grasped across their axis: the fingers dig into gaps between tubes, close while descending, follow the tube as it moves, and retry deeper after a miss. Colour sorting is on hold: picking from a pile was where it failed, so the picking task is being solved first.

1729/1918 tubes picked (90%, 95% CI 87%–93%) over 128 episodes with 15 (a few piles settle with 14 still in the box) tubes, full domain randomization, time cap 60 s + 45 s per tube, every started episode run to completion. Episodes fully cleared: 64/128. Physically valid: 0/128 (robot faults 9, physics faults 128). Non-finite episodes: 0, counted as 0 picked.

Result by pile size

Tubes at startEpisodesPickedPicked rateFully clearedValidMean timeTime capTimed out
14225/2889%1/20/2312 s735 s0/2
151261704/189090%63/1260/126396 s735 s1/126

A tube counts as picked once it is out of the source box and released. Valid means the robot never hit the table or room (touching and pushing the boxes is allowed), no box toppled (tilt past ~25°), robot penetration ≤ 3 mm, box contacts ≤ 0.8 mm, tube contacts ≤ 1.5 mm, joint speed ≤ 1.57 rad/s and finite physics. Tubes left in the box: 109 walled in (no side grasp clears the box wall and no roll freed them), 74 missed three grasps in a row, 0 never given up on (time ran out first). Per-episode results · Summary

Videos

15 tubes, current expert: 15/15 picked in 297 s (validity gate failed: box_penetration, ~1 mm)
15 tubes, current expert: 15/15 picked in 406 s (validity gate failed: box_penetration, tube_penetration)

Earlier expert, 1 to 10 tubes (23 September)

Before the dig-in grasps, live re-aim and CG-80 contacts; that version picked 49% of tubes over 1 to 10 tube piles.

Earlier expert, 1 tube: 1/1 picked in 30 s (valid)
Earlier expert, 3 tubes: 3/3 picked in 97 s (valid)
Earlier expert, 5 tubes: 5/5 picked in 166 s (validity gate failed: box_penetration)
Earlier expert, 10 tubes: 7/10 picked in 354 s (validity gate failed: robot_touched_box_or_room, box_penetration)

Simulation: stable and fast

PhysicsMuJoCo Warp, 1 ms step, 2 ms contact time constant, CG solver 80 iterations, elliptic cones
Collision geometrybox = floor plate + 4 tapered wall plates (visual mesh unchanged); tube = one capsule (hollow visual mesh)
Contacts per settled world~35 (was ~1,400 with 177-piece box shells and 32-sector tubes)
Throughput, physics only366 env-steps/s at 1,024 worlds (was 9 env-steps/s at 32 worlds)
Memory sizingnjmax 1024 rows/world, nconmax 128/world, naccdmax 32/world (was up to 17 GB at 512 worlds)
Non-finite (NaN) worlds, expert acting0 / 256 over 90 s; 0 in the 128-episode 15-tube evaluation
Box topples, 64 active 15-tube worlds, 125 s0 with CG-80 (16 with CG-40: unconverged impulses launched free boxes; Newton: 0 topples but 6 NaN)
Controller10 mm and 3 deg per step: 146 mm/s TCP, 44 deg/s wrist, 0.6 rad/s peak joint speed

What limits the expert

Fifteen tubes cover more than the 149 × 94 mm box floor, so the bottom layer is packed wall to wall, and a side grasp needs a corridor about 60 mm wide across the tube. The expert takes the top layers first, digs a finger into the gap next to a crowded tube so the descent pushes it free, and rolls or slides tubes that are still walled in. What remains are tubes pinned against the long walls under their neighbours and grasps that miss three times. Every episode still exceeds the 0.8 mm box-penetration gate: pressing into a three-layer pile pushes the box floor about 1 mm into the table. The earlier colour-sorting work and its videos remain on the tube sorting page.