# D415 bin-picking expert development

The D415/mount-v4 integration, revised expert and corrected success measurement
are implemented. Development benchmarks were physics-only. At the user's request,
separate `site_d415_single` and `site_d415_mixed` rollouts record external and
D415 wrist videos for a website update; they are not replays of the benchmarks.
The expert is
not yet a reliable bin-clearing demonstration generator; near-100% success is
not supported by these results.

The first published external videos accidentally used the textured room mesh,
not 3DGS. `site_d415_gs_single` and `site_d415_gs_mixed` replace them with
Gaussian-background / native-foreground recordings. External walls are now
opaque. These are fresh rollouts, separately scored; expert and physics settings
are unchanged. New recordings save `video_poses.npz` for future visual-only replay.

## Implemented changes

- Bin-picking-only D415 assembly with the approved pose, factory intrinsics,
  320×180 future wrist rendering, camera collision box and conservative mount
  collision hull. The bottle and tube-sort scenes retain their contracts.
- Unchanged arm position controller and RG2 position actuator/effort bounds.
- Collision-mesh grasp cross-sections, multiple depths/yaws/tilts, contact-strip
  width estimation, two-pad target-overlap checks, IK and sampled vertical-path
  collision checks in an isolated kinematic replica.
- Timeout recovery, slip detection, above-rim transport, low release, verified
  tabletop deposits, vertical retreat and regrasp of reachable tabletop drops.
- Source snapshots, phase-transition logs, planner rejection reasons, failure
  events and final physics-state snapshots in evaluation outputs.

The mount hull fills empty openings and can reject poses that an exact mount
decomposition would allow. Contact overlap is a geometric screen, not proof of
force closure. No live object is attached, pinned or teleported by the picker.

## Completed development measurements

Each row is a different development version, not a matched ablation or a
statistical performance comparison. Object families, horizons and pile states
vary. Even repeated seeds are not bitwise deterministic in the GPU simulator.
Attempts include attempts still in progress at a time limit.

| Run | Poured objects | Attempts | Verified deposits | Fully cleared bins |
|---|---:|---:|---:|---:|
| Original D455 baseline | 20 | 28 | 0 (2 falsely reported picks) | 0/2 |
| `d415_tilt_v5` | 19 | 15 | 3 | 0/2 |
| `d415_contact_v9` | 19 | 10 | 2 | 0/2 |
| `d415_single_v6` (isolated-object diagnostic) | 4 | 1 | 1 | 1/4 |
| `shallow_single_60mm` | 4 | 4 | 3 | 3/4 |
| `shallow_single_offset` | 4 | 9 | 4 | 4/4 |
| `shallow_pile_60mm` | 11 | 11 | 4 | 0/4 |
| `shallow_offset_60mm` | 11 | 30 | 4 | 0/4 |
| `shallow_exact_60mm` | 11 | 31 | 4 | 0/4 |

The shallow runs use a user-authorized 400×140×60 mm internal bin. This fits
the existing tabletop and reach band without relocating the robot. The historical
400×128×150 mm default remains available and unchanged; shallow geometry is an
explicit evaluation override. Lower walls also stop pouring earlier: these mixed
runs contain 11 objects across four worlds, not the requested maximum of 48.
The isolated runs use one bottle family, not a broad object benchmark. Four
successful isolated episodes do not establish near-100% reliability.

The original baseline's two counted picks had slipped before the drop point;
all 20 objects remained in the bins. See its `transitions.json`. Early D415
versions also exposed an overstrict placement check against arbitrary mesh
body origins; current scoring uses actual geometry bounds and contact tolerance.
Only support-verified post-release deliveries count in the current code.

`ejected` in the evaluation JSON is a legacy field meaning an active object
outside the bin without a verified deposit. It can include an object being
carried at the time limit; it is not a count of solver-launched objects.
`finished` also includes exhausting grasp candidates; use `bin_cleared` for
complete success. Per-run penetration and speed statistics are retained.

## Remaining limitations

The historical bin is 400×128 mm internally, with 150 mm walls. The saved
single-object check found no candidate for three initial placements; the one
attempted placement was delivered successfully. In one blocked state, 35
candidate rejections involved RG2 links/housing/bracket or the D415 mount
against bin walls. See `single_feasibility.json` and
`single_contact_feasibility.json`. These are failures of the current candidate
search and collision model, not a proof that every possible grasp is impossible.

Empty lifts, motion timeouts, transport slips and failed deposits still occur.
The expert does not yet deliberately rearrange wall-blocked objects. Geometry
changes are now authorized and shallow-bin experiments are recorded above.
Offset contact-strip candidates helped the isolated-bottle test but did not
improve mixed-pile delivery counts. That mixed run logged 11 motion timeouts,
11 empty lifts and three transport slips (other attempts succeeded or were
unfinished at the horizon). A subsequent planner change removes an
overconservative flat finger/wall rectangle filter, relying on posed collision
geometry instead; saved-state checks recover a previously rejected candidate
(`shallow_single_exact_clearance.json`). Better closure/contact planning and
rearrangement remain relevant before large-scale collection or RL.

The full `shallow_exact_60mm` rollout did not improve deliveries: four verified
deposits, 14 empty lifts, 11 motion timeouts and one transport slip. It reached
the 120 s simulated horizon with one world unfinished. Maximum robot/object
penetration was 2.27 mm, with no nonfinite state or anomalous-speed samples.
Although early progress showed fewer timeouts, that did not hold over the full
rollout. The newly admitted grasps are not validated as dynamically reliable.
Do not promote this development version or bin geometry as a solved expert.

## Verification and reproduction

`python -m pytest tests/test_external_rendering.py tests/test_binpick.py tests/test_tubesort_expert.py -q`:
**16 passed**. Checks cover external compositing and read-only poses, the approved D415 centre and intrinsics, original
gripper effort bound, unchanged bottle camera, mount collision presence,
placement false positives, contact tolerances and pad-local width estimation,
along with the existing trajectory/rotation helpers.

See `src/mjlab_gs/tasks/binpick/README.md` for the contract and commands. The
latest evaluator defaults to no camera sensors and no rendering. Use a fresh
output directory; each run's `sources/` records its actual implementation.