# Phase 1 bottle-picking report

**Final combined-DR test: 90/100 successful picks** (95% Wilson interval 82.6–94.5%). The selected checkpoint and execution rule were frozen before this test. The point-estimate engineering target is met. Phase 1 simulation work, ancillary evaluation and policy packaging are complete. Separate domain conditions achieved only 5–8/10; this report does not establish broad robustness or real-robot readiness.

## Scope and control contract

The pipeline uses the existing UR5 CB3, RG2, D455 with blue mount, bottle, table and pedestal in `~/real2sim/sim/scene.xml`. Environment/rendering code lives in `~/real2sim/phase1/real2sim_phase1`; data, training and evaluation live in this workspace.

The policy sees current 320×200 wrist RGB and 16 causal proprioception frames, spanning 0.75 seconds. Each frame contains six joints, measured gripper-spacing proxy, fixed-TCP position and six rotation-representation values. It receives no object pose, contact truth, expert phase, or wrench.

Actions are seven position commands: base-frame measured-pose deltas `[dx,dy,dz,drx,dry,drz]` and absolute normalized gripper opening, 0 closed / 1 open. Rotations compose on the left. Commands run at 20 Hz, with independent 125 Hz joint-position targets and 1 kHz physics. The gripper retains bounded internal effort; the network does not output Newtons. The user-reported hardware 6 N setting is not a calibrated simulated contact force.

ACT predicts 20 actions. The selected execution rule uses all 20 when predicted gripper opening is steady; it uses five when the opening range across that prefix exceeds 0.2 or the first opening lies strictly between 0.1 and 0.9. This rule uses the network output alone. Runtime history is updated on every observation tick, including queued actions, and cleared at episode boundaries.

See [the versioned contract](configs/phase1.json), [frozen selection](artifacts/frozen_policy_selection.json), and [gripper mapping](artifacts/gripper_mapping.json). The JSON documents the implementation; the simulator does not load it as its configuration.

## Data and training

The frozen selection contains **314 episodes, 75,360 transitions**: 249 training and 65 validation episodes, split by whole seed. There are 1,488 masked fault frames. Supervised action chunks cannot cross fault gaps or episode boundaries; observation history cannot cross episode boundaries or access future observations.

| Source | Attempted | Accepted by collection | Selected |
|---|---:|---:|---:|
| Main diverse trajectories | 360 | 272 | 262 |
| Corrected same-seed motion replacements | 10 | 10 | 10 |
| Grasp angle/height variation | 24 | 16 | 16 |
| Empty-closure recovery | 24 | 15 | 14 |
| Incomplete-closure recovery | 24 | 14 | 12 |

Replacements substitute for ten main episodes. They are separate attempts, not independent layouts. Archived orientation-bug and oscillatory pilot datasets were excluded. Original failures remain available. Some rejected reset/path attempts lack pose specifications; coverage reporting exposes those missing values.

The generator uses seeded piecewise Hermite paths, varied intermediate points/speeds/grasp poses, contact-gated lifting and explicit retries. We verified smoothness, feasibility and recovery behavior; no measured human trajectories were supplied, so matching human demonstration statistics is not established.

ACT uses a pretrained ResNet18, action CVAE and transformer. Image augmentation applies mild gain, white balance, noise, contrast, gamma and blur. Training used the same frozen list for both seeds; the selected history stage resumed the full-data seed-1 checkpoint at step 15,000. The selected artifact is step 18,000. Resume restores model/optimizer/normalization but starts a new seeded RNG sequence.

[Dataset manifest](artifacts/final_dataset_v1/manifest.json), [coverage](artifacts/final_dataset_coverage.json), [motion statistics](artifacts/final_motion_statistics_v2.json), and [integrity audit](artifacts/final_integrity_audit.json) provide the counts, distributions and hashes. The selected data's sampled TCP-speed maximum is approximately 0.052 m/s; median path length is 0.338 m. These sampled measurements omit sub-tick peaks and are not hardware limits.

## Validation selection and final test

| Configuration | Validation result |
|---|---:|
| History seed 1, step 18,000, fixed prefix 5 | 14/20 |
| Same checkpoint, fixed prefix 10 | 17/20 |
| Same checkpoint, prefix 20 / transition prefix 5 | 20/20 |

These are exploratory selection results on the same validation starts. They are not independent evidence of generalization or a robust multi-seed causal comparison. The final test was run once for the frozen candidate.

The 100 final starts were selected by expert/path feasibility from 128 candidates before learned-policy testing, then rechecked 100/100 under coordinated joint limiting. Results therefore apply to this **expert-feasible, in-range, combined-DR benchmark**, not arbitrary unfiltered starts or blind search. [Start-selection audit](evaluations/test_seeds.audit.json).

| Final metric | Result |
|---|---:|
| Primary successes | 90/100 |
| 95% Wilson interval | 82.6–94.5% |
| Stricter legacy-style successes | 83/100 |
| Reset errors / IK failures / nonfinite episodes | 0 / 0 / 0 |
| Episodes with unintended contacts | 0 |
| Maximum joint speed | 0.748 rad/s |
| Maximum penetration over all trials | 4.135 mm |
| Clipping events | 5,901 |

Primary success requires orientation-aware bottle clearance ≥10 cm for two continuous seconds within 12 seconds, with no unintended contacts, nonfinite state, IK failure, excessive joint speed or penetration ≥3 mm. Ten trials failed: seven did not complete the lift/hold in time and four exceeded the penetration limit; one trial failed both checks. The penetration maximum includes failed trials. Cartesian and joint limiting can each increment the clipping counter at one policy tick, so it is not a count of distinct actions.

The legacy-style metric additionally checks >15 cm minimum clearance and >95% bilateral contact over the final two seconds, with actuator-limit validity. It remains separate from primary success. No threshold was relaxed during selection or final testing.

[Final summary](evaluations/final_test_v1/summary.json), [all trial results](evaluations/final_test_v1/results.json), and [independent success recomputation](artifacts/final_test_semantics_audit.json). Rollout videos are saved in the final-test shard directories.

## Domain checks and demonstration comparison

The ancillary suite completed all 91 predeclared trials using new fixed seeds 30000–30009, disjoint from training, validation and final testing.

| Condition | Policy | Expert reference |
|---|---:|---:|
| Deterministic nominal | 1/1 | 1/1 |
| Nominal assets/appearance, wider starts | 6/10 | 10/10 |
| Bottle pose | 7/10 | 7/10 |
| Table pose | 6/10 | 9/10 |
| Base pose | 5/10 | 9/10 |
| Camera | 6/10 | 10/10 |
| Lighting | 7/10 | 10/10 |
| Materials | 6/10 | 10/10 |
| Image noise | 8/10 | 10/10 |
| Camera deviations expanded to 1.5× | 6/10 | 10/10 |

**These weaker condition results limit the 90/100 claim to its specific benchmark.** Nominal appearance disables asset-pose and appearance/camera DR while retaining wider robot starts; it is not an appearance-only intervention on the combined test. Individual appearance/camera conditions share exactly verified physical states with that baseline. Their expert reference is reused across 50 exact physical-state comparisons.

All predeclared starts remain in the denominator, including three bottle, one table and one base reset errors. There were no unintended contacts or nonfinite states among the 86 non-error trials. Expert-successful subsets are reported separately; an expert failure does not prove a start unsolvable. Ten trials per condition provide wide uncertainty intervals and do not establish causal benefits from any DR family. See [the complete domain report](evaluations/domain_suite_v1/REPORT.md) and [per-trial diagnostics](evaluations/domain_suite_v1/report.json).

The small matched demonstration pilot used 15 training and four validation episodes per variant, with one training seed and equal update budgets. Randomized success was 1/10 fixed-path, 2/10 diverse, and 1/10 diverse plus approach detour. It does not establish a reliable benefit from diversity. A 0.125-pixel legacy Gaussian camera offset is an additional confound; actual empty-grasp recovery was included in full training, not this matched detour comparison. [Comparison report](evaluations/matched_comparison/REPORT.md).

## Verification and throughput

- All 16 contract, history, masking and aggregation tests pass. Six signed EE response probes agree with the documented axes; direction cosines exceed 0.998.
- Nine selected trajectory replays, including every source/controller group and motion/visibility extrema, reproduce qpos, qvel and policy state with zero error.
- Shared native/hybrid camera projection and coherent body transforms pass on 28 feasible randomized resets, including the separate expanded-camera audit.
- All five dataset audits pass. A fresh integrity check matches all 314 selected files and 11 frozen source files.
- Fifty reset frames, recovery strips and three paired native-overview/recorded-wrist videos were inspected. [Overview videos and notes](artifacts/demonstration_overviews/README.md).

Isolated policy inference had **9.3 ms p95 and 11.2 ms maximum** across 148 refreshes in 2,000 measured calls, after 200 warm-up ticks. No call exceeded 50 ms. This includes preprocessing and CPU action output, but excludes camera acquisition, simulation, IK and hardware transport. [Inference benchmark](artifacts/selected_policy_latency.json).

Rendering measured **15.6 FPS native and 10.3 FPS hybrid** at 320×200 over 120 measured frames each. GPU 0 was idle for this measurement while GPU 1 continued evaluation. Simulation is below real time at its 20 Hz virtual policy clock; inference timing is a separate measurement. [Renderer benchmark](artifacts/selected_renderer_latency.json).

## Export and completion

The final [policy bundle](exports/bottle_pick_act_v1/README.md) contains inference weights, local runtime/model code, preprocessing/normalization, the control contract, provenance, evaluation reports and eight final-test videos. [Independent verification](artifacts/final_export_verification.json) checked 33 file hashes and exact agreement for all 240 reference actions with an empty PYTHONPATH, both execution prefixes and episode reset. Exported deltas still require the bounded Cartesian/IK controller adapter; this is an inference bundle, not an RTDE driver.

The evaluated source checkpoint SHA256 is `3e08006112cf3f071eb6548495ac35dde00e03dbe2374b37e37a2cc8785290d3`. Neither policy nor frozen simulation behavior changed after selection. See [the completion audit](PHASE1_COMPLETION_AUDIT.md) and [reproduction commands](README.md).

## Limitations and later work

The simulated position servo is an approximation of servoJ, not a calibrated hardware model. Gripper width/force response, contact parameters, TCP and hand-eye calibration remain approximate. Mount external collisions are disabled. Image noise is not a calibrated temporal D455 model. Gaussian lighting remains baked, and some wrist views expose black regions in the reconstruction. These limitations must accompany any transfer claim.

Phase 1 makes no real-robot connection. Before hardware execution, validate TCP/hand-eye calibration, distortion preprocessing, gripper mapping, camera timing, bounded IK and the RTDE transport/servo settings. Phase 2 contains Gaussian appearance DR, additional policy families and controlled improvements; prioritize weaknesses found by the completed domain checks before making a hardware-readiness claim.