> Historical bottle-lift-v3 report. Current goals are sampled around a reachable nominal TCP and a state teacher task is available; see [current report](teacher-report.md).

# MJLab lift-cube reward integration

The canonical bottle task now directly takes its reward dictionary and reward
curriculum from the local pinned MJLab `make_lift_cube_env_cfg()`
(commit `8ee51fbcf806a7419189f706d9e394cbeb7790fa`). No MJLab source changes
were needed for this update. No RL training was run.

## Reward recipe

| Upstream function / term | Defaults |
|---|---|
| `staged_position_reward` / `lift` | Weight +1; reach std 0.2 m, bring std 0.3 m |
| `bring_object_reward` / `lift_precise` | Weight +1; std 0.05 m |
| `action_rate_l2` | Weight −0.01 |
| `joint_pos_limits` | Weight −10; all robot joints |
| `joint_velocity_hinge_penalty` | Speed threshold 0.5 rad/s; all robot joints |

The native joint-speed penalty curriculum changes its weight from −0.01 to −0.1
to −1 at vector steps 0 / 12,000 / 24,000, applied by MJLab on reset. Rewards
are rates integrated over control dt (0.05 s). The staged term is
`exp(-reach_distance²/0.2²) * (1 + exp(-goal_distance²/0.3²))`.
The precise term is `exp(-goal_distance²/0.05²)`.

The two object selectors become `bottle`, and the reaching site becomes `tcp`.
A small `LiftingCommand` subclass samples an episode-fixed target directly above
the freshly reset bottle root by `objective.high_lift_height_m` (15 cm default).
It reads GPU free-joint qpos before forward, never teleports the object, and
respects randomized table/object placement and partial resets. Its target marker
uses inherited MJViser debug visualization.

The previous custom reach/grasp/lift/hold terms, contact/motion costs and
200-point success bonus are removed. Our strict 10-cm, two-pad, stable 2-second
hold success and all validity/drop/high-lift metrics remain. Success no longer
ends the episode by default: sustained holding keeps receiving dense reward.
The inherited command's 5-cm proximity diagnostics are separately named and
must not be interpreted as grasp-and-hold success. Proximity shaping can reward
invalid manipulation; training must establish whether it learns the intended task.

## Interface and compatibility

Actor state stays 16 values plus wrist RGB; action stays 7 values. The critic
has 62 values, appending bottle-to-goal world displacement to the previous 59.
The actor is for this fixed vertical-lift task, not an arbitrary goal-conditioned
policy. Task spec `bottle-lift-v3` includes function names, reward parameters,
goal command and penalty schedule. Strict full resume rejects older task specs;
actor-only loading still uses simulator contract `mjlab-gs-cartesian-v2`.

See the [complete task specification](task.md) for observation ordering,
DR ranges, success conditions and known simulation limitations.

## Validation

- Eight unit tests pass, including direct upstream function identity, reward
  ordering for approach/lift, task-spec sensitivity, and existing success guards.
- Four full-DR GPU worlds: finite actor `[4,16]`, critic `[4,62]`, RGB
  `[4,3,128,128]`; targets match randomized bottle starts.
- Partial resets preserve other worlds' goal and state. Twenty nominal control
  steps produce finite CUDA rewards and preserve the episode-fixed target.
- Native penalty curriculum verified immediately before/at both boundaries;
  saved declared task spec remains stable as runtime weights change.
- Timeout autoresets refresh goals and retain strict terminal success logging.
- Real browser controls and wrist feed are rechecked after restarting the viewer.

These checks establish task plumbing, not learned performance. Historical DR,
visual-parity and fixed-quota evaluation measurements remain under `reports/domain/`
and retain their original version-2 reward and critic settings.

## Current throughput

| Configuration | Worlds | RGB | Environment steps/s | Process GPU GiB |
|---|---:|---|---:|---:|
| Nominal | 64 | 128×128 | 754.1 | 1.95 |
| Full DR | 64 | 128×128 | 570.9 | 1.70 |

RTX A6000, 40 timed vector steps after five warmup steps, no learned policy or
optimizer. Includes Cartesian IK, 50 physics substeps, contact metrics, native
reward/command managers, Gaussian/native RGB and observation conversion. Process
memory is sampled through NVML. Different randomized scene occupancy/reset
frequency affects these short measurements; they do not establish maximum
capacity or a controlled speedup over older runs. All reported final observations
are finite. GPU arithmetic is retained; normal MJLab orchestration and command/
reset logging still involve host synchronization.