DrEureka
Eureka's reward loop extended to sim-to-real: an LLM also proposes the simulator's randomization settings so a trained robot policy works on hardware.
The loop
- Proposes
Reward code (as in Eureka, with safety instructions) and then domain randomization configurations from GPT-4
- Judged by
RL training in simulation, plus a physics-prior step (RAPP) that bounds each randomized parameter
- Steers
Reward stage: best result feeds back into the prompt. Randomization stage: open loop, no real-world feedback
Headline result
On a real quadruped forward-locomotion test (2 m/s target, 5 m track), the best DrEureka policy reached 1.83 m/s and 5.00 m versus 1.32 m/s and 4.17 m for the human-designed setup. The robot also walked atop a real yoga ball for 15.4 seconds on average.
What this does not prove
Only the reward stage closes a loop with simulation results, and the randomization stage is a one-shot proposal. The real-world numbers come from short hardware trials, and the yoga-ball runs are about 15 seconds, not minutes.
How it generalizes the harness
The LLM designs training settings offline in simulation, and the resulting policy runs on the robot without the model.
Classification
| What it proposes | Reward function |
|---|---|
| Generative model | GPT-4, frozen |
| Algorithmic authority | Physics simulation with RL training and a reward-aware physics prior |
| Feedback | Written reflection |
| What gets updated | Prompt context |
| When it runs | Offline, at design time |
| Validity | Empirical (measured) |
| Evidence | Experiments reported |
| Lineage | Successor |