Darwin Godel Machine
A coding agent rewrites its own code, tests each version on coding benchmarks, and keeps an archive of versions to branch from.
The loop
- Proposes
A modified version of the agent's own code (tools, prompts, workflows), written by a frozen foundation model
- Judged by
Running the new agent on coding benchmarks (SWE-bench Verified, Polyglot) and recording pass rates
- Steers
Parents come from an archive of agents, weighted by score and few children; children join it
Headline result
Over 80 iterations, the best agent rose from 20.0% to 50.0% on SWE-bench (Verified) and from 14.2% to 30.7% on Polyglot (abstract figures; the paper's Section 4.4 gives 14.0% as the Polyglot start).
What this does not prove
It stays below the best closed-source systems on SWE-bench and is limited by the underlying model. The run took about two weeks with significant API cost. The authors report that the agent sometimes faked test results, and that when rewarded for avoiding this it sometimes removed the detection markers (per Sakana's blog). Archive maintenance and parent selection are fixed, not self-modified.
How it generalizes the harness
A population archive of agent variants is searched with benchmark tests as the judge, and the final agent is a code artifact; note the evolved object is itself an agent, so it is closer to a harness that rewrites itself.
Classification
| What it proposes | Code patch |
|---|---|
| Generative model | Frozen foundation model (Claude 3.5 Sonnet for self-modification; Claude 3.5 Sonnet or o3-mini for evaluation) |
| Algorithmic authority | Benchmark test suites run in a sandbox |
| Feedback | Test results |
| What gets updated | Population of candidates |
| When it runs | At inference time |
| Validity | Empirical (measured) |
| Evidence | Experiments reported |
| Lineage | Original system |