ShinkaEvolve
Open-source program evolution that aims to need fewer LLM samples: careful parent choice, near-duplicate rejection, and a bandit that picks the LLM.
The loop
- Proposes
Diff edits, rewrites or crossovers of programs, written by an ensemble of LLMs
- Judged by
Task-specific evaluators that score each program and return text feedback
- Steers
Parents mix score and novelty across islands; an embedding filter rejects repeats; a UCB1 bandit picks the LLM
Headline result
ShinkaEvolve found a circle-packing (n=26) solution with sum of radii 2.63598 under its relaxed verifier, beating AlphaEvolve's reported solution within 150 program evaluations. It also improved ALE-Agent solutions on ALE-Bench by about 2.3% on average over 10 tasks, designed an AIME agent scaffold (gpt-4.1-nano, 7 LLM calls), and found a new mixture-of-experts load-balancing loss.
What this does not prove
The paper uses both '150 samples' and '150 generations' for the circle-packing run. The MoE loss was evolved at small scale (556M parameters, 2B tokens) and the authors note limited budget and similar architecture between search and test. AIME gains were smaller on 2023 problems, which the authors link to possible contamination.
How it generalizes the harness
Parent sampling, novelty filtering and bandit model selection are algorithmic controls over a population, and the evolved programs and loss functions are artifacts that run without the model.
Classification
| What it proposes | Program |
|---|---|
| Generative model | LLM ensemble, frozen (claude-sonnet-4, o4-mini, gpt-4.1 family; gpt-5-nano for meta and novelty) |
| Algorithmic authority | Task evaluators plus an embedding-similarity filter and UCB1 bandit |
| Feedback | Score |
| What gets updated | Prompt context and population of candidates |
| When it runs | At inference time |
| Validity | Empirical (measured) |
| Evidence | Experiments reported |
| Lineage | Derivative |