FunBO
Extends FunSearch to discover novel Bayesian Optimization acquisition functions in Python code, evaluated across black-box optimization suites.
The loop
- Proposes
An LLM mutates Python acquisition functions mapping Gaussian Process posterior mean and variance to utility scores.
- Judged by
A Bayesian Optimization benchmark harness runs full GP optimization loops with each candidate acquisition function on synthetic and hyperparameter tasks.
- Steers
An island-model evolutionary database ranks acquisition functions by average simple regret and seeds the next prompt.
Headline result
Discovered acquisition functions that outperform Expected Improvement (EI) and Upper Confidence Bound (UCB) on unseen black-box optimization and HPO benchmarks.
What this does not prove
Evaluating each candidate acquisition function requires running multiple full Bayesian Optimization trajectories during the offline search.
How it generalizes the harness
- The algorithm is the solver
- Inverted control
- The output outlives the model
- Search over a population
- AI at design time
- Intra-algorithmic surgery (inside the while-loop)
An evolutionary island algorithm calls the LLM offline to discover closed-form Python acquisition functions that subsequently run inside standard BO solvers.
Classification
| What it proposes | Heuristic |
|---|---|
| Generative model | Code-generating LLM inside an island evolutionary loop |
| Algorithmic authority | Gaussian Process Bayesian Optimization benchmark evaluator |
| Feedback | Score |
| What gets updated | Population of candidates |
| When it runs | Offline, at design time |
| Validity | Empirical (measured) |
| Evidence | Experiments reported |
| Lineage | Derivative |