LLMorpheus
Mutation testing framework where an LLM proposes fault-injecting code mutants by filling placeholders in JavaScript source; mutants are run against the project test suite.
The loop
- Proposes
An LLM is prompted at placeholders in chosen code locations and suggests replacement code, producing mutants of JavaScript packages.
- Judged by
The project test suite is run against each mutant to determine whether it is killed or survives.
- Steers
Surviving mutants expose concrete gaps in the existing test suite.
Headline result
On 13 real-world JavaScript packages, produced mutants resembling real bugs that StrykerJS, a state-of-the-art mutation testing tool, cannot produce.
What this does not prove
Distinguishing a genuinely surviving bug-revealing mutant from a semantically equivalent refactoring still requires diff inspection.
How it generalizes the harness
An automated mutation-testing harness compiles and runs the test suite over every proposed AST mutant to classify killed vs. surviving faults.
Classification
| What it proposes | Code patch |
|---|---|
| Generative model | Code-generating LLM mutant operator |
| Algorithmic authority | Compiler and automated mutation-testing execution engine |
| Feedback | Test results |
| What gets updated | Population of candidates |
| When it runs | Offline, at design time |
| Validity | Empirical (measured) |
| Evidence | Code released |
| Lineage | Original system |