Key takeaways
The through-line
A self-improving system is two loops: the inner one that does the work, and the outer one that changes how the inner one works. The whole discipline is making the outer loop safe — because a system that edits itself without evidence and without brakes degrades faster than it improves.
The ideas that carry
- The dual-loop architecture is the frame. Inner loop executes; outer loop observes, proposes, validates, promotes. Nothing changes without passing through the outer loop's evidence gate.
- Structured execution logging is the precondition. You cannot improve what you did not record in a form a program can analyse — free-text logs are a dead end here.
- Failure analysis is pattern extraction, not anecdote. Cluster failures, find the recurring shape, and fix the class rather than the instance.
- Automated prompt optimisation works when it is scored. APE-style search is only as good as the evaluation behind it; without a golden set it optimises toward its own noise.
- Harness patching is a bigger, riskier lever than prompt patching and needs a correspondingly stronger gate — proposals, shadow runs, staged rollout, one-click revert.
- Principle extraction turns fixes into durable knowledge so the same lesson isn't relearned every fortnight.
- *Topology evolution — changing the shape of the workflow — is the deepest lever and the last one you should unlock.*
- Governance is what separates this from a system that quietly rewrites its own safety rules. Immutable constraints, human approval on the outer loop, and a full audit of every self-applied change.
What you can now do
Instrument a system so its own failures are analysable. Run automated prompt optimisation against a real eval. Stage and gate self-applied changes with a revert path. Write governance rules a self-modifying system genuinely cannot edit.
If one thing sticks
Self-improvement without measurement is just drift with extra steps. The evaluation is the system; the optimiser is the easy part.