Self-Improving AI Systems — Core

Why

A fixed system uses the same prompt, tool definitions, routing logic, and workflow topology on every request. If the system prompt scores 72% accuracy today, it scores 72% next month unless a developer manually rewrites it. Every improvement requires a human in the loop. A self-improving system modifies its own operational parameters based on production feedback. The system prompt that scored 72% on day one can evolve to 81% by day 30 because the system analyzed its own failures, generated candidate improvements, tested them against held-out data, and deployed the winners. The developer's role shifts from writing prompts to defining objectives and review gates. The distinction is not about multiple competing models, synchronous versus asynchronous processing, or distributed server architecture. Self-improvement specifically refers to the system's ability to modify its own operational parameters such as prompts, tool configurations, routing rules, and workflow topology through an automated feedback loop.

Why

Online adaptation adjusts the system within a single session based on mid-conversation feedback. When a user corrects the agent by saying 'you missed the null pointer on line 34,' the agent adds this correction to its internal checklist for the remainder of the session. The change is ephemeral and does not persist after the session ends. The cost is minimal: each patch adds roughly 20-50 extra tokens to the system prompt. Offline evolution analyzes patterns across many requests between sessions, typically in a daily batch job. It identifies systematic failure patterns, generates candidate improvements, tests them against a held-out dataset, and stages the winners for human review. Changes from offline evolution are permanent and affect all future sessions for all users. The analogy is biological: online adaptation is phenotypic plasticity within a lifetime, offline evolution is genetic change across generations. Neither involves gradient updates to model weights, GPU versus CPU distinction, or text versus multimodal processing differences.

Why

Four categories of operational parameters are candidates for automated improvement. Prompts including system prompts, few-shot examples, and chain-of-thought templates are the highest-leverage target since prompt changes can shift accuracy by 5-20 percentage points with zero infrastructure changes. Tool configurations including descriptions, parameter schemas, and default values can raise correct tool selection from approximately 80% to 97% by improving description quality. Routing rules determining which model handles which query type can save up to 94% on simple queries by routing them to cheaper models. Workflow topology determining which processing steps run and in what order can conditionally insert steps like reranking that improve quality on specific query types. These parameters can be modified without retraining the model. Model weights, attention patterns, and vocabulary are internal model parameters that require retraining. Dataset splits are training infrastructure, not operational parameters. Infrastructure parameters like API endpoints and database connections are not performance parameters.

Why

The boundary between automatable and non-automatable parameters is crisp: if a change could cause harm that is not captured by the optimization metric, it requires human review regardless of the measured improvement. A prompt optimization run that discovers 'ignoring safety instructions increases user satisfaction scores by 8%' would be catastrophically wrong because the satisfaction metric does not capture the harm of removing safety constraints. Content filtering thresholds, toxicity detection sensitivity, PII redaction rules, refusal criteria, bias mitigation rules, and data access permissions are all parameters where a wrong change creates safety, ethical, or security failures that task completion metrics cannot measure. These are value judgments, not performance parameters. The exclusion is not primarily about regulatory compliance, though regulations may reinforce it. It is not about hardware limitations or API immutability. The fundamental issue is that optimization metrics are narrow measurements of specific objectives, and safety concerns are orthogonal to those objectives.

Why

The meta-agent pattern separates the self-improving system into two distinct agents. The production agent handles user requests through the task loop. The meta-agent proposes improvements to the production agent by reading production logs, analyzing failure patterns, generating candidate changes, and testing them in a staging environment. Critically, the meta-agent never directly modifies the production system. It has read access to production logs and write access to the staging prompt registry, but no access to user data or production endpoints. Changes flow through a controlled pipeline: observe failures, hypothesize improvements, test candidates, submit for human review, and deploy through canary rollout. The cost is modest: analyzing 100 failure logs and generating 3 candidate prompts costs approximately $0.30-0.50 in API calls, and evaluating each candidate against a 50-example test set adds $0.50-1.00 per candidate. A daily optimization cycle costs $2-5, which is negligible against production LLM spend for systems handling over 1,000 requests per day.