What self-improvement means

A fixed system uses the same prompt, the same tool definitions, the same routing logic, and the same workflow topology on every request. Its performance ceiling is set at deploy time. If the system prompt scores 72% accuracy on a classification task today, it will score 72% next month — unless a developer manually rewrites it. Every improvement requires a human in the loop: notice the failure, diagnose the root cause, edit the prompt, test, deploy.

An adaptive system modifies its own operational parameters based on production feedback. The system prompt that scored 72% on day one evolves to 81% by day 30, because the system analyzed its own failures, generated candidate improvements, tested them against held-out data, and deployed the winners. The developer's role shifts from writing prompts to defining objectives and review gates.

Two improvement modes

Online adaptation adjusts the system within a single session, mid-conversation. The agent observes that its first attempt at a code review missed a null-pointer bug, receives user feedback ("you missed the NPE on line 34"), and updates its internal checklist for the remainder of the session. The change is ephemeral — it does not persist after the session ends.

from openai import OpenAI

client = OpenAI()

class AdaptiveAgent:
    def __init__(self, base_prompt: str):
        self.base_prompt = base_prompt
        self.session_patches: list[str] = []

    def build_prompt(self) -> str:
        if not self.session_patches:
            return self.base_prompt
        patches = "\n".join(f"- {p}" for p in self.session_patches)
        return f"{self.base_prompt}\n\nLearned corrections (this session):\n{patches}"

    def respond(self, user_message: str, conversation: list[dict]) -> str:
        messages = [
            {"role": "system", "content": self.build_prompt()},
            *conversation,
            {"role": "user", "content": user_message},
        ]
        response = client.chat.completions.create(
            model="gpt-4o", messages=messages, temperature=0
        )
        return response.choices[0].message.content

    def apply_feedback(self, correction: str):
        self.session_patches.append(correction)

Online adaptation is low-risk because changes are scoped to one user and one session. The cost is the extra tokens in the system prompt — each patch adds ~20-50 tokens. Ten patches add 200-500 tokens, costing an extra 500 * $2.50 / 1e6 = $0.00125 per request at GPT-4o input pricing.

Offline evolution analyzes patterns across many requests between sessions and produces permanent changes. A nightly job reads the day's logs, identifies systematic failure patterns (the agent consistently misclassifies sarcastic reviews as positive), generates candidate prompt revisions, evaluates them against a test set, and stages the winner for human review. The change persists across all future sessions for all users.

The distinction maps to biological analogy: online adaptation is phenotypic plasticity (an organism adjusting within its lifetime), offline evolution is genetic change (the population shifts across generations). Both are necessary. Online adaptation handles novel situations the system was not designed for. Offline evolution raises the baseline so fewer situations require online adaptation.

What can improve

Four categories of operational parameters are candidates for automated improvement:

Prompts. The system prompt, few-shot examples, chain-of-thought templates, and output format instructions. This is the highest-leverage target — prompt changes can shift accuracy by 5-20 percentage points (Zhou et al. 2022) with zero infrastructure changes.
Tool configurations. Tool descriptions, parameter schemas, and default values. An agent that misroutes 15% of queries to the wrong tool may have a description problem, not a model problem. Rewriting "search stuff" to "Search the product catalog by SKU, name, or category. Returns: product_id, name, price_usd, in_stock" can raise correct tool selection from ~80% to ~97%.
Routing rules. Which model handles which query type. A router that sends simple FAQ queries to GPT-4o-mini ($0.15/M input) instead of GPT-4o ($2.50/M input) saves 94% on those requests. The routing threshold — the confidence score below which a query escalates to the larger model — is a tunable parameter that the system can optimize.
Workflow topology. Which agents or processing steps run, and in what order. A RAG pipeline might discover that adding a reranking step between retrieval and generation improves answer quality by 12% on factual queries but adds 200ms latency. The system can learn to conditionally insert the reranking step only when the query is factual.

What should not improve without oversight

Automated improvement must not touch parameters where a wrong change creates safety, ethical, or security failures:

Safety constraints. Content filtering thresholds, toxicity detection sensitivity, PII redaction rules. A prompt optimization run that discovers "ignoring safety instructions increases user satisfaction scores by 8%" would be catastrophically wrong.
Ethical policies. Refusal criteria, bias mitigation rules, fairness constraints. These are value judgments, not performance parameters.
Data access permissions. Which databases, APIs, and file systems the agent can read or write. An optimizer that grants the agent access to a production database to improve answer accuracy has made a security decision, not a performance decision.
Authentication scopes. OAuth scopes, API key permissions, user impersonation capabilities. Expanding access to improve functionality is a privilege escalation.

The boundary is crisp: if a change could cause harm that is not captured by the optimization metric, it requires human review regardless of the measured improvement. Performance metrics do not measure safety — they measure task completion.

The meta-agent pattern

A self-improving system has two distinct agents: the production agent (handles user requests) and the meta-agent (proposes improvements to the production agent). The meta-agent reads production logs, analyzes failure patterns, generates candidate changes, and tests them in a staging environment. It never directly modifies the production system.

import json
from datetime import datetime

class MetaAgent:
    def __init__(self, production_config: dict, log_store, test_set: list[dict]):
        self.production_config = production_config
        self.log_store = log_store
        self.test_set = test_set
        self.client = OpenAI()

    def analyze_failures(self, window_hours: int = 24) -> list[dict]:
        logs = self.log_store.get_recent(hours=window_hours)
        failures = [l for l in logs if l["quality_score"] < 0.7]
        if not failures:
            return []

        failure_summary = json.dumps(failures[:20], indent=2)
        response = self.client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": (
                    "Analyze these agent failures. Identify recurring patterns "
                    "and categorize by root cause. Return JSON array of "
                    "{pattern, frequency, example_input, suggested_fix}."
                )},
                {"role": "user", "content": failure_summary},
            ],
            temperature=0,
            response_format={"type": "json_object"},
        )
        return json.loads(response.choices[0].message.content)["patterns"]

    def generate_candidate(self, failure_patterns: list[dict]) -> dict:
        current_prompt = self.production_config["system_prompt"]
        patterns_text = json.dumps(failure_patterns, indent=2)

        response = self.client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": (
                    "You are a prompt engineer. Given the current system prompt "
                    "and recurring failure patterns, generate an improved prompt "
                    "that addresses the failures without regressing on working cases. "
                    "Return JSON: {improved_prompt, changes_made, rationale}."
                )},
                {"role": "user", "content": (
                    f"Current prompt:\n{current_prompt}\n\n"
                    f"Failure patterns:\n{patterns_text}"
                )},
            ],
            temperature=0.7,
            response_format={"type": "json_object"},
        )
        candidate = json.loads(response.choices[0].message.content)
        candidate["created_at"] = datetime.utcnow().isoformat()
        candidate["created_by"] = "meta-agent"
        return candidate

    def evaluate_candidate(self, candidate_prompt: str) -> dict:
        correct = 0
        for case in self.test_set:
            response = self.client.chat.completions.create(
                model="gpt-4o",
                messages=[
                    {"role": "system", "content": candidate_prompt},
                    {"role": "user", "content": case["input"]},
                ],
                temperature=0,
            )
            predicted = response.choices[0].message.content
            if self._matches(predicted, case["expected"]):
                correct += 1
        return {"accuracy": correct / len(self.test_set), "n": len(self.test_set)}

    def _matches(self, predicted: str, expected: str) -> bool:
        return expected.lower().strip() in predicted.lower().strip()

The meta-agent has read access to production logs and write access to a staging environment. It proposes changes as diffs — a candidate prompt paired with its test results — that a human reviews before deployment. The separation ensures that the production agent's behavior changes only through a controlled pipeline: observe, hypothesize, test, review, deploy.

The cost of running the meta-agent is infrastructure overhead. Analyzing 100 failure logs and generating 3 candidate prompts costs ~$0.30-0.50 in API calls. Evaluating each candidate against a 50-example test set adds ~$0.50-1.00 per candidate. A daily optimization cycle costs $2-5 — a rounding error against production LLM spend for any system handling >1,000 requests/day.