What Is a Harness

Every agent system has two loops. The inner loop is where the model decides what to do next — it reads context, selects a tool, interprets the result, and decides whether to continue. The outer loop is your code deciding when to invoke that inner loop, what to feed it, how to recover from failure, and when to stop. The outer loop is the harness.

The harness wraps the model call. The model does not wrap the harness. This asymmetry is the foundational architectural decision. Your code is the entry point, the state owner, and the recovery authority. The model is a stateless function called within that structure. When a model API call returns, the harness resumes control unconditionally — even if the model's response is malformed, truncated, or empty. The harness never cedes control flow to the model.

The six components of a harness

Task queue: an ordered set of work items the harness feeds to the inner loop one at a time, or in dependency order. Can be as simple as a Python list or as complex as a priority queue with dependency DAGs. The queue determines the macro-level execution plan — the model handles micro-level decisions within each task. A well-designed queue enables parallelism (independent tasks on separate workers) and resumability (crash recovery picks up at the next unfinished item).
Context builder: assembles the prompt from stable prefixes (system prompt, tool definitions), dynamic middle layers (retrieved documents, prior results), and the growing conversation tail. Owns token budget allocation. The context builder is the most performance-critical component — it runs before every model call, and its token-counting accuracy determines whether you waste budget on unnecessary content or crash from overflow.
Evaluator: determines whether the agent's output satisfies the task's success criteria. Can be a simple schema check, a test suite execution, or a second model call with adversarial framing. The evaluator is the harness's quality gate — without it, the loop runs until a hard limit (turn count, token budget) is hit, producing unpredictable output quality.
State persistence: serializes the full agent state (conversation history, pending tasks, completed results, token counters) so the harness can resume after crashes, deploys, or timeouts. Implementations range from in-memory dictionaries (prototyping) to Redis/PostgreSQL (production). State must be serializable — no closures, no live connections, no thread-local data.
Recovery: codified policies for every failure mode — rate limits, tool errors, output truncation, model refusals, context overflow. Each policy is a branch in a state machine, not an exception handler. Recovery policies must be deterministic: given the same failure state, the same recovery action fires. This makes debugging possible — you can replay a failure from logs and verify the recovery path.
Observability: structured logs of every loop iteration — input tokens, output tokens, tool calls, latency, cost, evaluator verdicts, cache hit rates. Without this, debugging a 40-turn agent run is forensic archaeology. A production harness emits OpenTelemetry spans or structured JSON logs at minimum. Every model call gets a unique trace ID linking it to the parent task, iteration number, and wall-clock time.

Claude Code's architecture as reference

Claude Code's core loop (the query.ts module, as described in Anthropic's engineering blog and the open-source release, 2025) implements a pure state machine. The harness maintains an immutable parameters object and a mutable state object. Immutable parameters include the system prompt, model ID (claude-sonnet-4-20250514), available tool definitions, max output tokens (16,384), and the permission configuration. Mutable state includes the message history, tool execution results, abort signals, and token accounting.

Each iteration of the loop follows this path:

assemble_messages(state) → call_model(params, messages) → stream_response → dispatch_tools → update_state → check_exit → loop

There is no recursion. There is no callback hell. The loop is a while with explicit continue conditions. The state machine has named transitions: RUNNING, TOOL_EXECUTING, AWAITING_CONTINUE, DONE, ERROR. Every transition is logged with a timestamp, token count delta, and the triggering condition.

The streaming architecture deserves attention. Claude Code streams responses token-by-token. As tool-use blocks arrive, they are parsed incrementally — the harness doesn't wait for the full response before beginning tool dispatch. This means tool execution can start before the model finishes generating its reasoning text, shaving 1-3 seconds per turn on complex responses. The implementation uses an event-emitter pattern: the streaming parser emits tool_start, tool_input_delta, tool_input_complete events that trigger dispatch.

from dataclasses import dataclass, field
from enum import Enum
from typing import Any

class LoopState(Enum):
    ASSEMBLING = "assembling"
    CALLING = "calling"
    STREAMING = "streaming"
    TOOL_EXECUTING = "tool_executing"
    EVALUATING = "evaluating"
    RECOVERING = "recovering"
    DONE = "done"
    ERROR = "error"

@dataclass(frozen=True)
class HarnessParams:
    system_prompt: str
    model: str
    tools: list[dict]
    max_turns: int = 50
    token_budget: int = 180_000
    max_output_tokens: int = 16_384
    temperature: float = 0.0

@dataclass
class HarnessState:
    messages: list[dict] = field(default_factory=list)
    current_state: LoopState = LoopState.ASSEMBLING
    turn_count: int = 0
    total_input_tokens: int = 0
    total_output_tokens: int = 0
    cache_read_tokens: int = 0
    cache_creation_tokens: int = 0
    task_queue: list[str] = field(default_factory=list)
    completed_tasks: list[str] = field(default_factory=list)
    pinned_context: list[str] = field(default_factory=list)

def run_harness(params: HarnessParams, state: HarnessState, task: str):
    state.messages.append({"role": "user", "content": task})
    
    while state.current_state != LoopState.DONE:
        state.current_state = LoopState.ASSEMBLING
        context = build_context(params, state)
        
        state.current_state = LoopState.CALLING
        response = call_model(params.model, context, params.max_output_tokens)
        
        state.total_input_tokens += response.usage.input_tokens
        state.total_output_tokens += response.usage.output_tokens
        state.cache_read_tokens += response.usage.cache_read_input_tokens
        state.cache_creation_tokens += response.usage.cache_creation_input_tokens
        
        if response.has_tool_calls:
            state.current_state = LoopState.TOOL_EXECUTING
            results = execute_tools(response.tool_calls, timeout=30)
            state.messages.extend(format_tool_results(results))
        else:
            state.messages.append({"role": "assistant", "content": response.content})
        
        state.current_state = LoopState.EVALUATING
        state.turn_count += 1
        
        if should_stop(params, state, response):
            state.current_state = LoopState.DONE
    
    return state

Note the frozen=True on HarnessParams. This is enforced immutability — any attempt to mutate params during the loop raises FrozenInstanceError. A bug that accidentally modifies the system prompt on turn 15 of a 30-turn session is catastrophic and nearly impossible to diagnose from output alone.

The agent vs. the harness

The agent is the model plus its tools — the reasoning engine that decides what action to take given context. The harness is everything else: the loop control, the context assembly, the recovery logic, the evaluation, the persistence, the observability. The agent is a function. The harness is the program that calls that function.

This distinction matters because the highest-leverage code in any agent system is the harness, not the model. Anthropic's research (presented at their developer conference, 2025) demonstrated that a well-designed harness with structured recovery, context management, and evaluation around Claude 3.5 Haiku consistently outperformed Claude 3.5 Sonnet used zero-shot on SWE-bench style tasks. The weaker model with a production harness scored 38% on their internal code-repair benchmark; the stronger model without scaffolding scored 24%. The harness contributed more than the 62% price increase between the two models.

This result has been replicated externally. The Aider project (Gauthier 2024) showed that their edit-format harness — which structures model output into parseable diff blocks, validates syntax, and retries on parse failure — improved benchmark scores by 15-25 percentage points independent of the underlying model. The harness was the variable, not the model.

The implication is economic: improving your harness — better context assembly, better recovery policies, better evaluation — has higher expected ROI than switching to a more expensive model. The model is a commodity input billed per token. The harness is your competitive advantage.

The flow of control

User request
    → Task queue (decompose into subtasks if needed)
    → For each task:
        → Context builder (assemble prompt within token budget)
        → Inner loop (model calls + tool execution until stop)
        → Evaluator (did the output satisfy the task?)
        → If no: recovery policy (retry with feedback, decompose further, escalate)
        → If yes: persist result, advance to next task
    → Final output assembly
    → Observability flush (total cost, latency, turn count, cache hit rate)

Each box in that flow is a function you write. Each arrow is a state transition you control. The model only occupies the "inner loop" box — typically 30-40% of total execution time but 90%+ of cost. Everything else is deterministic code under your authority, executing in microseconds per call.

The harness pattern is not new. Compilers have had optimization passes orchestrated by a pass manager since LLVM (Lattner 2004). Database query planners have had cost-based evaluators selecting execution strategies since System R (Selinger 1979). The agent harness is the same architectural pattern applied to LLM calls: a controller that orchestrates, evaluates, and recovers around an expensive operation that it does not itself perform.

Quantifying the harness advantage

Consider a concrete task: "Fix the failing test in test_auth.py." A zero-shot approach sends the test file and error message to the model and asks for a fix. A harnessed approach:

Reads the test file (tool call 1)
Reads the source file under test (tool call 2)
Reads the error traceback (tool call 3)
Generates a fix
Runs the test suite to verify (tool call 4)
If tests fail, reads the new error and retries (tool calls 5-6)

The zero-shot approach costs ~$0.03 (one call, ~8K input tokens, ~2K output tokens with Claude Sonnet). The harnessed approach costs ~$0.15 (4-6 calls, ~40K total input tokens with caching). But the zero-shot approach succeeds on roughly 35% of non-trivial test failures; the harnessed approach succeeds on 65-70% (measurements from SWE-bench Lite, Jimenez et al. 2024). The cost-per-success is $0.086 for zero-shot vs. $0.214 for harnessed — but the harness produces 2x more successful outcomes. For production systems where a failed fix means human intervention ($50-200 in engineer time), the harnessed approach has dramatically better economics.

The harness doesn't make the model smarter. It gives the model better inputs (targeted context from tool calls), verifies its outputs (test execution), and recovers from failures (retry with error feedback). Intelligence stays constant; information quality and error correction improve.