Loop Engineering & Agent Harnesses — Core

Why

The agent is the reasoning engine: the model combined with its tool definitions that decides what action to take given context. The harness is everything else: the loop control, context assembly, recovery logic, evaluation, state persistence, and observability that wraps the model call. The harness owns control flow unconditionally. When a model API call returns, the harness resumes control, even if the response is malformed, truncated, or empty. The model is a stateless function called within the harness structure. Research has shown that a well-designed harness around a weaker model consistently outperforms a stronger model without scaffolding. Anthropic demonstrated that Claude Haiku with a production harness scored 38% on their code-repair benchmark while Claude Sonnet without scaffolding scored only 24%. The harness does not handle GPU allocation or model loading, which are infrastructure concerns. It does not split into client and server components. It does not separate planning from execution; both happen within the harness's control. The highest-leverage code in any agent system is the harness, not the model.

Why

A production harness has six components that work together. The task queue holds ordered work items the harness feeds to the inner loop. The context builder assembles prompts from stable prefixes, dynamic middle layers, and the growing conversation tail while managing token budgets. The evaluator determines whether the agent's output satisfies success criteria. State persistence serializes the full agent state for crash recovery and resumability. Recovery policies define deterministic responses to every failure mode including rate limits, tool errors, and context overflow. Observability provides structured logs of every iteration including tokens, latency, cost, and evaluator verdicts. The RAG pipeline components describe a retrieval-augmented generation system, not a harness. The request-response lifecycle components describe an API gateway, not an agent harness. The security pipeline components are important but are a different architectural layer. Each harness component is a function you write, and each transition between them is a state change you control. The model only occupies one component, typically accounting for 30-40% of execution time but over 90% of cost.

Why

Harness parameters, including the system prompt, model identifier, tool definitions, and temperature settings, are set once before the loop starts and must never change during execution. The primary reason for enforcing immutability through dataclass(frozen=True) is to prevent bugs where code accidentally modifies these values mid-loop. A bug that appends a tool result to the system prompt instead of to the message history would cause the model's behavior to shift unpredictably on subsequent turns, and this bug would be invisible in output logs because system prompts are not typically included in response logging. The frozen annotation raises a FrozenInstanceError on any mutation attempt, catching the bug at the point of introduction rather than manifesting as mysterious behavior changes 15 turns later. Memory efficiency and interpreter optimization are not significant factors for a frozen dataclass. Thread safety is a secondary benefit but not the primary motivation for immutability in harness design. The key principle is: immutable parameters define what the agent is, mutable state tracks what the agent has done.

Why

The inner loop checks a priority-ordered list of exit conditions after each iteration. The checks run in order of urgency. First, external abort signal, which fires immediately on timeout, user cancel, or budget exhaustion. Second, turn limit reached, which prevents infinite loops by capping the total number of iterations. Third, token budget exhausted, which prevents runaway cost. Fourth, the model returned end_turn with no tool calls, indicating the model believes it has finished. Fifth, all pending tasks are marked done. If none of these conditions are met, the loop continues. This deterministic priority ordering ensures the most critical checks run first. The model does not output explicit confidence scores that control loop termination. Real-time user feedback applies to turn-based loops but not to the inner loop's iteration logic. The tool-call-to-text ratio does not determine loop termination; a text-only response without tool calls triggers the end_turn exit. The key design principle is that the harness decides when to stop, not the model.

Why

Context arrives in four layers ordered by stability. The system prompt and tool definitions are the most stable, changing only between deployments and forming the cache anchor for prompt caching. The task description is set once per request and sometimes augmented by evaluator feedback. Retrieved documents are dynamic, arriving when the model reads files or queries APIs, and can range from zero to over 100,000 tokens. Conversation history is the most volatile, growing monotonically with every iteration as assistant responses and tool results accumulate. This ordering matters for two reasons. First, prompt caching: the stable prefix generates cache hits on every subsequent call, with Anthropic's caching providing a 90% discount on cached tokens. Second, compaction targets: when context exceeds the budget, conversation history is the primary compaction target since it is the most volatile and often contains redundant information from earlier exploration. The layers involving model weights and attention matrices are internal inference mechanics, not context assembly. API configurations are infrastructure concerns, not prompt context.