AI Agents in Production — Core

Why

The core distinction is who decides what happens next. A chain executes a predetermined sequence of steps defined by the developer: retrieve documents, stuff them into a prompt, call the LLM, parse the output. The execution path is fixed at write time. An agent executes a variable sequence of steps where the LLM receives an observation, decides what action to take, executes it, observes the result, and decides again. The execution path emerges at runtime based on the model's reasoning. A chain can make multiple LLM calls in sequence and still be a chain as long as the sequence is predetermined. Both chains and agents can process structured data and various input types. Both can be synchronous or asynchronous. The key boundary is control flow ownership: in a chain, the developer controls the path; in an agent, the model controls the path within constraints set by the developer. Most production systems sit between these extremes, with the model deciding within guardrails.

Why

ReAct interleaves three components at each step. The Thought is free-text reasoning where the model articulates what it knows and what it needs. The Action is a structured tool call selected based on that reasoning. The Observation is the tool's output, which becomes input for the next iteration. The key insight from the original research is that models that reason before acting outperform both chain-of-action models that only act without explicit reasoning and chain-of-thought models that only reason without grounding. On HotpotQA, ReAct achieved 34.3% exact match versus 29.4% for act-only and 28.7% for CoT-only. Reasoning before acting helps the model select the right tool and formulate better arguments. Observing results after acting prevents hallucination by grounding the next reasoning step in actual data. Plan-Execute-Validate is a different pattern that does not interleave reasoning with action at each step. Query-Retrieve-Generate describes RAG, not an agent loop. Encode-Transform-Decode describes a neural network architecture, not an agent pattern.

Why

When each step in an agent plan succeeds independently with probability 0.95, the probability that all steps succeed is 0.95 multiplied by itself 10 times, which equals approximately 0.60 or 60%. This compound error rate is the fundamental reliability constraint on agents. A 5-step plan succeeds with 0.95 to the fifth power equals 0.77, while a 20-step plan succeeds with only 0.95 to the twentieth power equals 0.36. The first option incorrectly assumes per-step accuracy equals overall accuracy, ignoring the multiplicative nature of independent probabilities. The 50% answer incorrectly applies a binomial distribution centered at 0.5, which would only apply if each step had a 50% success rate. The 85% answer uses an invalid formula that does not correspond to any standard probability calculation. Strategies to improve overall success include raising per-step accuracy through better tool descriptions, reducing the number of steps through more capable tools, and using structured outputs to eliminate parsing errors. Improving tool descriptions from vague to precise can increase correct tool selection from approximately 80% to 97%.

Why

When a model decides to call a tool, it produces a structured JSON object containing the function name and arguments, but it never executes the function itself. The model's output is a declaration of intent, not an execution. The developer's application code receives this JSON, executes the corresponding function locally, and sends the result back to the model as a new message. This separation is a critical security property: the model cannot execute arbitrary code because it only produces data describing what it wants to do. The runtime controls what actually happens. The model does not execute functions in a sandboxed environment; it has no execution capability. Modern tool calling APIs use structured JSON output, not free-text parsing, which eliminates the parsing errors common in earlier ReAct implementations. The model does not generate executable Python code; it produces a schema-conformant JSON tool call. Both OpenAI and Anthropic implement this same pattern where the model outputs structured tool calls and the application handles execution, though their specific message formats differ.

Why

The model has no access to the function's implementation. It sees only the tool name, description, and parameter schema. The description is therefore the primary signal for two critical decisions: which tool to call for a given query, and what arguments to pass. Vague descriptions like 'search stuff' cause the model to confuse similar tools and guess at argument formats. Precise descriptions that state what the tool does, what it returns, and what each parameter expects with examples allow the model to match queries to tools reliably. Improving descriptions from vague to precise has been shown to increase correct tool selection from approximately 80% to 97% on internal benchmarks. Tool descriptions are not cached across sessions; they are sent as part of each API call. They do not determine the maximum number of concurrent tool calls. They are not compiled into model weights at fine-tuning time; they are prompt-level instructions processed at inference time. An effective description follows a pattern: first sentence says what the tool does, return value describes what the caller gets back, parameter descriptions include constraints and examples, and edge cases describe what happens on failure.