System prompts and instruction design

The system prompt occupies a privileged position in the context window. In the OpenAI and Anthropic APIs, the system message is processed before the user message and receives elevated attention weight — empirically, instructions in the system prompt are followed more reliably than identical instructions placed in the user message (OpenAI 2023 best practices). The system prompt is the constitution of your application: it defines the model's role, behavioral boundaries, output format, and decision-making rules for every subsequent interaction.

Anatomy of a system prompt

A production system prompt has distinct sections, each serving a specific function:

Role definition — who the model is and what domain it operates in
Behavioral constraints — what the model must and must not do
Output format specification — the exact structure of responses
Edge case handling — what to do when input is ambiguous, malformed, or out of scope
Examples (optional) — concrete input/output pairs that anchor behavior

The order matters. Role and constraints go first (highest attention), format specification in the middle, examples at the end. This aligns with the attention patterns described in the "lost in the middle" findings — beginning-position content is most reliably followed.

from openai import OpenAI

client = OpenAI()

# Classification system prompt — production pattern
CLASSIFICATION_SYSTEM_PROMPT = """You are a customer support ticket classifier for a SaaS product.

TASK: Classify each ticket into exactly one category and one priority level.

CATEGORIES (mutually exclusive):
- billing: payment failures, subscription changes, refund requests, invoice questions
- technical: bugs, errors, integration issues, API failures, performance problems
- account: login issues, password resets, permission changes, account deletion
- feature_request: new feature suggestions, enhancement requests, workflow improvements
- general: questions about documentation, onboarding help, how-to queries

PRIORITY LEVELS:
- critical: service is down, data loss, security breach, payment processing broken
- high: major feature broken, blocking user workflow, affecting multiple users
- medium: minor bug, workaround available, single user affected
- low: cosmetic issue, feature request, general question

RULES:
- If a ticket mentions multiple categories, classify by the PRIMARY action needed.
- If priority is ambiguous, default to medium.
- Never classify a billing issue as low priority.
- If the ticket is in a language other than English, still classify it (do not translate).

OUTPUT FORMAT (strict JSON, no markdown):
{"category": "<category>", "priority": "<priority>", "confidence": <0.0-1.0>, "reasoning": "<one sentence>"}"""

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": CLASSIFICATION_SYSTEM_PROMPT},
        {"role": "user", "content": "I've been charged twice for my Pro plan this month. My card shows two $49 charges on the same day. Please refund the duplicate."}
    ],
    temperature=0
)
print(response.choices[0].message.content)

Specificity beats length

A common failure mode is writing long, vague system prompts. A 3000-token system prompt full of general guidance ("be helpful, be accurate, think step by step") performs worse than a 500-token prompt with precise, actionable rules. The model does not need motivation — it needs constraints.

Compare these two instructions for the same task:

Vague (ineffective):

Try to extract entities from the text. Be thorough and make sure you get all the important ones. Format your output nicely.

Specific (effective):

Extract all named entities from the input text. Return a JSON array where each element has keys "text" (exact span from input), "type" (one of: PERSON, ORG, LOCATION, DATE, MONEY), and "start_char" (0-indexed character offset). If no entities are found, return an empty array [].

The specific version constrains the output space. The model knows exactly what types are valid, what the output structure is, and what to do in the empty case. Ambiguity in the system prompt produces ambiguity in the output.

Measured effect: Wei et al. (2023) found that adding explicit output format constraints to prompts increased format compliance from 67% to 94% on structured extraction tasks, with no loss in extraction accuracy.

Negative vs positive framing

Negative instructions ("do not", "never", "avoid") are less reliably followed than positive instructions ("always", "only", "must"). This is not a quirk — it reflects how language models process instructions. A negative instruction requires the model to first activate the concept being negated and then suppress it. The concept activation can bleed into generation.

import anthropic

client = anthropic.Anthropic()

# Negative framing — less reliable
negative_prompt = """You are a medical information assistant.
Do not provide diagnoses.
Do not recommend specific medications.
Do not give treatment plans.
Never say "I'm not a doctor" or similar disclaimers."""

# Positive framing — more reliable
positive_prompt = """You are a medical information assistant.
Provide only general health education information from peer-reviewed sources.
When asked about symptoms, describe what conditions are commonly associated with them and state that a healthcare provider can evaluate further.
When asked about treatments, describe categories of treatments that exist for a condition (e.g., "lifestyle changes, medications, and surgical options") without naming specific drugs or dosages.
Cite sources by name and year when available."""

message = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    system=positive_prompt,
    messages=[
        {"role": "user", "content": "I have a persistent headache on one side of my head that throbs. What should I take?"}
    ]
)
print(message.content[0].text)

The positive framing tells the model what to do at each decision point. The negative framing only tells it what to avoid, leaving the model to infer what it should do instead — and that inference is unreliable.

Persona vs task framing

Two primary approaches to system prompt architecture:

Persona framing defines who the model is: "You are a senior tax accountant with 20 years of experience specializing in small business taxation." The persona implicitly carries domain knowledge, communication style, and behavioral norms. It works well for open-ended conversational applications where the interaction style matters as much as the content.

Task framing defines what the model does: "Classify the input text into one of the following categories." The task framing works better for structured, repeatable operations where consistency matters more than personality. Classification, extraction, transformation, and validation tasks benefit from task framing because the output is constrained and measurable.

In practice, production systems combine both: a brief persona clause followed by detailed task specifications.

from openai import OpenAI

client = OpenAI()

# Extraction system prompt — task framing with minimal persona
EXTRACTION_PROMPT = """You extract structured event data from natural language descriptions.

INPUT: A natural language description of an event or meeting.
OUTPUT: A JSON object with the following fields (all required):

{
  "title": "string — event name, max 80 chars",
  "date": "string — ISO 8601 date (YYYY-MM-DD), or null if not specified",
  "time": "string — 24h format (HH:MM), or null if not specified",
  "duration_minutes": "integer — estimated duration, default 60 if not stated",
  "location": "string — physical or virtual location, or null",
  "attendees": ["array of strings — names mentioned"],
  "action_items": ["array of strings — any follow-ups or tasks mentioned"]
}

RULES:
- Parse relative dates against today's date (provided in user message).
- If a time range is given (e.g., "2-3pm"), compute duration_minutes from the range.
- Extract attendee names exactly as written (do not infer full names from first names).
- If no action items are explicitly stated, return an empty array."""

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": EXTRACTION_PROMPT},
        {"role": "user", "content": "Today is 2025-03-15. Message: Let's sync with Sarah and Mike next Tuesday at 2pm in the downtown office for about 90 minutes to finalize the Q2 budget. Mike should bring the revised spreadsheet."}
    ],
    temperature=0
)
print(response.choices[0].message.content)

Chat system prompts — managing conversation dynamics

Chat applications require system prompts that handle multi-turn dynamics: topic changes, user corrections, context accumulation, and conversation boundaries.

CHAT_SYSTEM_PROMPT = """You are a technical support agent for CloudDB, a managed database service.

IDENTITY:
- Name: CloudDB Support
- Scope: CloudDB product only (PostgreSQL, MySQL, Redis offerings)
- Escalation: If the user's issue requires account-level changes (billing, plan upgrades, data deletion), provide the escalation form link: https://clouddb.example/support/escalate

BEHAVIOR:
- Ask clarifying questions before troubleshooting. Minimum information needed: database engine, plan tier, error message or symptoms.
- Provide step-by-step instructions, one step at a time. After each step, ask if it resolved the issue before proceeding.
- When providing CLI commands, specify the exact clouddb-cli version required.
- If the user asks about a competitor product or a service you do not support, state clearly: "That's outside my scope. I can help with CloudDB's [closest equivalent feature] if that's useful."

CONVERSATION RULES:
- If the user changes topic, acknowledge the switch: "Switching to [new topic] — let me help with that."
- If you previously gave incorrect information in this conversation and the user corrects you, acknowledge the error explicitly and provide the corrected answer.
- Do not repeat information already provided in this conversation unless the user asks for a summary.
- Maximum response length: 200 words unless the user asks for detailed explanation.

OUTPUT STYLE:
- Use code blocks for any CLI commands, SQL queries, or configuration snippets.
- Use bullet points for lists of options or steps.
- No greeting or sign-off after the first message in a conversation."""

Version-controlling system prompts

System prompts are code. They determine application behavior as directly as any function. They should be version-controlled, reviewed, tested, and deployed through the same pipeline as application code.

import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path

class SystemPromptRegistry:
    """Version-controlled system prompt management."""

    def __init__(self, prompts_dir: str = "prompts/"):
        self.prompts_dir = Path(prompts_dir)
        self.prompts_dir.mkdir(parents=True, exist_ok=True)

    def register(self, name: str, content: str, metadata: dict = None) -> str:
        """Register a system prompt version. Returns version hash."""
        version_hash = hashlib.sha256(content.encode()).hexdigest()[:12]
        record = {
            "name": name,
            "version": version_hash,
            "content": content,
            "metadata": metadata or {},
            "registered_at": datetime.now(timezone.utc).isoformat(),
            "token_count": len(content.split()) * 1.3  # rough estimate
        }

        filepath = self.prompts_dir / f"{name}_v{version_hash}.json"
        filepath.write_text(json.dumps(record, indent=2))
        return version_hash

    def load(self, name: str, version: str = None) -> str:
        """Load a prompt by name (latest) or name+version (specific)."""
        if version:
            filepath = self.prompts_dir / f"{name}_v{version}.json"
            record = json.loads(filepath.read_text())
            return record["content"]

        # Load latest version by timestamp
        candidates = sorted(
            self.prompts_dir.glob(f"{name}_v*.json"),
            key=lambda p: json.loads(p.read_text())["registered_at"],
            reverse=True
        )
        if not candidates:
            raise FileNotFoundError(f"No prompt registered with name: {name}")
        return json.loads(candidates[0].read_text())["content"]

registry = SystemPromptRegistry("prompts/")
v1 = registry.register(
    "classifier",
    CLASSIFICATION_SYSTEM_PROMPT,
    metadata={"model": "gpt-4o", "task": "ticket_classification", "author": "eng-team"}
)
print(f"Registered classifier prompt, version: {v1}")

Production practices: store prompts in a dedicated directory with semantic file names. Include a content hash in the version identifier. Log the prompt version hash alongside every API call for debugging and A/B testing. Run regression tests against a held-out evaluation set before deploying prompt changes. Track token count per version — a prompt that grows from 500 to 2000 tokens over iterations is accumulating cost at every call.

The system prompt is the highest-leverage artifact in an LLM application. A single-word change can shift classification accuracy by 5-10 points. Treat it with the same rigor as a database schema migration: test it, review it, deploy it deliberately.

← Previous