Event-Driven Agent Architectures

Webhook endpoints convert external platform events into agent work. A GitHub issues.opened event arrives as a POST request with a JSON payload containing the issue title, body, author, repository, and labels. The agent never polls — the platform pushes.

Trigger taxonomy

Webhooks deliver events in near-real-time (typically < 1 second from platform event to POST delivery). GitHub supports 40+ event types across push, pull_request, issues, deployment, and workflow_run categories. Slack's Events API delivers message, reaction, and channel events with a 3-second challenge-response verification handshake on setup. PagerDuty pushes incident creation, acknowledgment, and resolution events. Each platform uses a different payload schema, authentication mechanism, and retry policy — there is no standard. GitHub retries up to 3 times over 24 hours; Slack retries 3 times within 1 hour; PagerDuty retries with exponential backoff up to 4 hours.

Payload sizes vary dramatically. A GitHub push event with 20 commits and file diffs can exceed 200KB. A Slack message event for a short text message is 2-3KB. A PagerDuty incident event is 5-10KB. Agents must handle payloads at the large end without memory pressure — streaming JSON parsers or early truncation of irrelevant fields (diff content in push events is rarely needed for triage).

Database change streams capture row-level mutations. Postgres LISTEN/NOTIFY fires on channel events triggered by table-level triggers — a new row in support_tickets sends a notification on the new_ticket channel with the row ID as the payload. The subscriber receives events in-process via the database connection. Limitations: LISTEN/NOTIFY payloads are capped at 8000 bytes, notifications are lost if no subscriber is connected (no persistence), and high-throughput tables (>1000 inserts/second) can overwhelm the notification queue.

Change Data Capture (CDC) via Debezium streams the full row contents from the write-ahead log (WAL), capturing every INSERT, UPDATE, and DELETE without polling or triggers. CDC adds 50-200ms of latency versus direct webhooks but captures changes that originate from any source (migrations, manual SQL, other services, background jobs). Debezium publishes change events to Kafka, which provides persistence, ordering, and consumer group semantics — a crashed agent resumes from its last committed offset without missing events.

Cron schedules trigger agents on fixed intervals. A report generator runs every Monday at 09:00 UTC. A stale-issue closer runs every 6 hours, scanning for issues with no activity in the past 30 days. A compliance checker runs nightly, auditing permission sets across all repositories. Cron-driven agents need state tracking — a high-water mark (timestamp or sequence number of the most recent processed item) stored in a database — to avoid reprocessing events from prior runs. Without a high-water mark, a 6-hour scan reprocesses everything from the past 30 days on every invocation.

Queue messages decouple event producers from agent consumers. SQS provides at-least-once delivery with configurable visibility timeouts (default 30 seconds, extendable to 12 hours for long LLM calls — set this higher than your worst-case agent runtime, or the message becomes visible again and a second worker picks it up). Dead-letter queues capture messages that fail after a configurable number of attempts (default 3), preventing poison messages from blocking the queue. Redis pub/sub delivers messages to all connected subscribers with no persistence — a subscriber that disconnects for 10 seconds misses every message published during that window. Kafka provides ordered, persistent, replayable streams with consumer group semantics for parallel processing — the gold standard for high-throughput event pipelines, but operationally heavy (ZooKeeper/KRaft coordination, partition management, retention policies).

The event pipeline

Raw webhook payloads are not agent-ready. A GitHub issues.opened event contains 200+ fields across nested objects — most irrelevant to triage. The pipeline between event receipt and agent dispatch has four stages.

Receive. Accept the HTTP POST, return 200 immediately (within 100ms — GitHub retries after 10 seconds on timeout, so a slow handler triggers duplicate deliveries), and enqueue the payload for async processing. Never run agent logic synchronously inside the webhook handler. The handler should do three things: read the body, enqueue it, and return. Everything else happens asynchronously.

import hashlib
import hmac
import json
from fastapi import FastAPI, Request, HTTPException

app = FastAPI()

WEBHOOK_SECRET = b"whsec_your_secret_here"

async def verify_github_signature(request: Request) -> bytes:
    body = await request.body()
    signature = request.headers.get("X-Hub-Signature-256", "")
    if not signature.startswith("sha256="):
        raise HTTPException(status_code=401, detail="Missing signature")
    expected = hmac.new(WEBHOOK_SECRET, body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(signature[7:], expected):
        raise HTTPException(status_code=401, detail="Invalid signature")
    return body

Validate. Verify the request is authentic. GitHub signs webhook payloads with HMAC-SHA256 using a shared secret configured when the webhook is registered. The signature arrives in the X-Hub-Signature-256 header as sha256=<hex_digest>. Compute HMAC(secret, request_body) and compare using hmac.compare_digest — a constant-time comparison that prevents timing attacks (an attacker who can measure response time differences of 1-10 microseconds can brute-force HMAC signatures byte-by-byte with a standard timing oracle). Reject unsigned or incorrectly signed requests with 401. Slack uses a different scheme: a signing secret combined with a timestamp header (X-Slack-Request-Timestamp) to prevent replay attacks — verify the timestamp is within 5 minutes of current time before checking the signature.

Enrich. The raw event contains references (repository ID, user login, issue number) but not the full context the agent needs. Enrichment fetches supplementary data: the repository's README and contributing guide (for tone and conventions), the author's recent issue history (frequent reporters of false positives get lower initial priority), similar open issues found via embedding search (to detect duplicates before the agent wastes time on a known problem), and the project's labeling taxonomy (so the agent only applies labels that exist). Enrichment adds 200-500ms of latency but dramatically improves agent accuracy — a triage agent with full repo context achieves 89% label accuracy versus 62% with the raw event alone (Yang et al. 2025).

Dispatch. Route the enriched event to the correct agent or workflow. A subscription filter maps event types to handlers:

from typing import Any

DISPATCH_TABLE: dict[str, str] = {
    "issues.opened": "triage_agent",
    "issues.labeled": "assignment_agent",
    "pull_request.opened": "review_agent",
    "pull_request.review_submitted": "merge_agent",
}

async def dispatch(event_type: str, payload: dict[str, Any]) -> None:
    handler = DISPATCH_TABLE.get(event_type)
    if handler is None:
        return
    await enqueue_task(handler, payload)

Subscription filters prevent unnecessary agent invocations. Without filtering, every GitHub event — pushes, comments, label changes, wiki edits — triggers agent processing. A busy repository generates 500-2000 events per day; only 50-100 of those are issues.opened or pull_request.opened events that the triage agent cares about. Filtering at the dispatch level avoids waking the agent for irrelevant events, saving compute and API costs.

Template interpolation

LLMs work with natural language, not raw JSON. Template interpolation converts a structured webhook payload into a task description the agent can reason about:

def build_triage_prompt(payload: dict, enrichment: dict) -> str:
    issue = payload["issue"]
    repo = payload["repository"]
    return (
        f"Triage the following GitHub issue.\n\n"
        f"Repository: {repo['full_name']} ({repo['description'] or 'no description'})\n"
        f"Issue #{issue['number']}: {issue['title']}\n\n"
        f"Body:\n{issue['body'][:3000]}\n\n"
        f"Author: {issue['user']['login']} "
        f"(previously filed {enrichment['author_issue_count']} issues, "
        f"{enrichment['author_false_positive_rate']:.0%} false positive rate)\n\n"
        f"Similar open issues:\n"
        + "\n".join(f"- #{s['number']}: {s['title']} (similarity: {s['score']:.2f})"
                    for s in enrichment["similar_issues"][:5])
        + f"\n\nAvailable labels: {', '.join(enrichment['labels'])}\n"
        f"Team members: {', '.join(enrichment['assignees'])}\n\n"
        f"Respond with JSON: {{\"labels\": [...], \"assignee\": \"...\", "
        f"\"priority\": \"low|medium|high|critical\", \"is_duplicate_of\": null|number, "
        f"\"summary\": \"...\"}}"
    )

Truncate the issue body to a fixed budget. gpt-4o's 128K context window is generous, but enrichment context competes for space, and cost scales linearly with input tokens ($2.50/M input tokens for gpt-4o as of mid-2026). A 3000-character body truncation covers 95%+ of issues — the rare >3000 character issue is typically a data dump or log paste that the agent should not process verbatim anyway.

Idempotency

Webhooks guarantee at-least-once delivery, not exactly-once. GitHub retries failed deliveries up to 3 times over 24 hours. Network intermediaries (load balancers, proxies, CDN edge nodes) can replay requests on timeout. The same event may arrive 2-5 times under normal operation, and during infrastructure incidents, replay bursts of 10+ are possible.

The agent must produce the same outcome regardless of how many times it receives a given event. The standard approach: store a unique event identifier (GitHub provides X-GitHub-Delivery as a UUID per webhook delivery) in a deduplication store before processing, and skip events whose ID already exists.

from redis.asyncio import Redis

redis = Redis(host="localhost", port=6379)

async def is_duplicate(delivery_id: str) -> bool:
    was_set = await redis.set(
        f"webhook:seen:{delivery_id}", "1", nx=True, ex=86400
    )
    return was_set is None

@app.post("/webhooks/github")
async def github_webhook(request: Request):
    body = await verify_github_signature(request)
    delivery_id = request.headers.get("X-GitHub-Delivery", "")
    if await is_duplicate(delivery_id):
        return {"status": "duplicate, skipped"}
    
    payload = json.loads(body)
    event_type = request.headers.get("X-GitHub-Event", "")
    action = payload.get("action", "")
    await dispatch(f"{event_type}.{action}", payload)
    return {"status": "accepted"}

The nx=True flag on Redis SET ensures atomicity — even if two identical webhook deliveries arrive simultaneously on different workers, only one will succeed at setting the key. The ex=86400 TTL (24 hours) bounds memory usage without risking re-delivery within GitHub's retry window (also 24 hours).

For database-backed idempotency, use a unique constraint on the delivery ID column and catch the duplicate key violation. Postgres's INSERT ... ON CONFLICT DO NOTHING is atomic, handles concurrent inserts cleanly, and returns rowcount=0 for duplicates — no exception handling needed, just check the row count.

There is a subtlety: idempotency at the receive layer (deduplication) is necessary but not sufficient. The agent's actions must also be idempotent. Applying a label that already exists is idempotent (GitHub's API silently succeeds). Posting a comment is not — two deliveries of the same event will post two identical comments. The fix: include the delivery ID or event ID in the comment body (hidden in an HTML comment: ), and before posting, check if a comment with that marker already exists.

← Previous