Patterns
Anthropic
Context engineering
Problem
The window is finite and ordered. Unstructured stuffing of a repository into the prompt displaces the contract that was sitting in the system message.
Mechanism
Layer the context: invariants and output schema first, then tool definitions, then retrieved evidence, then dialogue. Compact completed tool traces. Cache the stable prefix. Isolate untrusted content (Anthropic’s “context isolation”) so retrieved HTML cannot override policy.
Failure mode
Overflow silently drops the developer message. Duplicate chunks crowd out the schema. A 200K window used as an unlayered dump still fails, just later and more expensively.
Anthropic
Model Context Protocol
Problem
Every host reinvented tool glue: Claude Desktop, IDEs, internal agents. The model cannot see your filesystem, DB, or browser unless you expose a stable surface.
Mechanism
MCP is a client-host-server protocol for tools, resources, and prompts. The host owns permissions and sampling. Servers are reusable. Prefer MCP when the tool surface must outlive a single prompt and run across runtimes.
Failure mode
Wrapping application workflows as many MCP tools when one procedure would suffice. Untrusted servers and confused-deputy failures.
OpenAI · Anthropic · Google
Tool-calling agent loop
Problem
A completion is a token stream. Side effects live in the host runtime. A model report of a search is not itself a search.
Mechanism
Emit a structured tool call (name + args). Execute in the host. Append the observation. Repeat until a final message. This is ReAct with a schema, Responses API function calling, Claude tool use, and Gemini function calling.
Failure mode
Unbounded loops, missing idempotency, arguments that do not match the schema, and hiding errors from the model so it hallucinates success.
OpenAI · Google
Constrained decoding
Problem
A request in prose to return JSON does not constrain the token lattice. Downstream code needs a type.
Mechanism
Structured outputs / JSON schema / responseSchema constrain the token lattice so invalid JSON cannot be sampled. Pair with a scorer that rejects extra keys and wrong types. The schema is part of the product.
Failure mode
Schemas so loose they accept anything; schemas so tight the model cannot represent a legitimate refusal. Mixing prose and JSON in one field.
OpenAI · Anthropic · Google
Evals as regression
Problem
Non-deterministic code has no unit test if inspection is only informal review of chat transcripts.
Mechanism
Freeze (input, properties, scorer). Properties: schema validity, citation presence, tool-shape, refusal, “did not invent /v4/dream”. Run on every prompt or model bump. Pass rate is the release gate.
Failure mode
Evals that only check BLEU against a golden essay. Evals the team is allowed to ignore. A missing eval for the last production incident.
Anthropic · OpenAI
Prompt caching
Problem
Agents resend 20K tokens of tools + policy on every hop. You pay and you wait.
Mechanism
Cache the stable prefix (system + tools). Anthropic cache_control and OpenAI cached input tokens make the hot path the delta: new user turn, new observation. Design context so the prefix actually is stable.
Failure mode
A prefix that mutates every request (timestamps, shuffled tools) busts the cache. Caching untrusted retrieved docs as if they were policy.
Google · Anthropic · OpenAI
RAG vs long context vs agentic retrieval
Problem
Long context makes it possible to concatenate large documents. Retrieval remains a system: search, rank, cite.
Mechanism
Long context for a single working set you already trust. RAG for a corpus you cannot fit or must permission. Agentic retrieval (search → read → refine) when the query is underspecified. Grounding APIs (Google) are a retrieval tool.
Failure mode
Top-k cosine on the wrong embedding of the wrong chunk, then instructing the model to use only the context while the context is uninformative.
Agent-to-agent
Problem
A single agent with a large undifferentiated tool surface is hard to test. Multiple agents without a protocol lack a typed interface.
Mechanism
A2A (and similar): typed tasks, artifacts, and capability cards between agents. Each agent has a narrow tool surface and its own evals. The orchestrator is a runtime.
Failure mode
Agents debating in prose with no schema. Recursive delegation with no budget. Sharing a single undifferentiated context blob.