Context Engineering: Why Prompt Engineering Isn't Enough

Telling an AI agent "do this" isn't enough anymore. A single-turn prompt still works for a one-off task; but when you're dealing with a multi-step code change, navigating a file system, or an agent session that runs for hours, the question stops being about word choice and becomes which information is given to the model when, in what form, and at what cost to the context window. This is called context engineering, and Anthropic positions it not as a replacement for prompt engineering, but as its natural continuation.

💡 Pro Tip: Think of context engineering not as "writing better prompts" but as "budget management that decides what to send the model on every turn" — the question isn't "what should I write" but "what do I keep in context and what do I leave out".

Table of Contents

• Why prompt engineering stopped being enough
• Why this is an engineering problem
• The core components of context engineering: instructions, knowledge, tools, memory
• Context budget: token cost and attention dilution
• The persistent layer: rule files, examples, project memory
• Delegating context to sub-agents and keeping verbose work out of the main context
• A working pattern that doesn't break the prompt cache
• An observation from my own project: what happens without context discipline
• Update (September 2026)
• FAQ
• What is context engineering, and how is it different from prompt engineering?
• How do you give the right context to an AI coding agent?
• How should a CLAUDE.md file be written?
• Is prompt engineering dead?
• Does a bigger context window reduce the need for context engineering?
• Conclusion
• Sources

Why prompt engineering stopped being enough

Prompt engineering is about optimizing the phrasing of a single message: the right words, the right examples, the right format. Anthropic's own positioning is clear: "At Anthropic, we view context engineering as the natural progression of prompt engineering" — meaning one doesn't cancel the other out, it builds on top of it.

The problem is that prompt engineering mostly operates at the single-turn level. Anthropic states this limit directly: "as we move towards engineering more capable agents that operate over multiple turns of inference and longer time horizons, we need strategies for managing the entire context state". When an agent reads a file, runs a test, interprets the error, and decides the next step, there's no longer a "best prompt" — there's a _system_ deciding, every turn, which history, tool result, and file content stays in context.

There's a physical fact that complicates this further: context rot. The more tokens you fill the window with, the lower the model's ability to correctly recall that information. This is why "bigger context window = better result" is wrong — as the window grows, every irrelevant token in it reduces the "visibility" of the next genuinely important token.

Why this is an engineering problem

Where prompt engineering is a one-time writing task, context engineering is a recurring maintenance task: building a mechanism that decides, on every agent turn, "should I keep this information, should I summarize it, or should I never send it at all". So you're not designing a single prompt template, you're designing a _harness_ (the skeleton the agent runs in: system prompt + tool definitions + memory strategy + compaction rules).

One way to make this distinction concrete is to put two different questions side by side. Prompt engineering asks "what should I tell the model this turn". Context engineering asks "throughout this agent session, when should which information enter context, and when should it leave context". The first is a writing skill, the second is a systems-design skill — and both are needed at once, because even the best-designed context management can't compensate for a poorly worded instruction.

The core components of context engineering: instructions, knowledge, tools, memory

Anthropic refers to these components in its own terminology as "system prompts, tools, examples, message history"; in the table below I match them to their practical counterparts — the mapping is mine, the descriptions are Anthropic's own sentences.

The instruction (system prompt) must sit "at the right altitude". In Anthropic's words: "The optimal altitude strikes a balance: specific enough to guide behavior... yet flexible enough..." A too-rigid instruction boxes the agent in; a too-vague instruction leads to inconsistent behavior.

Tools are a contract, not a feature list. The most common failure mode is building tool sets that overlap each other or cover too broad a domain. Anthropic says this directly: "One of the most common failure modes we see is bloated tool sets that cover too much functionality..." A small, clearly defined set of tools delivers more reliable results than a large, blurry one.

Examples should be canonical cases, not a rule list. "we recommend working to curate a set of diverse, canonical examples..." — meaning real, diverse example interactions rather than a "don't do this, do that" list.

Memory is the subject of the next section, since it requires a persistent layer that goes beyond message history.

Context budget: token cost and attention dilution

LLMs have a limited "attention budget", just like humans. In Anthropic's words: "LLMs have an 'attention budget' that they draw on when parsing large volumes of context." Every new token draws from this budget — every line you throw into context takes a share of the model's attention.

There's an architectural reason for this: the transformer architecture produces n² pairwise relationships for n tokens. Anthropic summarizes it this way: "This results in n² pairwise relationships for n tokens." And gives the consequence along with it: "As its context length increases, a model's ability to capture these pairwise relationships gets stretched thin, creating a natural tension between context size and attention focus." In other words, as context grows, the model's capacity to relate every token pair to each other stretches thin — this is the architectural root of what we call "context rot".

This isn't an abstract worry. Per MIT NANDA's _The GenAI Divide_ report (Fortune, August 18, 2025), most enterprise generative AI pilots fail to produce measurable impact: "The 95% failure rate for enterprise AI solutions represents the clearest manifestation of the GenAI Divide." The root cause is familiar: "The core issue? Not the quality of the AI models, but the 'learning gap' for both tools and organizations." — the problem isn't the model's "intelligence", it's the quality of the context given to it and the way that context is wired in.

There are three concrete ways to manage the context budget in practice:

• Just-in-time retrieval: Don't load everything up front; let the agent navigate the file system (glob/grep) when it needs to.
• Compaction: When a conversation nears the context limit, discard raw tool outputs, keep architectural decisions and unresolved issues.
• Delegation to sub-agents: Run verbose/exploration-heavy work in a separate, clean context window, send only the summary back to the main context.

1# Bad example: loading all project files into context up front
2find src -name '*.ts' -exec cat {} + > /tmp/full-context.txt   # tens of thousands of tokens, mostly irrelevant
3 
4# Good example: the agent only searches when it needs to
5grep -rn "handleAuthRefresh" src/lib/auth-v2/ | head -20
6# → only a few hundred relevant tokens enter context

Here's a rough guide to which of these three approaches to pick and when:

None of the three are mutually exclusive: in practice, a short rule file is preloaded, source code is searched just-in-time, and a long log analysis is delegated to a separate sub-agent. The selection criterion always comes down to the same question: is this information needed every turn, or only on some turns?

The persistent layer: rule files, examples, project memory

Anthropic describes how Claude Code handles CLAUDE.md files as a "hybrid strategy": "CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment..." In other words, project rules (short, dense, frequently used information) are loaded up front; but the project files themselves are searched by the agent when needed, not all pushed into context in advance.

1# CLAUDE.md — anatomy of a good rule file
2 
3# Project summary
4 
5- Domain, stack, critical paths (2-3 lines, not a long narrative)
6 
7# Working rule
8 
9- Deploy has a single canonical script; no manual command chains
10- The canonical tree is the server, the local repo is stale — the agent shouldn't trust it
11 
12# Absolute Prohibitions
13 
14- Don't build a new module with mock data
15- Don't leave test data in production

Beyond this, structured note-taking (agentic memory) comes into play: recording progress to a persistent file outside the context window. Anthropic describes it this way: "this simple pattern allows the agent to track progress across complex tasks, maintaining critical context and dependencies..." A NOTES.md or progress file lets the agent remember which decisions were made and which bugs remain unresolved even after compaction or a session restart.

This idea also became an official tool with the Sonnet 4.5 launch: Anthropic released a file-based memory tool in beta on the Claude Developer Platform: "As part of our Sonnet 4.5 launch, we released a memory tool in public beta on the Claude Developer Platform..." Persistent memory is no longer just a disciplined practice — it's a first-class capability the platform itself supports.

Delegating context to sub-agents and keeping verbose work out of the main context

A single agent trying to hold the entire state of a large project in its own context window doesn't scale. Anthropic's recommended model is an "orchestrator + specialist sub-agents" split: "Rather than one agent attempting to maintain state across an entire project, specialized sub-agents can handle focused tasks with clean context windows." The main agent holds and coordinates the high-level plan; sub-agents do the deep, verbose work (file exploration, log reading, interpreting long output) in their own clean windows and return only a summary.

1{
2  "subagent_task": "changelog-diff-summary",
3  "input_scope": "CHANGELOG.md last 30 release entries",
4  "context_window": "isolated, independent of the main session",
5  "return_to_parent": "≤1500 characters summary + file path",
6  "raw_output_kept_in": "scratchpad, never enters the main context"
7}

The complement to this is compaction: a session nearing the context limit gets summarized and continues with a fresh window, but not everything is deleted. Anthropic describes its implementation in Claude Code this way: "The model preserves architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs..." In other words, which architectural decision was made and which bug is still open is preserved, but a file's full raw content or a repeated tool output is discarded.

A working pattern that doesn't break the prompt cache

Another dimension of managing the context budget is keeping the same prefix (system prompt, tool definitions, persistent rule file) constant across turns. Prompt caching relies on the model recognizing the same leading token sequence without "re-reading" it every time; so adding a changing timestamp, a random ID, or a differently ordered tool list to the start of the system prompt on every turn breaks the cache and makes every turn more expensive and slower.

The practical rule is simple: put what's fixed at the start, what's variable at the end. The system prompt + tool definitions + persistent rules should stay at the very start and remain byte-for-byte identical across turns; variable content like the current turn's user message or the current turn's tool result should be appended at the end.

1// Bad: a different timestamp prefix every turn breaks the cache
2const systemPromptBad = `Today is ${new Date().toISOString()} — rules: ...`;
3 
4// Good: fixed prefix, variable content in a separate message
5const systemPromptGood = `Rules: ...`; // byte-for-byte identical every turn
6 
7async function runTool(name: string, pattern: string): Promise<string> {
8  return `${name} result: ${pattern}`;
9}
10 
11async function buildTurnContext() {
12  const toolResult = await runTool("grep", "handleAuthRefresh");
13  return { systemPrompt: systemPromptGood, timestamp: Date.now(), toolResult };
14}

This rule is even more critical in a sub-agent architecture: if a sub-agent restarts (fork/resume) without using the same tool list and the same system prompt prefix, that sub-agent's cache starts from zero. This rule is model-independent; I look at today's state of cache-read economics in the Update section.

An observation from my own project: what happens without context discipline

I won't share numbers — I don't have a measured, verifiable benchmark — but I have a recurring observation: give an agent an irrelevant chunk of context (a whole old file, an unrelated log dump) and its next suggestion usually drifts off-topic, influenced by that chunk. Restrict context to just the relevant file or function, and the suggestion is more accurate and arrives faster. This is why I work the way I do: keeping CLAUDE.md short, quoting relevant lines instead of having agents read whole files, keeping sub-agent reports short — different faces of the same discipline.

I generally ask "which file, which rule does this agent actually need" before asking "how much context can I give this agent". The order matters: scope gets narrowed first, then the budget gets calculated.

Update (September 2026)

The body of this article was written based on the tools and versions available on December 9, 2025. Since then, context engineering practice has moved forward on a few concrete points:

Skills became an enterprise standard. Claude Skills, announced October 16, 2025, gained organization-wide management and a skill directory with the December 18, 2025 update. The practical effect: instead of "writing prompts", building "reusable context packages that load only when needed" has become an organizational discipline. (source: claude.com/blog/skills)

"Context anxiety" was documented, then aged out with the model. Anthropic documented that Claude Sonnet 4.5 tends to end a task early when it "senses" it's nearing the context limit (context anxiety), and added a context reset to the harness (March 24, 2026). But testing the same fix on a stronger model (Opus 4.5) found the behavior had already disappeared — the resets became unnecessary "dead weight" (April 8, 2026). Lesson: context engineering isn't a static recipe, it needs re-evaluating with every model upgrade. (sources: anthropic.com/engineering/harness-design-long-running-apps, anthropic.com/engineering/managed-agents)

Session, harness, and sandbox were separated. Managed Agents, announced by Anthropic on April 8, 2026, turned the conversation log (session), the decision loop (harness), and the execution environment (sandbox) into separate, swappable abstractions — the goal being that the context management contract stays stable even if the harness implementation changes.

One setting was added, another lost its effect. Per the Claude Code changelog, v2.1.261 added bashOutputMaxChars and taskOutputMaxChars, raising how much command and background-task output is shown inline before it's written to a file (inline threshold ≤128K characters). In v2.1.277, the TaskOutput tool was removed; per the changelog, taskOutputMaxChars and TASK_MAX_OUTPUT_LENGTH "no longer have any effect" — today only bashOutputMaxChars applies.

1{
2  "bashOutputMaxChars": 4000
3}

Format choice created a measurable cost. An academic study (arXiv 2602.05447, summarized by Simon Willison on February 9, 2026) ran 9,649 experiments across 11 models × 4 formats (YAML, Markdown, JSON, TOON): despite TOON's smaller file size, models' unfamiliarity with the format made token consumption rise noticeably on large schemas (10,000 tables) versus YAML — researchers called this the "grep tax". Lesson: "smaller file" doesn't always mean "lower token cost" — format choice is part of the context budget too.

Cache-read economics got even cheaper on newer models. Per the pricing page, cache-read's standard multiplier is 0.1x of the base input price ("All other models use the standard 0.1x multiplier"); on the new default models it drops lower still: on Claude Opus 5.5 a cache hit is 5% of standard input price ($0.20/MTok), on Claude Fable 5.1 and Claude Mythos 5.1 it's 2.5% ($0.25/MTok). So "keep the prefix fixed" has a measurable effect not just on speed, but directly on cost. (source: platform.claude.com/docs/en/about-claude/pricing)

FAQ

What is context engineering, and how is it different from prompt engineering?

Prompt engineering optimizes the phrasing of a single message and is most effective in single-turn interactions. Context engineering is a broader discipline for multi-step, agent-based tasks: deciding on every turn which information (instructions, tools, examples, history) gets sent to the model, in what form, at what context cost. Anthropic defines it as the "natural progression" of prompt engineering — one doesn't replace the other.

How do you give the right context to an AI coding agent?

Use just-in-time retrieval instead of loading everything up front: let the agent navigate the file system with glob/grep when it needs to. Keep the system prompt "at the right altitude" — neither too rigid nor too vague. Keep the tool set narrow and clearly defined, avoid blurry/overlapping tools. Delegate verbose work (log reading, broad exploration) to sub-agents and let only a summary return to the main context.

How should a CLAUDE.md file be written?

Keep it short and dense: project summary, working rules, critical paths in a few lines. CLAUDE.md files get loaded into context up front, so don't put long narrative or rarely-used detail in it — leave that in separate files the agent finds via glob/grep when needed. Use a separate progress/notes file (structured note-taking) for persistent project decisions and unresolved issues.

Is prompt engineering dead?

No. It's still the most effective approach for simple, single-turn tasks. The skill of writing prompts doesn't die; what changes is that it alone isn't enough for multi-step agent tasks, which need a context management layer (tool design, memory strategy, context budget) on top.

Does a bigger context window reduce the need for context engineering?

No, the opposite has been shown: even in a larger context window, the model's attention capacity keeps getting stretched thin by n² relationship growth (context rot). A bigger window doesn't mean less discipline is needed — you still have to decide what information to keep and what to discard.

Conclusion

Context engineering doesn't invalidate prompt engineering; it complements it with a context-budget discipline: keeping instructions at the right altitude, the tool set narrow, examples canonical, memory persistent, and verbose work delegated to sub-agents. To combine this with the fundamentals on the prompt side, see a 10-year archive of prompt engineering patterns — this article is its natural continuation. For concrete examples of how a large window doesn't eliminate context rot, codebase analysis with Claude's 1M context window is a good companion. Struggling to decide which tool (skill, subagent, hook, MCP) to use and when? Choosing between skill, subagent, hook, and MCP in Claude Code turns this article's sub-agent idea into a practical decision tree. For persistent memory and project context management, see memory and context management in Claude Projects; for the prompt-cache discipline from the cost angle, see cutting cost 10x with prompt caching.

Sources

• Effective context engineering for AI agents — Anthropic's primary engineering article; the source of the instruction/tool/example/memory components, context rot, compaction, and sub-agent architecture discussed here.
• MIT report: 95% of generative AI pilots at companies are failing — the 95% finding from MIT NANDA's _The GenAI Divide_ report and the "learning gap" root cause.
• Claude Developer Platform pricing — current cache-read pricing table (for the Update section).
• Claude Code CHANGELOG — release notes on bashOutputMaxChars/taskOutputMaxChars and prompt-cache stability.
• Harness design for long-running application development — the article documenting the "context anxiety" behavior.
• Managed Agents — the session/harness/sandbox separation and how the context-anxiety fix became unnecessary on Opus 4.5.
• Structured context engineering for file-native agentic systems (summary) — Simon Willison's summary of the TOON/YAML "grep tax" finding (original: arXiv 2602.05447).