How AI Coding Agents Build Context (and Why It Breaks)

AI coding agents now write or assist with nearly 100% of code at companies like Coinbase, yet developer productivity gains stall below 15% at most organizations. The gap between those two numbers is the most important story in software engineering right now. It’s not a model capability problem. It’s a context problem. When agents lack durable, repo-embedded understanding of your conventions and architecture, they produce code volume without producing delivery. Understanding how AI coding agents build context — and where that context evaporates — tells you why your tool investment isn’t paying off the way the demos promised.

The Context Engineering Discipline

The bottleneck for coding agents moved from model intelligence to context quality in 2026, and it has a formal name. Anthropic formalized the term context engineering in September 2025, defining it as finding “the smallest possible set of high-signal tokens that maximise the likelihood of some desired outcome.” The distinction from prompt engineering matters more than it sounds. Prompt engineering optimizes a single request’s phrasing. Context engineering optimizes the entire information environment — system instructions, tool definitions, retrieved code, memory, and prior tool outputs — across a multi-step session.

Here’s why that distinction changes how you work. A perfect prompt inside a bloated, irrelevant context still produces mediocre output. A mediocre prompt inside a surgically curated context often produces great output. Once you’ve experienced that a few times, you stop polishing prompts and start engineering context as a systems discipline.

The mental model that makes this click: context is RAM, not storage. The model is the CPU, and its context window is working memory. Just as an operating system carefully curates what fits into limited RAM, context engineering decides what fits into the limited context window on every single turn. Every token competes for the model’s finite attention budget.

The Context Decay Tax

A pattern I’ve observed — what I call the Context Decay Tax — is the fastest-growing bottleneck in AI coding, and it’s invisible on most teams’ dashboards. The tax isn’t model capability or context window size. It’s the evaporation of session understanding at every handoff. Agents reach near-full automation at companies like Coinbase, yet the same industry data shows productivity stalling because convention and context die at session close and must be re-derived from scratch.

One developer watched a 30-file change ship in 8 minutes via an AI agent, then a teammate’s agent spent 25 minutes reverse-engineering what the first agent did to extend it without breaking anything. That 25-minute reverse-engineering time is pure tax. It scales with every handoff: dev-to-dev, agent-to-agent, sandbox-to-CI, sandbox-to-prod. The agent is a consumer of your context, not the source of truth. Anything you want the agent to do reliably across sessions has to live in the repo, not in the conversation.

This tax compounds with agent count. When engineers run 5-10 agents simultaneously, as Coinbase reports, each agent starts cold unless you’ve engineered durable context artifacts. More agents mean more re-derivation, not less. The scale of agent count amplifies the re-derivation tax rather than removing it.

Context Rot: Why Bigger Windows Make Agents Dumber

Larger context windows sound like the obvious fix — just give the agent more room. The data says otherwise. Context rot is the measurable degradation in an LLM’s performance as its input context grows longer. A Chroma research study tested eighteen models including GPT-4.1, Claude 4, and Gemini 2.5, finding that performance degrades as context length increases across every model. A model that aces a needle-in-a-haystack question at 1,000 tokens starts missing the same needle at 100,000, even though the answer is sitting right there.

The mechanism is straightforward once you internalize it. A context window is not memory. It’s attention, and attention is a budget. Every token you add competes with every other token for the model’s finite ability to focus. A 200,000-token window doesn’t mean the model reads 200,000 tokens well. It means it can hold them. Holding is not the same as using.

This hits coding agents hardest because they live in long sessions with big files and windows that fill with the agent’s own generated output. Newer content — retrieved code chunks, tool outputs, fresh conversation turns — displaces older history. Your carefully stated constraint from message three gets buried under forty turns of tool output by message forty. The same model, same prompt quality, worse output. That’s context rot.

The tradeoff is real: larger context windows give you full-repo visibility but cost you attention degradation and higher token bills. There’s no free lunch at scale.

The Adoption vs. Productivity Paradox

The industry data tells two contradictory stories simultaneously, and you need to hold both in your head. Coinbase’s AI-assisted code development reached 95-100% by July 2026, up from roughly 40% in February, with engineers running 5-10 agents simultaneously and cumulative output equal to about 1,200 human employees. That’s massive real-world scaling of agent-written code.

Meanwhile, in a longitudinal study Atlassian ran with DX, the volume of AI-authored code nearly doubled over three months while developer productivity gains stalled at around 15%, below 10% at many organizations. Research from Anthropic’s Societal Impacts team reinforces this paradox: developers use AI in roughly 60% of their work but report being able to “fully delegate” only 0-20% of tasks.

Both are true. Code volume rises. Delivery stays flat. The gap is the Context Decay Tax in action. Agents produce enormous volumes of code, but without durable context infrastructure — convention files, knowledge graphs, governed memory — each session re-derives what the last session already knew. You’re paying token costs for re-discovery instead of paying for forward progress.

How Agents Actually Build Context Today

When an agent opens your codebase, it needs four things in a specific order, and most teams only provide the first one reliably:

Stack and entry points — language, framework, where the app boots, what runs in tests. Most agents get this from package.json and a glance at the entry file.
Convention — how files are named, where routes live, where DB schemas live, import order. Agents infer this from neighboring files, badly.
Component contracts — which UI primitives are blessed, what their props are, what variants exist, what to never use. Agents guess, or they pick whatever library they were trained on most loudly.
Tried-and-tested recipes — for recurring tasks like adding a webhook handler or writing a migration, what does the team-agreed version look like. Agents invent this from scratch every single time.

That last item is where the hours disappear. The agent doesn’t know your team agreed webhook handlers go in a specific directory with signature verification as the first line. So it writes a handler that almost matches the one already in your codebase but doesn’t, and you spend 40 minutes diffing them. Convention files like AGENTS.md — a vendor-neutral Markdown standard that gives agents project-specific operational guidance — can cut this waste significantly, as we’ve covered in our AGENTS.md explainer. Human-curated files deliver a 35-55% reduction in agent-generated bugs, while auto-generated or bloated files add hidden token costs and hurt reliability, as detailed in our real-world examples guide.

The problem is that most of this context lives in developers’ heads, not in the repo. Every session is a fresh boot. The only thing carried across turns is what the agent pulled into its window, which means the only thing carried across sessions is what the model can re-read when it starts.

The Emerging Context Infrastructure Layer

A new tooling layer is forming to address the gap between code volume and delivery, and it’s worth watching closely. Concho AI introduced a platform on July 14, 2026 that turns enterprise codebases into a knowledge layer for AI agents, exposing findings to tools like Claude through the Model Context Protocol. Rather than asking developers to adopt entirely separate tooling, it acts as a middle “fact layer” — a knowledge graph plugin that lets agents explore what an application means, including its architecture and business behavior, rather than inspecting individual files.

This matters because it addresses the core problem: as projects age, reach millions of lines of code across several languages, and pass through multiple generations of developers, agents lose the ability to work efficiently. Older codebases contain “shadow” code that exists for a purpose but whose operation and value was lost when developers left. Generative AI has made producing new code inexpensive. Recovering knowledge from embedded and old systems remains considerably harder.

Open-source approaches are emerging too. Libraries like context-fabrica combine semantic retrieval, graph traversal, and temporal recall so agents can reason about their memory rather than just retrieving vaguely similar text. Every query returns scored results with full breakdowns — semantic match, graph relation, recency, confidence — so agents can justify why a memory was relevant, not just that it was. This is the kind of infrastructure that makes the Context Decay Tax visible and manageable. For a deeper dive into how harness configuration affects agent performance on large codebases, see our configuration guide.

Pricing: What Context Costs at Scale

The June 2026 billing reset eliminated free-tier subsidies for most major AI coding tools, making context engineering a cost-control imperative, not just a quality concern. Here’s what the major tools cost now:

Tool	Pricing	Key Feature	Target Audience
GitHub Copilot Enterprise	$39/user/month + ~$0.04/premium request	Auto-compaction at 80% window	Teams wanting managed automation
Cursor Business	$40/user/month	Repo-scale vector indexing, @-references	Teams wanting retrieval control
Claude Sonnet 5 (API)	$2/M input, $10/M output (intro through Aug 31)	1M-token context window	Teams running custom agent loops
Grok 4.5 (API)	$2/M input, $6/M output	500K-token context window	Cost-conscious agentic workflows

The pricing math for a 50-developer team makes the stakes concrete. A team using GitHub Copilot Enterprise at $39/user/month incurs $1,950/month ($23,400/year) in base subscription alone, excluding metered premium requests at roughly $0.04 each, per Tech Insider’s pricing analysis. A comparable Cursor Business deployment at $40/user/month costs $2,000/month ($24,000/year). Those are floor numbers — the metered layer is where the real damage happens.

Here’s why that matters for context specifically. Every redundant token an agent burns re-deriving context you could have persisted in a convention file is a premium request you didn’t need to make. When agents start cold every session, they explore your repo, pull files into the window, and churn through tokens to reconstruct understanding that a 20-line AGENTS.md could have provided in a fraction of the cost. Our analysis of repository context strategies found that enterprise AI cost is largely input tokens from agents re-exploring repos, and local context layers cut token use.

The Decision Framework

By late 2026, the decisive factor for AI coding ROI is not which model or agent a team buys. It’s whether they engineer repo-persistent context as a first-class artifact. Teams without it will see agent-produced code volume rise while real delivery stays flat — exactly as Atlassian’s data shows. Here’s how to think about the tradeoffs:

If your codebase is small and greenfield, ephemeral sandbox agents work fine. You don’t need heavy context infrastructure yet. Start with a minimal convention file and add structure as the repo grows.
If your codebase is large and mature, you need a knowledge layer. Convention files alone won’t scale to millions of lines across multiple languages. Evaluate tools that expose semantic understanding through standard protocols rather than locking you into a single vendor’s context pipeline.
If your team runs multiple agents simultaneously, governed memory is non-negotiable. Without it, each agent re-derives context independently, and your token bill scales linearly with agent count instead of sublinearly. Open-source memory layers with provenance tracking let agents justify their answers rather than hallucinating from stale context.
If you’re on a flat-rate plan, audit your usage before the metered transition hits. The spread between cheapest and priciest models is 24x on some platforms. Context engineering — giving agents the right tokens in the right order — is your biggest lever for staying inside included allowances.

The teams that win will be the ones who treat context as infrastructure: versioned, tested, governed, and persisted across sessions. Everyone else will keep paying the Context Decay Tax — in tokens, in time, and in code that ships fast but doesn’t deliver. The question isn’t whether you need context engineering. It’s whether you’ll build it before your bill forces you to.