On this page
Stop Agentic Drift: Spec-Driven Development with AGENTS.md
tl;dr
Spec-driven development prevents agentic drift better than AGENTS.md alone. ETH Zurich found AGENTS.md context files increase inference costs by more than 20 percent without improving task success rates, while spec-first workflows cut runtime by 28.64 percent.
The industry converged on AGENTS.md as a universal instruction standard for AI coding agents, but the data tells a messier story: the only large academic study found these files increase inference costs by over 20 percent without improving task success rates, while vendors interpret the same file in conflicting ways — Codex silently truncates at 32 KiB, Claude Code gives CLAUDE.md precedence, and Cursor still reads .cursorrules. The standard created to eliminate per-vendor fragmentation has fragmented at the interpretation layer instead.
What Is Agentic Drift and Why Does It Matter?
Agentic drift happens when an AI coding agent gradually diverges from your intended architecture, conventions, and security boundaries across a long session. Without a structured specification, the agent fills gaps with whatever patterns it infers from local context — duplicating logic, introducing tight coupling, and bypassing existing utility libraries. The result is code that passes tests but creates maintenance debt, security vulnerabilities, and regression cycles that human reviewers catch too late.
Spec-driven development (SDD) addresses this by making the specification the source of truth. You define what the system must do, what it must not change, and how to verify it works — before any code gets written. The spec then guides the agent through planning, implementation, and validation. GitHub’s Spec Kit passed 130,000 stars in its first year, and AWS launched Kiro, an IDE organized entirely around this workflow, signaling that major platforms are betting on specs as the primary artifact.
AGENTS.md sits at the intersection of these two trends. It’s an open, plain-Markdown file placed at the repository root that instructs AI coding agents on build, test, and change procedures. The format was formalized as an open specification in August 2025 and donated to the Linux Foundation’s Agentic AI Foundation in December 2025. Over 60,000 repositories now use it, and 20+ tools read it natively, including OpenAI Codex, Cursor, GitHub Copilot, and others. But adoption doesn’t equal effectiveness — and the evidence is contradictory.
How Does AGENTS.md Fit Into Spec-Driven Development?
Spec-driven development requires writing a specification before code, with the spec serving as the source of truth for both humans and AI coding agents. In practice, you define requirements and acceptance criteria first, derive a technical plan and task list, then hand those to a coding agent. The agent implements against the tasks, and the spec remains the reference for verification.
AGENTS.md plays a specific role in this pipeline: it provides the standing repository context that every agent session needs — build commands, test commands, code-style conventions, and architectural boundaries. The spec file (often in docs/specs/) covers the task-level requirements for a specific feature. Neither substitutes for the other. As one practitioner put it, AGENTS.md covers the repo layer while the spec covers the task layer.
Tools like GitHub Spec Kit automate the transition from specification to execution across 30+ coding agents using structured slash commands. Kiro uses a similar shape: requirements.md holds user stories and acceptance criteria, design.md holds technical architecture, and tasks.md holds discrete, trackable tasks. The workflow is tool-agnostic by design — if your spec is complete enough to fully describe expected behavior, any capable AI coding tool should be able to implement it.
For teams running multiple agents across a shared codebase, AGENTS.md becomes the portable convention layer. Cursor and Copilot ingest constraints natively while Claude Code links directly via a single import directive. This is where the theory meets reality: the same file gets interpreted differently by each tool, and those differences create the drift the standard was meant to prevent.
What Does the Evidence Say About AGENTS.md Effectiveness?
Here’s where it gets uncomfortable. An ETH Zurich study evaluated four coding agents (Claude Code with Sonnet-4.5, Codex with GPT-5.2 and GPT-5.1 Mini, Qwen Code with Qwen3-30B-Coder) on SWE-Bench and a new CTXbench with 138 real GitHub issues from 12 niche projects. The researchers tested three variants per task: no context file, agent-generated context file, and the original developer-written file. The result: providing context files did not improve task success rates on either benchmark, while increasing inference costs by more than 20 percent on average.
That finding contradicts a 124-pull-request study that found 28.64 percent lower median runtime with spec-driven approaches. The discrepancy likely comes down to file quality and task type. The ETH Zurich study tested uncurated, real-world files — many of which suffer from what researchers call “configuration smells.” In a review of 100 popular open-source repositories containing AGENTS.md or CLAUDE.md, 91 had at least one configuration smell: lint leakage in 62 percent, context bloat in 42 percent, skill leakage in 35 percent, conflicting instructions in 28 percent, init fossilization in 24 percent, and blind references in 16 percent.
Lint leakage occurs when instructions restate rules already enforced by formatters or linters. Context bloat adds codebase overviews the agent could discover itself. Skill leakage includes tool-specific guidance that doesn’t transfer. These smells waste tokens, confuse skill selection, and degrade reliability. The studies agree on one thing: specific commands and non-obvious rules help, while generic overviews only add cost.
Which Tools Actually Read AGENTS.md — and How?
The vendor landscape is fragmented despite the standard. OpenAI Codex reads AGENTS.md before work but silently truncates it at 32 KiB by default without warning. Two of 27 real files measured in one analysis exceeded that limit, meaning critical instructions get dropped silently. Codex concatenates project instructions from the repo root down to the current directory and stops adding files once the combined size reaches the project_doc_max_bytes threshold.
Claude Code reads AGENTS.md natively since version 2.1.277 (September 18, 2026), but only when no CLAUDE.md exists in the working directory or above it; CLAUDE.md takes precedence. This creates a precedence trap: if a repository has both files, AGENTS.md is ignored entirely. The workaround — symlinking AGENTS.md to CLAUDE.md or adding a one-line @AGENTS.md import — was standard advice until the fallback shipped, and it’s still necessary for teams that maintain CLAUDE.md for Claude-specific hooks or skills.
Cursor reads AGENTS.md but also maintains .cursorrules, which it prioritizes for editor-specific behavior like file globs and mode switches. Aider layers .aider.conf.yml on top. Enterprise tools quietly append hidden policies to whatever they find. The result: a file meant to unify instruction management has become another layer of configuration drift. Teams that treat AGENTS.md as a set-and-forget artifact will see diminishing returns as conventions evolve and vendor interpretations diverge.
| Tool | AGENTS.md Support | Precedence / Quirks | Team Pricing |
|---|---|---|---|
| OpenAI Codex | Native, reads first | Silent 32 KiB truncation | $25/user/month |
| Claude Code | Since v2.1.277, fallback only | CLAUDE.md takes precedence | $25/user/month |
| Cursor | Native | Also reads .cursorrules | $30/user/month (whichagent.dev) |
| GitHub Copilot | Native | Also reads .github/copilot-instructions.md | $19/user/month |
What Are the Security Risks of Auto-Loading Instruction Files?
The auto-loading behavior that makes AGENTS.md convenient is also a live attack vector. Researchers at Forever Security discovered a malicious AGENTS.md file in a customer’s Downloads folder that tricked an AI coding agent into installing a rogue npm package. They reproduced the attack against OpenAI Codex (succeeding more than 90 percent of attempts) and Claude Code (roughly 50 percent). The mechanism is simple: a web page can silently drop an AGENTS.md file into ~/Downloads without user interaction, and when the agent launches in that directory, it treats the file as legitimate instructions.
“There is nothing here for OpenAI or Anthropic to patch. An agent that follows the AGENTS.md in its folder is doing exactly what it was built to do,” the researchers noted. The Codex variant modified ~/.codex/config.toml to register an attacker-controlled MCP server, persisting the compromise across all future sessions. The Claude Code variant attempted to leak OAuth tokens by pointing the agent at an attacker-controlled API endpoint. About half of Forever Security’s customers run coding agents inside their Downloads folder, and a quarter of agent sessions open a real browser — making this a practical threat, not a theoretical one.
The fix isn’t a vendor patch; it’s operational hygiene. Treat ~/Downloads as hostile ground for agents. Check for instruction files you didn’t write. Watch agent configurations for unexpected changes. Keep approval gates and sandboxes on. Rotate credentials on the assumption that login tokens are compromised if something looks wrong. This shifts AGENTS.md from a convenience feature to a governed configuration artifact that requires review gates — exactly the opposite of the zero-config onboarding the standard promises.
How Should Teams Structure Their Spec-Driven Workflow?
The pattern that works: one source of truth for specifications, thin adapters per tool. Put the guidance in files no vendor owns (AGENTS.md for repo conventions, specs/ for feature requirements), and give each tool a one-line pointer. Swapping a model then costs one new pointer, not a rewrite.
For AGENTS.md specifically, keep it brief. Every line competes for the model’s attention with the task in front of it. A minimal file covers: what the project is, how to run it (exact commands), how the team works (spec-first, test-first, one task per commit), and the handful of rules that can’t bend (protected paths, secret handling, ADR process). Skip codebase overviews, generic style guides, and “don’t” lists — they add tokens without improving outcomes. In Augment Code’s evals, a numbered workflow cut pull requests with missing files from 40 percent to 10 percent, while a file of 30 bare “don’t” rules doubled the time of a simple task.
For spec files, use a structured format: requirements as user stories with GIVEN/WHEN/THEN acceptance criteria, design documents for interfaces and constraints, and task lists small enough to implement and verify in one session. Tools like Spec Kit and Kiro enforce this structure, but you can implement the same discipline with plain Markdown and plan mode in Claude Code, Codex, or Cursor. The key is making the spec executable — acceptance criteria specific enough to become tests, interfaces specific enough to generate stubs, non-goals explicit enough to bound scope creep.
In monorepos, use hierarchical AGENTS.md files: a root file for shared conventions, per-package files for build commands and package-specific rules. Agents that support nested reading (Codex, Cursor, Copilot) will concatenate context from root to working directory. This avoids the single-file bloat that triggers Codex’s 32 KiB truncation while keeping conventions consistent across packages.
What’s the Cost Tradeoff for Teams Adopting This Approach?
Cursor Teams costs from $30/user/month (whichagent.dev). A 50-developer team using Cursor Teams at $30/user/month would spend $18,000/year on subscriptions alone (50 × $30 × 12) (whichagent.dev). That’s before token overages, which are metered separately and can spike when context files bloat.
The tradeoff isn’t just subscription cost. It’s the maintenance tax of keeping instruction files current across vendor interpretations. For large engineering teams, the open standard is not a set-and-forget convenience but a runtime operational contract that requires active maintenance to avoid performance decay.
If you’re a small team (under 10 developers) with a single primary agent, the overhead may not pay off. If you’re running multiple agents across a shared codebase and PR reviews keep catching the same convention violations regardless of which agent wrote the code, AGENTS.md becomes a leverage point — but only if you treat it as governed configuration, not documentation. The spec-driven layer (requirements, design, tasks) delivers the real drift prevention; AGENTS.md just ensures the agent knows how to run your tests and where to put the code.
Recommendation: Start with a minimal AGENTS.md (under 2 KB) containing only non-inferable commands and hard boundaries. Add a spec-driven workflow for features larger than a bug fix — write the spec first, derive tasks, implement in fresh sessions with tests written from acceptance criteria. Review AGENTS.md in every sprint retro; if it hasn’t changed in three sprints, it’s probably stale. And never launch an agent from ~/Downloads.
What’s your team’s experience with instruction file drift — have you seen the truncation or precedence issues in practice?
Recommended Reading
-
Can AI Coding Agents Replace Junior Devs? What the Data Says
2026 data shows AI coding agents absorb routine junior dev tasks like boilerplate and scaffolding, but do not replace junior engineers one-for-one. Instead, they raise the skill floor for entry-level roles and shift review burden to senior staff, creating hidden costs and pipeline risks for engineering teams.
-
AI Coding Interview Prep: Costs, Risks, and What Works
Free AI interview prep tools beat paid copilots for hiring success. Fabric found 38.5% of interviews crossed its cheating threshold, with 48% in technical roles.
-
Google Shuts Down Jules: Agentic Coding Consolidation Impact
Google is shutting down Jules, and the real switching cost is lost agent memory rather than code migration. Teams should audit scheduled tasks, environment variables, and setup scripts before the platform goes read-only.