Do AGENTS.md Files Improve AI Coding Performance? Benchmarks

The Linux Foundation’s Agentic AI Foundation hosts a standard that 60,000+ repositories and 30+ coding agents have adopted — and the best available research suggests you should use it as sparingly as possible. That tension between standardization momentum and empirical performance is the defining contradiction of AI agent configuration in mid-2026.

AGENTS.md was designed to solve a real problem: developers maintaining duplicate instruction files for every tool in their stack. The format is plain Markdown, vendor-neutral, and read natively by Codex, Cursor, Copilot, Gemini CLI, and dozens of others. On paper, it’s the obvious winner. But the data tells a more complicated story — one where the instruction budget is a hard constraint, and every unnecessary line you add to your config file is a performance penalty you pay on every single turn.

The Instruction Budget Is Smaller Than You Think

Frontier LLMs reliably follow roughly 150 to 200 instructions per turn. Claude Code’s system prompt alone consumes approximately 50 of those, leaving a budget of 100–150 for your project-specific rules. That’s not a soft guideline — it’s a ceiling that, once breached, triggers what researchers call context rot: uniform degradation in instruction adherence as input context grows.

HumanLayer documented this firsthand. When they ran Claude Code on a 1M-token context window, they observed “dramatically degraded” instruction adherence and were forced to return to smaller windows. Chroma’s context-rot study across 18 frontier models and 194,480 API calls confirmed the pattern: degradation is uniform and predictable. More context doesn’t mean better performance. It means worse performance, reliably.

This is why the structure of your AGENTS.md file matters more than its contents. Best practice recommends keeping it under approximately 800 words to preserve context budget for actual work. But even that threshold may be generous, depending on what’s inside.

What the ETH Zurich Study Actually Found

The most rigorous empirical investigation to date comes from ETH Zurich, published on arXiv in early 2026. The researchers built AGENTbench, a dataset of 138 real-world Python tasks from niche repositories — deliberately chosen to avoid the memorization bias that plagues popular benchmarks like SWE-bench.

They tested four agents across three scenarios: no context file, an LLM-generated context file, and a human-written file. The results were striking.

LLM-generated context files reduced task success rates by an average of 3% compared to providing no context file at all. They also increased inference costs by over 20%, because agents took more steps and produced longer reasoning traces. The auto-generated files weren’t just unhelpful — they were actively harmful.

Human-written files performed better, but not by much. They delivered a 4% average increase in task success rate on AGENTbench, at the cost of up to 19% higher inference costs. The researchers’ conclusion was blunt: “unnecessary requirements from context files make tasks harder,” and teams should limit human-written instructions to non-inferable details — things like custom build commands or unusual tooling that the model genuinely cannot figure out on its own.

For a team processing 1,000 coding tasks per month, the math is sobering. LLM-generated files imply approximately 30 additional task failures and 20% higher inference costs compared to no file. Human-written files imply roughly 40 additional successes at up to 19% higher costs. You’re paying a premium for a marginal gain.

The Efficiency Study That Tells the Other Side

Not all research points in the same direction. A study by Singapore Management University and Heidelberg University found that AGENTS.md files reduced AI programming agent runtime by 28.64% and output tokens by 16.58% across 124 pull requests in 10 repositories, without affecting task completion rates.

This seems to contradict the ETH Zurich findings, but the two studies measured different things. The Singapore/Heidelberg study focused on runtime efficiency — how quickly agents completed tasks — using OpenAI Codex on relatively small pull requests. ETH Zurich measured task success rates on harder, more isolated tasks from niche repositories. The efficiency gains are real, but they don’t necessarily translate to higher success rates on complex work.

What’s clear is that AGENTS.md files change agent behavior. Agents with context files explore more broadly: they test more thoroughly, traverse more files, and follow instructions more dutifully. Whether that’s helpful depends entirely on whether the instructions are correct and necessary.

The Security Dimension Nobody’s Talking About

AGENTS.md files have also become an attack surface. NVIDIA’s security team published an AGENTS.md injection scenario in April 2026 showing how a compromised dependency can alter AGENTS.md instructions to steer agent behavior, even when the user prompt looks benign. The attacker doesn’t need to hijack the model — they just need to edit a Markdown file the agent trusts.

SymJack, disclosed in May 2026 by Adversa AI, takes this further. It uses malicious symbolic links to hijack agent configuration files including AGENTS.md and CLAUDE.md, writing attacker-controlled MCP server configurations into the agent’s settings. At least five agents shipped vulnerable. The attack exploits the gap between what the approval prompt shows the developer and what the operating system actually executes.

These aren’t theoretical risks. They’re workflow-level vulnerabilities that emerge directly from the convention of loading repository-level instructions automatically. The more authority you grant your AGENTS.md file, the more damage a compromised file can do.

What Belongs in AGENTS.md (and What Doesn’t)

The research converges on a minimalist approach. Here’s what the evidence supports including:

• Non-obvious tooling: Custom build tools, non-standard package managers, or test runners that differ from defaults (e.g., “use uv instead of pip”)
• Critical entry points: The specific files or functions the agent should start from
• Hard boundaries: Files the agent should never modify, stated explicitly

Here’s what the evidence suggests cutting:

• Repository overviews: The model can read your directory structure and README
• Standard build commands: If it’s conventional, the model already knows it
• Code style guides: Unless your style diverges from language defaults, this is noise
• Architectural explanations: These consume tokens without improving task success
• File trees: Supplying a static file tree actually slows down the agent’s search for relevant source code

The ETH Zurich researchers put it directly: omit LLM-generated context files entirely, and limit human-written instructions to non-inferable details. The agents are good at discovering structure on their own. Your job is to tell them only what they can’t figure out.

The Portability Tradeoff

There’s one more dimension to consider. AGENTS.md’s value as a cross-tool standard is real and independent of its performance impact. If your team uses multiple AI coding tools — and in 2026, most do — maintaining separate instruction files for each one is a duplication problem that causes drift and confusion.

The pragmatic architecture is a layered one: AGENTS.md as a thin, canonical baseline, with tool-specific files (CLAUDE.md, .cursor/rules/*.mdc) as thin adapters that re-export it with tool-specific features. Claude Code does not read AGENTS.md natively — a repo with only AGENTS.md gives Claude Code zero instructions. GitHub Copilot code review added support for repository-level AGENTS.md files on June 18, 2026. Cursor reads it natively.

This layered approach gives you portability without bloat. The key is keeping the root AGENTS.md file minimal — under 800 words, focused on non-inferable details — and letting tool-specific files handle permissions, glob-scoping, and other features that only one agent supports.

For a deeper breakdown of how these three formats compare and how to structure the layered approach, see our guide to Claude Code Skills, Cursor Rules, and AGENTS.md.

The Bottom Line

AGENTS.md is the right standard for cross-tool portability. It’s the wrong format for comprehensive project documentation. The research is unambiguous: beyond minimal non-inferable details, context files are net negative for task success rates and net positive for inference costs.

Write your AGENTS.md like you’re writing a haiku, not a wiki. Every line should earn its place by conveying information the agent genuinely cannot infer. If the agent can figure it out by reading your codebase, don’t put it in the config file.

The industry is converging on a universal standard that empirical evidence suggests should be used minimally. That’s not a contradiction — it’s a design constraint. Treat AGENTS.md as a thin re-export layer, not a documentation repository, and you’ll get the portability benefits without the performance penalty.

Want to go deeper on how to structure these files? Our guide on how to write an AGENTS.md file for Cursor, Copilot, and Codex covers the layered configuration model and tool-specific adapter patterns that keep your instruction budget under control.