AI Coding Agents vs. Delivery Speed: Harvard Jellyfish Data

The most comprehensive study of AI coding agents at scale just landed, and it’s uncomfortable reading for anyone who’s been betting on these tools to accelerate delivery. Harvard researchers Fiona Chen and James Stratton analyzed 300 million work events across more than 700 software firms, and their finding is blunt: AI coding agents generate roughly 30% more code, but they produce no statistically significant increase in shipped software. The productivity gains don’t disappear — they get absorbed by a downstream bottleneck that most teams haven’t rearchitected for.

This isn’t a story about model capability. It’s a story about pipeline design, and it has direct implications for how you spend your AI tooling budget.

What Did the Harvard Jellyfish Study Actually Measure?

The dataset behind this study is unusually granular. Chen and Stratton obtained aggregated analytics from Jellyfish, an engineering analytics platform that tracks the day-to-day output of development teams. The data spans roughly 300 million individual work events — commits, pull requests, and issue management activity — covering more than 700,000 employees at over 700 software firms from January 2021 through March 2026, as Ars Technica reported.

The researchers didn’t survey developers about how productive they feel. They measured operational telemetry: what actually happened in version control, issue trackers, and review workflows before and after each firm adopted AI tools. Adoption timing was inferred from GitHub activity, bot signatures, and licensing API data.

The methodology is a staggered difference-in-differences design, which compares firms that adopted at different times. That’s a strength — it captures real-world usage at scale — but it’s also a limitation. Larger firms adopted earlier, so adoption timing isn’t random, and the authors caution against reading the numbers as purely causal, according to Dark Factory’s analysis. The data is also proprietary (Jellyfish clients who opted in), the sample skews toward firms already using analytics tooling, and the paper remains unrefereed. Treat the findings as a strong signal from an observational dataset, not a settled verdict.

How Did AI Coding Agents Change Code Output?

The raw output numbers are striking. After firms adopted AI coding agents, total lines of code rose about 30%, commits rose 20%, and pull requests rose 23% per worker-month, the Harvard working paper found. Drilling down, the agent effect works out to roughly 1,494 extra lines of code per worker-month on a baseline of 5,017, with commits rising 4.57 on a baseline of 22.58 and pull requests rising 1.22 on a baseline of 5.34, AI Bacon detailed.

The study separates two tool categories, and the difference matters. Reactive AI coding assistants — autocomplete-style tools that help humans write code — increased lines of code by 12% and commits by 9%, though only the commit effect reached statistical significance, Pondero’s analysis shows. Fully autonomous coding agents, which write and submit code with minimal human direction, produced the much larger 30% jump. That distinction is important: if your team is comparing tools, the autonomy level of the agent directly determines how much code volume you should expect — and how much review capacity you’ll need.

A separate 2026 Management Science study of 4,867 developers at Microsoft, Accenture, and an anonymous Fortune 100 company found that developers with AI coding assistants completed 26% more tasks, TechTarget reports. The direction of the effect is consistent across studies. The question is whether those extra tasks translate into shipped features — and that’s where the multi-agent vs. single-agent debate gets interesting. Our benchmarking of multi-agent coding systems found that multi-agent setups only justify their added cost for difficult, decomposable production tasks, not routine work.

Why Didn’t Delivery Speed Improve Despite AI Coding Agents Generating More Code?

Here’s the core finding, and it’s what I call the bottleneck migration pattern: AI coding agents have shifted the bottleneck from code generation to code review. The average time from pull request submission to merge rose 49% after AI coding agent adoption, from a 7.03-day baseline to 10.48 days, Dark Factory notes. That’s roughly 3.45 additional days per pull request across the review queue.

The review burden didn’t just get longer — it got denser. The share of pull requests requiring changes nearly doubled after AI adoption, increasing by 0.12 relative to a baseline mean of 0.13. Comments per pull request increased 35%, and the share of workers performing code reviews increased 14%, the Tech Beat summary reports.

The study authors conclude that productivity gains are “absorbed by downstream constraints in the production process” and that code review becomes a significant bottleneck, Ars Technica covered. They interviewed 21 developers, product managers, and engineering leads to corroborate the pattern. One senior engineer described how a tool like Claude Code “can write a draft in two seconds that’s correct 90 percent of the time” for routine work but “makes frequent mistakes” on harder tasks, “and you end up spending more time reviewing,” the Pondero interview analysis describes.

Bain & Co.’s 2026 Global Technology Report found a similar pattern from the other direction: developers complete roughly 21% more tasks with AI tools, but time spent reviewing outputs rose 91%, CIO Dive reports. Purna Doddapaneni, a partner at Bain, put it plainly: “The bottleneck has moved from writing code to trusting it. Output goes up, but the system settles at the speed of its slowest human checkpoint.”

What Does the Review Bottleneck Actually Cost Teams?

The subscription cost of AI coding tools is only the visible part of the bill. For a 50-developer team, the annual subscription spend ranges from $11,400 for GitHub Copilot Business at $19 per seat per month, per 100xFounder, to $24,000 for Cursor Teams at $40 per user per month, to $75,000 for Claude Team Premium seats at $125 per seat per month, per AIxJo’s pricing comparison. But those are the sticker prices. Usage-based overages can push daily agent costs to $60–$200 per developer, according to the same AIxJo analysis.

Here’s how the three tools compare on the dimensions that matter for a team evaluating them:

Tool	Annual Subscription (50 devs)	Key Differentiator	Best For
GitHub Copilot Business	$11,400 (100xFounder)	IDE integrations across editors, agent mode	Teams already standardized on GitHub
Cursor Teams	$24,000 (AIxJo)	AI-first editor with cloud agents	Teams wanting deep codebase context
Claude Team Premium	$75,000 (AIxJo)	Terminal agent with highest usage limits	Teams running autonomous agents at scale

The real cost, though, isn’t in the subscription. A 50-developer team adopting AI coding agents at the study’s observed rates would generate roughly 74,700 extra lines of code per worker-month equivalent annually (50 developers × 1,494 extra lines × 12 months), a calculation based on the Harvard study data shows — while seeing no statistically significant increase in resolved Jira issues. That extra code has to be reviewed, and the Harvard data says the review process absorbs all of it.

DX’s Q3 2026 AI Impact Report adds another layer. In organizations where 80% or more of code is AI-authored, the median wait for a first review is 82 minutes, and median PR size is up 122% year over year, DX found. Cache-read tokens reached 94% of total token usage — meaning most of the AI spend isn’t generating new code, it’s re-reading existing context to understand what to change. About 33% of AI sessions went to maintenance and 23.3% to fixes, with only 11% going to building new capabilities.

The debugging burden tells a similar story. The Undo survey found that teams spend an average of 9.8 hours per week producing code but 16.9 hours per week debugging, accounting for 42% of the average working week. Thirty-five percent of generated code reaches production before the team has fully understood it, and 80% of respondents said coding agents struggle to solve difficult problems in complex codebases. Seventy-nine percent of engineering leaders say AI agents can generate code significantly faster, but the shift in effort toward debugging means the overall release cycle is “no faster than before.”

The total cost of a 50-developer AI coding deployment cannot be fully projected from the available research. The data lacks review-capacity costs, infrastructure overhead, and the dollar value of the 49% review-time increase that absorbs the productivity gains. What we do know is that the subscription line item understates the real economic impact by an unknown but potentially large margin.

How Should Engineering Teams Respond to the Bottleneck Migration?

The response isn’t to abandon AI coding agents. The response is to stop treating them as a pure productivity tool and start treating them as a workload redistribution tool — which is what the data says they actually are. The bottleneck has moved, and the fix is organizational, not technological.

First, measure the right metrics. Commits, PRs, and lines of code are output metrics. Resolved Jira issues, shipped features, and change failure rates are outcome metrics. The Harvard study found that output metrics moved dramatically while outcome metrics didn’t budge. If your team is optimizing for output, you’re optimizing for the wrong stage of the pipeline, the study’s central finding concludes.

Second, invest in review capacity before scaling agent usage. The study found that the share of workers doing code reviews increased 14% after AI adoption — firms naturally shifted people toward review — but the review queue still grew faster than capacity. Automated code review tools are widely adopted (80% of firms by March 2026), but AI generates only 23.3% of review comments, leaving humans responsible for most review work, the Tech Beat summary reports. The tools are live but the workflow hasn’t caught up.

Third, context is the infrastructure you’re probably underinvesting in. Our post on how AI coding agents build context explores why context decay — not model capability — is the real gap between adoption and productivity. Agents perform better when they have access to current technical standards, product requirements, and organizational knowledge. Maintaining and versioning that context may become as important as maintaining code itself.

Fourth, look at what the organizations that are getting ahead did. Spotify more than doubled monthly merged changes from roughly 8,100 to 17,000 year over year, InfoQ reports. They did it by redesigning quality controls — monitoring, testing, rollout safeguards, and rollback capacity — to handle the higher volume. Anthropic hit a similar wall: merges per day grew tenfold from 2,500 to over 30,000, making source control their biggest development bottleneck, the Anthropic engineering blog describes. They built an in-house version control system (Scion) and a code review surface (Antique) in five months. These aren’t universal templates — they’re proof that the bottleneck is real and that rearchitecting the downstream infrastructure is the only way through it.

The enterprise trend line confirms the direction. VentureBeat Intelligence found that among August respondents whose organization deploys autonomous AI agents, 56% already let an agent push some changes to production on automated evaluation results alone or are building toward it — down from 75% in July, VentureBeat found. The share expecting to keep a human reviewing agents’ production changes for the foreseeable future doubled from 20% to 42% over the same month. Enthusiasm for autonomous deployment is declining as teams confront what the Harvard data describes: the review bottleneck is real, and it’s not going away on its own.

The Bain data suggests the gap between aspiration and reality is widening. Leaders anticipate a 148% improvement in release-cycle speed and a 95% boost in developer productivity over the next one to two years — far ahead of the 20–27% gains organizations are capturing today. The tools are working. The system around them isn’t scaling. Our post on how engineering teams actually adopt AI coding tools found that the highest-performing organizations build orchestration and review infrastructure before scaling agent usage — not after. Until you fix the system, the agents just move the queue from writing to reviewing.