Ok, the numbers first: with AI agents, developers write 30 percent more lines of code and deliver 23 percent more pull requests. And the result does not change. That is what a Harvard study of 700,000 developers in 718 companies says, and t3n ran it this week under the headline that AI does not really make developers more productive. I read the article twice, and I think the headline is wrong. The study says something else, and that is more interesting: the bottleneck has moved. From writing to understanding. Exactly where I have suspected it for a year.
The numbers
Fiona Chen and James Stratton of Harvard analysed data from the engineering platform Jellyfish: 300 million work items, January 2021 to March 2026, over 700,000 employees in 718 companies. Three metrics: lines of code, commits, pull requests.
Reactive assistants, meaning autocompletion in the editor, bring according to the study 12 percent more lines, 9 percent more commits, 5 percent more pull requests, and only the commits are statistically solid. Semi-autonomous agents, which take over whole tasks, lift all three: 30 percent more lines, 20 percent more commits, 23 percent more pull requests.
And then the part that made the headline: no measurable effect on company output, no effect on employment. Instead, the average time to review a pull request rises by 49 percent. The share of pull requests where changes are requested nearly doubles. Comments per pull request rise by 35 percent. The share of people doing reviews rises by 14 percent. And that stays, even when the company introduces AI tools for review.
Plus 21 interviews. A senior developer says, in substance: for routine work the draft is there in two seconds and right nine times out of ten. For demanding tasks it does not save much, it makes mistakes, and you spend the time checking. A junior: the AI does 95 percent, and the remaining 5 percent consist of finding out what it got wrong.
The bottleneck has moved
Every system has a bottleneck, and whoever removes one gets the next. That is not an AI insight, that is the assembly line. For decades the bottleneck in software development was writing: a human types, thinks, types, and that takes time. So we optimised everything around it, better editors, frameworks, libraries, autocompletion.
Now writing is practically free. I wrote that in "Code is getting cheap", and the study delivers the number: 30 percent more, without anyone working longer. But the code still has to pass through a human before it may go to production. And the human does not read 30 percent faster. He reads slower, if anything, because he trusts the code less, and rightly so: almost twice as many pull requests come back with change requests.
That is the developer from "Only press Enter", just with numbers. The man in the post described how he confirms outputs twelve hours a day. The study says: companies are pushing their people exactly there, 14 percent more reviewers. They are turning developers into checkers. And checking does not scale, because understanding does not scale.
We have been measuring the wrong thing for years
Here is the point I find more important about the study than the result: the three metrics themselves. Lines of code, commits, pull requests. Those are the numbers on every dashboard, and those are the numbers AI pushes up most easily. Writing more lines was never the goal. Fewer lines for the same result would be one. I wrote down in "Only press Enter" what I would measure: how much complexity we removed, how well the team understands the system, whether the product got better.
And that is the nasty thing about the 30 percent: they look like success. A CTO who only sees the dashboard buys even more agents. The lines keep rising. Review gets even longer. And at some point the whole team sits in review and nobody builds anymore.
Why review takes longer
The study says review takes longer. It does not say why. I say what I see in projects, as experience, not as measurement.
The agent does not know the system. It knows the file, maybe the folder. It does not know the decision from two years ago about why the module is built this way, and not the dependency nobody documented. So it builds something that is locally right and globally wrong. The reviewer has to have the global picture in his head. That takes time.
The agent writes too much. Two thousand lines where two hundred would do. Duplicates, hard wiring, its own helper functions instead of the existing ones. I called that the giant classes in the enterprise piece. Every one of those lines someone has to read.
The agent signs off on its own work. It reports "done", and the human starts from zero. There is no intermediate step where someone else, human or agent, has checked the rules before the reviewer looks. So the reviewer checks everything, style, architecture, logic, security, in one pass.
There is no memory. The reviewer objected to exactly the same pattern yesterday. The agent does not know that today. Every pull request is the first. That is why change requests double: they are the same change requests, every day.
None of these four points gets better when the model gets bigger. They get better when the system around the model is built differently.
How I would build it
Not speed up a developer at typing. The study just measured that, it brings 30 percent more lines and zero more result. But build the whole process so that the code arriving at the reviewer has already been through everything else.
The system before the code. The agent does not get the task, it gets the task plus the architecture rules plus the decisions that belong to this module. That is the knowledge build-up I wrote about in "Software is a film": what the project knows about itself has to be in the context before the first line is written. Otherwise the human checks afterwards what the agent could have known beforehand.
Less code as the goal. The agent that delivers two hundred lines where it could have delivered two thousand has won. That has to be in the task and has to be measured. A sprint in which code disappears is a good sprint.
Four eyes before a human looks. Whoever writes does not check. A second agent checks the architecture rules, a third the tests, one the security, each only its own thing, and never the one that wrote. The human then gets a pull request where style and rules are already done, and checks what only he can check: whether it is the right thing.
A memory for reviews. Every change request from a human is a data point. If the same reviewer objects to the same pattern three times, that belongs in the rules, automatically. Change requests have to go down, not double. If they do not, the system is built wrong, no matter how many lines it writes.
How we do it at NeoCoder
This is not just theory, it is the reason NeoCoder is built the way it is built. When we started, the first decision was not which model to use. The first decision was: the agent may not write a line before it knows the system. That sounds banal. It is the opposite of what most tools do, which start with the file that happens to be open.
Concretely, at our place it means: before an agent gets a task, NeoCoder builds up the knowledge about the project. Which modules there are and where their boundaries lie. Which architecture rules apply, and as rules that are checked, not as a document nobody reads. Which decisions were made earlier and why. Which dependencies exist between areas, including the ones that are in no diagram but only in the git history. That is the chain I wrote about in "Software is a film": state, action, result, next state, recorded with every work package.
And what have we improved with it? Three things I see, still without the numbers I demand in the next section.
The agent builds where it belongs. Because it knows the boundaries, it does not put a second helper function next to the existing one and no cross-access between modules that should not know about each other. That used to be the most frequent change request in review. It is not gone, but it has become rare.
No agent signs off on its own work. We built that in from the start and it is the rule that brings the most. Whoever writes does not check. Another agent checks the rules, one the tests, and only then comes the human. He sees a pull request that is already through the boring part.
The review flows back. A change request from a human does not stay lying in the pull request. It becomes an entry in the project knowledge, and with the next work package the agent knows this rule exists. That is the part that needs the most discipline, because it is tempting to just make the correction and move on.
None of this is magic. It is order: first the system, then the code. And exactly this order is what the study is missing. The 30 percent more code arise with tools that start with the code. We start with the system and would rather have 30 percent less review. Whether that is true comes now.
What I have to prove myself
And now the place where I hold myself to account, because otherwise I do exactly what I accuse the others of. Everything in the last section is a thesis. The study proves the bottleneck. It does not prove that my approach solves it. Only a measurement can, and I do not have it yet.
What we have to measure at NeoCoder, with and without knowledge build-up, on the same tasks: time per review, the share of pull requests with change requests, the number of comments, and the defect rate after merge. Exactly the study's numbers, just here. If review time drops with knowledge build-up and change requests go down, the thesis is confirmed. If not, I will write that here too. Like with the model that played a draw against the baseline and stayed on the shelf.
What stays
The study is good because it is honest. 30 percent more code, 49 percent longer review, zero effect at the end. The headline above it is wrong. AI does not make developers less productive, it shifts the work to a place where we have not organised it yet. Whoever buys more agents now buys more review. Whoever builds the system around the agents, with knowledge before, checks in between and memory after, may get what everyone promised. May. That is the part each of us has to measure ourselves.
It is not the speed at which code is produced that decides the next years. It is whether someone understands the system that comes out of it. The study shows we have moved the bottleneck, not removed it. The answer is not less AI and not more reviewers. The answer is a process in which the code that reaches the human is already small, rule-compliant and known. Whether that works has to be measured. I am starting with that.
Source: Noëlle Bölling, "Laut Harvard-Studie: Warum Software-Entwickler durch KI nicht wirklich produktiver sind", t3n, 9 October 2026, on Fiona Chen and James Stratton, "Artificial Intelligence in the Enterprise: Bottlenecks in Software Production", Harvard, data from Jellyfish. Figures and quotes paraphrased. t3n.de
Written by me. The thoughts, the values, the learnings, the mistakes: all mine. Grammar and spelling are corrected by a twin model of my own, trained on my texts. Sometimes that turns into a mix. Of course I read everything again, but nobody is infallible. Whatever stays in is mine too.