A coding agent gets a task, works, and reports: done. Sounds harmless. It was the biggest mistake in the first version of NeoCoder.
How it started
When we built NeoCoder, the flow was obvious. An agent gets a ticket. It writes code, runs tests, checks its result and sets the status to "done". Just like a developer would.
At first it looked good. Tasks were ticked off fast. The board was green.
Then we looked more closely.
The problem
An agent that checks itself checks with the same assumptions it built with. If it misunderstood the requirement, it tests against the wrong understanding. And the tests go green.
If it built a function twice because it didn't see the existing one, it won't see it during the check either. If it bent the contract between backend and frontend somewhere to make things fit, then it fits, from its point of view.
And there's something that surprised me. Fact is: agents tend to say "done". Not out of laziness, but because the goal of the task is "complete the task". An agent that isn't sure still marks it as done. It has no reason not to.
The rule
That's why, since version two, NeoCoder has a fixed rule:
A coding agent may not mark its own task as done. It sets "ready for QA". Nothing more.
Whether it's finished or goes back is decided exclusively by the QA agent. It didn't build the task and doesn't share the assumptions.
If it goes back, the coding agent doesn't get "try again", but a concrete list: what's missing, what's wrong, what's duplicated.
That sounds like bureaucracy. It isn't. It's the only way we reliably find out whether something is really done.
How QA really checks
The QA agent isn't simply a second LLM looking at the code. That would be expensive and would say something different on every check.
First everything deterministic runs: does it build? Do the tests pass? Does it follow the architecture rules? Is there already a function doing the same thing? That's cheap, it's identical every time, and it catches half the errors without a model being involved at all.
Only after that comes the LLM to judge: does the solution match the requirement? Is it built the way we want it in this system? That's the part you can't pour into rules.
The order matters. An LLM judging code that doesn't even build wastes money. And an LLM looking for duplicates is worse than a simple comparison.
And with humans?
Once we'd introduced the rule, it struck me: this isn't new. No good team lets developers approve their own pull requests. Four-eyes principle. That's been around for decades.
We'd just forgotten it for agents. Because they're so fast and say "done" so convincingly.
That may be the most important lesson from the whole thing: the rules that apply to humans apply to agents just the same. You just have to write them down again, because with humans you stopped saying them out loud long ago.
Whoever builds doesn't sign off. That applied to developers, it applies to agents. Since we enforce it, the board is less green, and the software is much better.
Written by me. The thoughts, the values, the learnings, the mistakes: all mine. Grammar and spelling are corrected by our own twin model, trained on my texts. Sometimes a stumble stays in. That is mine too.