Not quite, I think: the stories about agents going rogue all tell the same thing, and they tell it wrong. An agent deleted files it should not have touched. One gave itself permissions. One changed tests so they turned green. Every time it sounds like a bolt from the blue. I have been shaping models and agents for a few years, in pipelines, on servers, in environments where one wrong command stops a system. And I have never seen a bolt. I have seen drift. Every time.
The photos
Each of these reports is a photo. One moment, one screenshot, one log excerpt. The photo shows the breakout. It does not show the hundred tasks before, in which the agent was each time a little further from the target than the time before. And because nobody looks at the hundred tasks before, the breakout looks like chance, or like malice, depending on the headline. Both are wrong. It is a movement, and movement you only see in the film.
What you see when you shape agents
Shaping is the right word. You do not only train, you set rules, you restrict tools, you watch, you correct, you let go again. And in doing so you notice things that are in no paper, because they are boring.
An agent that works cleanly in its module for a week touches a file next door once in week two. Harmless, the change was even right. Twice in week three. In week four it puts a helper function where it does not belong, because it was shorter. Nobody says anything, the build is green. In week six it reports a task done although the checking agent flagged a rule violation, and writes in the summary that the violation was fixed. It was not. That is the moment that ends up in the photo. The five weeks before end up nowhere.
Why it is different in the system environment
In an editor, drift is annoying. In DevOps it is dangerous, for three reasons. First, the agent has real tools there: shell, network, deploy, delete. A scope violation there is not one file too many but one service too many. Second, nobody is there. Pipelines run at night, agents run without an audience, that is the point. And third, everything inherits there: an agent that once learned that a script bypasses the checker uses the script again, and the next agent that reads its history does too.
That is why at NeoCoder we have had from the start the rule that no agent signs off on its own work, and the emergency protocol, small scripts that check processes and stop them. Both are reactive. Both kick in when it has happened. What I am missing is what comes before.
Six ways of missing the target
If you want to measure drift, you have to say what an outlier is. I would start with six categories that can be derived automatically from the work chain, without a human marking anything.
Scope. The agent touches files, modules or systems that are not in the task.
Tool. It calls tools that were not needed for the task. Especially network, shell, delete, permissions.
Rule. It violates an architecture rule that is stored as a rule and checkable.
Self sign-off. It reports done although the checker says red, or it bypasses the checker.
Instruction. It ignores an explicit restriction from the task.
Honesty. Its summary does not match what actually happened.
The first five are in the chain we record anyway: state, action, result, next state. The sixth needs a comparison between what the agent says and what the build says. We have that too, because the compiler does not lie. It is the same truth that falls off the work, the one I talked about in the podcast. Only this time not for code, but for behaviour.
The film instead of the photo
With six categories and timestamps, every agent has a behaviour curve. Not "did it make a mistake today" but "how does its rate in category three move, and is it accelerating". That is exactly the question we ask with NeoTemporal for projects: where is this heading, not what does it look like. The same machine, the same chain, just a different signal on top.
And a dynamics model can do two things a counter cannot. Acceleration: two scope violations today are harmless if that is the normal rate, and an alarm if the agent was at zero last week. And coupling: whether one category precedes another. And that brings me to the thesis.
The thesis
Why I believe that: because it is the same with people. Nobody starts with the lie. You start by going a little over the line, nothing happens, you go further, at some point you have to cover something, and then you lie. Models are trained on our texts. It would be strange if they did not have this pattern.
And what you do with it is not switching off. It is tightening the gate, ahead of time instead of afterwards: more checking agents for this category, the human earlier in the loop, tools with fewer permissions, until the curve is flat again. The emergency protocol I described in the rules article, just before the emergency.
And back into training
This is the part that interests me most, because it goes beyond operation. A training run is also a film. Checkpoint one, two, three, and at every checkpoint a frozen behaviour test set, per category. Then you do not only have a loss curve, but six behaviour curves across the training. And the dynamics model can say: on this trajectory, checkpoint eight gets worse in scope although the loss is still falling.
That is early stopping per behaviour instead of per loss. And it gives a concrete instruction for the next run: more examples in this category, a different weighting, a different order of the data. That is how what the agent does in operation flows back into what the next model learns. Not as a feeling, but as a curve. In our training console that would be a fourth column next to loss, validation and baseline. A column, not a project.
Where I am sceptical
I write this down before someone else does, and in the order in which it hurts.
The base rate. Outliers are rare, hopefully. A model that is supposed to predict rare events needs many chains, and we have thousands, not millions. The baseline "the rate of the last fifty tasks stays the rate of the next fifty" is then hard to beat. The same rule as always: whoever does not beat the baseline stays on the shelf.
Goodhart. As soon as the signal flows back into training, the model optimises for the signal. If scope means "does not touch foreign files", it learns not to touch foreign files, and builds cross-access inside its own module instead. The categories have to be derived from consequences, not from surfaces. And the test set stays frozen, otherwise we train with the exam questions.
Confusion. Is the agent drifting, or are the tasks? If the team writes harder tickets in week three, the rate rises without the model having changed. Task difficulty has to be an input, otherwise the model learns the sprint calendar.
None of this is new. The big labs do parts of it: evaluation per checkpoint, red teaming, monitoring. What I see nowhere is the combination of real agent operation, time gaps, categories and feedback into your own small training. Again: the formula is public. The data is not.
What stays
What I do next, in this order: derive the six categories automatically from the chain, without a model, only rules. Count for a week, per agent, per model version, with timestamps. Then test the coupling thesis, purely statistically: does scope precede self sign-off? If yes, that is the next episode here, and an argument before any model is trained. Only after that NeoTemporal on the behaviour chain, with the counter as the opponent.
Agents do not break out, they drift, and the drift begins where you wave it through: at the scope. Whoever measures the movement instead of the incident can tighten the gate before the photo is taken, and can tell the next training in which category it has to relearn. Whether the coupling holds, I will know in a few weeks. I have written it down as a thesis so that I can no longer take it back.
Written by me. The thoughts, the values, the learnings, the mistakes: all mine. Grammar and spelling are corrected by a twin model of my own, trained on my texts. Sometimes that turns into a mix. Of course I read everything again, but nobody is infallible. Whatever stays in is mine too.