A model that maps company descriptions to a catalogue of around 900 activities. Sounds like a manageable task. It is, until you start counting.
The task
A company describes in a few sentences what it does. The model should say which of the 900 activities in the catalogue fit. Not one, often several. And the descriptions are the way people write: incomplete, full of jargon, sometimes in three languages at once.
900 categories means: many of them sit very close together. The difference between two activities is sometimes a single word. That is exactly where models get unsure.
How the model is built
We didn't leave it to a single model. That works badly with 900 classes.
The embeddings pick the twenty or thirty candidates out of the 900 that are semantically close. That's fast and cheap. The LLM then decides between those candidates. That's slower and more expensive, but it only sees the cases where it's really about nuance.
And the feedback loop collects everything that went wrong. That is our most valuable data. Not the 49,000 examples that were right. The thousand that weren't.
Let's do the maths
Until the model was usable we needed 30 to 50 training runs. Each run with around 50,000 examples, one to two hours on an A100 or H100.
At around 3 dollars per GPU hour that is 150 to 250 dollars of raw compute for development. In production we retrain monthly, which adds a few dozen dollars a year.
Fact is: that is nothing. For one single model like this, compute is not the problem.
Where the money really goes
The problem is the 50,000 examples.
Someone has to collect them. Check whether the mapping is right. Extend them with synthetic data so that rare categories have enough examples too. Add hard negatives, meaning cases that look similar but belong to a different category. And after every training run, evaluate again: is the new version really better than the old one, or just different?
Part of that runs through LLM APIs. Generating synthetic examples, proposing mappings, flagging errors. Per pass that quickly costs a four-figure amount. Considerably more than the training itself.
Someone has to know that "plumbing" and "heating installation" are two different activities, even though the same company often does both. Someone has to decide what happens when a description fits no category at all. That is domain knowledge, and it is the most expensive part of the whole project.
What I take from it
When someone asks me today what a model costs, I ask back: who understands the data?
Not: which GPU do we need. Not: which framework. But: who sits next to it and says whether the answer is right? And how often do we have to do that until it's good enough?
The answer to those questions sets the price. Everything else is a rounding error.
150 dollars for the GPU. A four-figure amount for the API. And the most expensive part is the person who has the 900 categories in their head. That is what a "small" model really looks like.
Written by me. The thoughts, the values, the learnings, the mistakes: all mine. Grammar and spelling are corrected by our own twin model, trained on my texts. Sometimes a stumble stays in. That is mine too.