Nobody Saves Money on the Model
A team just swapped in a model that costs twice as much per token, and their bill went down. Here is why that is not a paradox, and what it means for anyone trying to make AI cheaper at scale.
Last week a team at Cognition, the company behind the coding agent Devin, published a number that reads like a typo. They replaced Opus 4.8 with Fable 5. Fable 5 costs about twice as much per token. Their bill went down.
Not their quality. Their bill.
Here is the part of their table that matters:
Fable 5, inside their new architecture: score 57.6, cost $3.00 per task.
Opus 4.8, on its own: score 48.8, cost $3.24 per task.
The expensive model, wired up correctly, was better and cheaper than the cheap model on its own. Not a trade. Both columns at once.
If you have ever sat in a meeting where someone proposed saving money by dropping down a model tier, that result should stop you. It stopped me, because I had run the opposite experiment, and I had lost.
What they actually built
Cognition split the work between two models instead of routing between them.
The expensive model is the driver. It plans, it interprets what the request actually means, it makes the judgment calls, and it does the final review. It takes very few actions itself and it reads only what it must.
The cheap model is the sidekick, and it is not a helper function. It is a full agent with its own tools that goes and does the mechanical work: fetching context, running the slow tests, carrying out the routine implementations once someone has decided what they are.
Both hold their own memory. Both run at the same time. And this is not a demo. Eighty-eight percent of Cognition's own internal code changes now run through that automated split.
The reason it works fits in one line. You pay for thinking once, and for doing it many times.
The problem is that the opposite is also true
Everyone in this field has met the other result. You move a workload to a cheaper model to save money, and it costs you more. The cheap model misunderstands the task, produces something confidently wrong, and a person spends an afternoon unpicking it. The invoice went down and the total cost went up.
That happens, and I am not going to argue with it, because I have the receipts.
So we have two findings that appear to be at war. Expensive models save money. Cheap models cost money. Both are observed, both are honest, and most of the advice you will read picks one and ignores the other.
I ran the losing experiment
I built a cost router. The logic was the logic everyone reaches for: work out how hard the task is, send the easy ones to a cheap model, keep the expensive one for the hard ones.
It saved nothing. Not a little less than I hoped. Nothing.
I want to be precise about why, because the failure is more useful than a success would have been. I routed by task type. The axis that pays is ambiguity.
Every individual call my router made was defensible. This one looks like a simple rename, send it to the cheap model. That one looks like architecture, keep it upstairs. And the total refused to move, because "simple rename" was a description of the work, not a description of how much had already been decided. Half the jobs I was labelling easy still had open questions inside them, and an open question handed to a cheap model is the single most expensive thing you can buy.
The law
Once you see it, the war stops.
A cheap model is expensive on an open question and cheap on a closed one.
An open question is one where something still has to be decided. What are we actually building. What does done mean here. Which of these two readings of the request is the real one. Give that to a cheap model and it will not tell you the question is open. It will pick an answer, sound sure, and hand you something plausible that you now have to check line by line.
A closed question has had the deciding done. Rename this function in these three files, and the test that proves it is this one. There is no judgment left in the task, and a cheaper model does it for a fraction of the price with nothing at risk.
Which gives the driver model a job description nobody writes down. Its work is not "the hard parts." Its work is to turn open questions into closed ones. And that conversion has a name we already use for it. It is called a plan.
This also explains the finding that keeps embarrassing people who try to build a committee of cheap models and vote. A group at Princeton tested that directly last year and found that running the single best model several times and combining its own answers beat mixing different models together, on every one of the thirteen mixed configurations they tried, using roughly half the forward passes. Their explanation is blunt: mixing models of different quality drags the average quality down. You cannot vote your way to judgment. Where the question is still open, quality dominates, and diversity is not the free lunch it looks like.
Cheap is not a property of the model
Here is where I think the whole conversation is framed wrongly, including by the people getting the right answers.
Everyone calls this "big model, small model." My own setup says that is not the axis.
The models I push my mechanical work to are not small. They are large, capable models. They cost me nothing at the margin, because I bought them on flat monthly subscriptions instead of by the token. That is not a smaller brain. It is a different contract.
So there are two dials, and they are independent. Where does the marginal cost live, and where does the judgment live. A fine-tuned small model is cheap because it was narrowed. A large model on a flat rate is cheap because of how you bought it. Both are "the cheap one," and treating them as the same thing is how people end up sending an open question to a bargain and wondering why the quarter went sideways.
The sentence I had already written
I found a note I made a few weeks ago, while building the thing that hands my grunt work off to those subscription models. I had written the law without recognising it:
A vague spec produces confident garbage, and the cheaper the model, the truer that is.
Read that again as a cost statement, because that is what it is. A cheap model is not a discount. It is a multiplier on the quality of your specification. Specify tightly and it multiplies your savings. Specify loosely and it multiplies your mess.
Which is why the discipline I run alongside it is not optional. The expensive model writes the entire work order: the task, the files, what done means, and the exact command that proves it. And when the cheap model comes back and says the job is finished, that claim is worth nothing. I read the difference it made, and I run the test myself. It has never once been the model's word that closed a task.
So the small-model story is not a cost play
This is the part I keep chewing on, because it inverts something I believed.
Everyone is excited about training small models to do one narrow job extremely well, and the excitement is framed as a cost saving. It is not, or at least the saving is not where people are pointing.
If you can specify a task tightly enough to train a small model on it, you have proved the task is closed. All the ambiguity was removed by somebody, at some point, doing the expensive work of deciding. The training run is just collecting the winnings.
A small model is not a discount. It is a receipt for judgment already spent.
And the same is true of every cost reduction I have ever managed to make stick at scale. Every one of them was bought earlier, by an act of judgment that closed a question. The saving showed up in the invoice. It was created somewhere else entirely.
The boundary, in their words
Buried in Cognition's own write-up, past the numbers, is the caveat I would have led with:
The sidekick fails when judgment is the deliverable.
Their example is a hard feature whose subtle intent got lost the moment it was handed down. The work came back correct and wrong at the same time.
Anyone who has run a team knows exactly where that line sits, and knows it is not about the seniority of the person you handed it to. You can delegate the work. You cannot delegate the judgment about what the work is for. The failure looks identical from the outside either way: something arrives, it is technically defensible, and it is not what the thing was for.
The honest caveat
One thing about that Cognition result deserves saying out loud, because they said it themselves and nobody repeating the headline has.
Their best row was measured on Fable 5 during a stretch when the model was briefly pulled from sale, suspended in June under a US export-control order and restored at the start of July. Their own note adds two more caveats: those numbers were taken before the interruption, and that configuration was never tuned the way the others were. The model is back on sale now. The untuned-config caveat is not.
The pattern transfers. But the exact recipe is one vendor's best-case number on a setup they admit they never optimized, which makes it directional, not a benchmark. A result nobody has reproduced is a claim, not a finding, and that distinction is worth keeping close in a year when every week produces a new number.
What to do with this
You do not save money by hiring cheaper people. You save money by putting your best judgment on the plan, and then making the execution mechanical enough that it does not need judgment. Every leader has run the cheaper-people experiment at some point. Most of us have the scar to show for it.
The unit of cost was never the hour. It was the outcome.
So the question to take into your next architecture review is not which model you are using, or what it costs per million tokens. Those are the numbers on the invoice, and the invoice is a lagging indicator of a decision somebody already made.
Ask what you are paying per solved problem. Then ask who is doing the thinking.
Justin Johnson writes Run Data Run. His book on building with AI, Builder Leader, is at builder-leader.com.