Model price is an obvious place to start, but it is also only one place a system can spend.
Once a system can control context, routing, model choice, and execution placement separately, cost becomes an architecture question. Those controls can change what reaches the model, where requests go, and which infrastructure bears the work even when the underlying model choice stays the same.
So instead of stopping at which model is cheapest?, I would ask:
Which layer should own this cost decision, and what does that layer trade away to spend less?
Model price still matters, but it stops being the whole map.
Five places a system can spend
A useful decomposition looks like this:
workload
↓
context policy
↓
routing / gateway policy
↓
model + provider choice
↓
execution placement
↓
result
measurement watches the path and checks whether the saving actually happened
This is not an implementation order that every system should copy; it is a way to separate decisions that are easy to collapse together.
Context policy decides how much state is allowed into the paid generation path. Routing policy decides which classes of work may use which routes. Model choice sets a capability and price envelope. Execution placement decides which infrastructure bears the work. Measurement decides whether the cheaper-looking path was actually cheaper for the workload that matters.
The same model can sit underneath very different cost behavior because the controls around it are different.
Context can get cheaper without changing the model
Kapa's July 2026 write-up is the cleanest example in this source set.
Kapa added a small listwise model between retrieval and generation. It sees the question and the retrieved chunks together, then removes chunks the final generator is unlikely to need. At the operating point Kapa selected, it reports about 68% of retrieved chunks removed, about 96% recall preserved on its labeled test, and about a 34% reduction in per-query cost after the pruner's own cost. The same configuration added about 0.7 seconds of latency.
Those numbers belong to Kapa's workload, retrieval pipeline, threshold, models, and measurement setup. They are not a general RAG benchmark, and they definitely do not support the much looser "10x" wording that prompted this investigation.
The useful part is where the control sits. The final generator's price did not change. The system changed how much paid material reached it.
That is a real lever, but it is not free. Less context can mean lower spend, and it can also mean recall risk, another model call, and more latency. Long histories, retrieved files, tool outputs, scratchpads, and repeated instructions create the same basic question: which state has earned the right to stay in the expensive path?
That question exists before model shopping starts.
The diagnostic version of that risk is in The Context Window Is the Last Mile: once a detail has been compressed away, adding capacity later does not bring it back, so what a pruner drops becomes a debugging problem as well as a cost one.
Routing is a spending rule
The second layer sits between the application and whichever models or providers it can use.
An earlier reference pointed to a Northwood Systems project called foreman; reopening it now resolves to the project named Millwright. Its README describes a self-hosted LLM router with operator-defined cheap, mid, and frontier roles. Policy decides which role a request may use; the router selects the lowest estimated-cost healthy route allowed inside that role and can preserve provider/model affinity for prompt-cache reuse.
That design matters less to me as a specific tool recommendation than as a boundary. The application can describe the work, while a gateway can own the rule about which route that work is allowed to consume.
Once that boundary exists, "use the frontier model only when the task requires it" stops being a convention scattered across application code and becomes explicit policy.
But a router can also make the system worse. It can destroy cache locality, send work to a cheaper model that is not good enough, add failover and health-check state, or create one more service that has to be understood when something fails. Millwright's own cost-analysis documentation draws an important line here: it separates measured ledger evidence from modeled candidate economics and treats candidate results as cost hypotheses rather than quality results or production routing recommendations.
Routing is not a discount. It is a place to put a spending rule.
Self-hosting moves the cost surface
Deployment is another decision that often gets folded into model choice.
A model may be available behind a provider API, through a managed inference service, or on infrastructure controlled by the operator. Even inside self-hosted execution, there are separate questions about what lives in fast memory, what is streamed from storage, and how CPU and GPU work overlap.
Colibrì makes that separation unusually visible. Its project documentation treats VRAM, RAM, and storage as one inference hierarchy, with placement, scheduling, I/O, residency, and CPU/GPU overlap as system-level choices. It also says plainly that there is no SLA on speed and that optimizations have to earn their place through reproducible end-to-end measurement. The profitable configuration depends on the machine and workload.
That is the qualification I would keep.
Self-hosting does not make cost disappear; it changes what the bill contains. Provider pricing can bundle hardware, serving, utilization risk, power, capacity planning, and operations into a per-token number. Moving execution back under operator control can expose those costs directly as hardware, energy, storage, engineering time, maintenance, reliability, and idle capacity.
Sometimes that shift will be worth it. Sometimes the provider invoice is the cheaper problem to have.
Model choice still belongs in the picture
None of this makes the model itself unimportant.
A model can still dominate the bill after context is pruned. A routing policy still needs actual candidate models. Self-hosted infrastructure still has to run something. Quality, latency, tool behavior, context limits, licensing, hardware fit, and unit economics remain model-level constraints.
The difference is that the catalogue no longer tells the whole story.
Two systems can use the same model and send radically different amounts of context. Two systems can have access to the same model set while one sends every request to the most expensive option and the other reserves it for a narrower task class. Two systems can run the same open-weight model while paying very different infrastructure and operational costs.
The catalogue leaves out more than cost. The Exit Route Is Part of Model Choice works the same gap from the dependency side, where access conditions, interface assumptions, and where durable state lives can make a technically available alternative unusable in practice.
The model identity can stay constant while the cost behavior changes around it.
Two systems, one model, two different bills.
Estimated savings are not workload evidence
The final layer does not execute the request at all; it measures whether the supposed saving was real.
Frugon is useful here because its documentation is explicit about the difference between its default offline analysis and workload-specific measurement. Its normal analysis reads call logs and estimates switching or routing opportunities. It warns that the quality assumptions behind those estimates are not guarantees for a particular set of prompts, and it provides an optional path that samples the actual workload through candidate models.
That distinction is bigger than Frugon:
price-table comparison → estimate
replayed real workload → stronger workload model
observed production usage → measured cost
measured quality + cost → decision evidence
The labels will vary by system. The important thing is not to let them collapse into one another.
A cheaper model can create more retries. Context pruning can save tokens and lose something the answer needed. Routing can reduce unit cost and hurt cache behavior. Self-hosting can reduce provider spend while increasing operational burden. A path that is cheaper per call can still be more expensive per successful answer, completed workflow, or reviewed artifact.
Cost per token is useful. Cost per useful work is harder to compute.
Every control layer has a tax
There is an obvious failure mode in this framework: turning cost control into architecture for its own sake.
A pruning layer adds another model call and latency. A gateway adds another service and policy surface. Self-hosting adds infrastructure and operational responsibility. Measurement adds instrumentation and analytical work. Supporting several models can create compatibility and quality drift.
For a small workload, any one of these layers can cost more than it saves.
So I would give every cost-control layer a deletion test:
If this layer disappeared, would we lose more in useful-work economics than we save in complexity?
If the answer is no, the layer probably has not earned its place.
This is not an argument that architecture is always more important than models. For a one-call application, model price may be the dominant and perfectly sensible optimization target. Complexity has a fixed cost. It needs a reason to exist.
Cost control is not free, and it is not exempt from its own test.
The question I would keep
"Which model should I use?" is still a valid question.
Once the system has more than one source of spend, I would add another one beside it:
Which layer should own this cost decision?
Sometimes the answer is the model catalogue. Sometimes it is the context budget, the router, or the deployment boundary. Sometimes the saving is still an estimate, and the right next step is measurement rather than implementation.
A model has a price. A system has an economy.
The architecture determines how much of that economy any single model price is allowed to control.
One number on a pricing page. Five places the spending actually happens.
// End of transmission. The bill has layers. — ZYANE