Three reasons to self-host, only one of which is money
The first is regulatory or contractual: an obligation that forbids sending certain data to a third party. This is not a cost argument and should not be evaluated as one — you are buying a guarantee, and it is worth paying more for.
The second is architectural: an air-gapped environment with no outbound connectivity. There is no API option, so the comparison does not arise.
The third is cost at sustained volume, and it is the only one where the maths decides. It is also the reason most often cited and least often calculated properly.
What the cost comparison actually includes
The API side is simple: tokens in, tokens out, at a published rate. Easy to model, scales linearly, no fixed commitment.
The self-hosted side has four components, and people routinely count one. There is hardware, whether capital or rented by the hour. There is power and cooling, which is not trivial for a machine drawing hundreds of watts continuously. There is engineering time to build the deployment, which is real money spent once. And there is ongoing operational time — driver and CUDA versions, model updates, capacity planning, and being the person who fixes it when it stops at an inconvenient hour.
That fourth item is the one that flips conclusions, because it never appears on an invoice. A GPU server needing a few hours of skilled attention each month has a running cost that competes with a considerable API bill.
Utilisation is the variable that decides it
Owned hardware costs the same whether it runs at five per cent or ninety-five. An API costs nothing when idle. So the comparison is not really about price per token — it is about how much of the day your hardware is genuinely busy.
A GPU serving an internal tool that thirty people use during office hours is idle most of the time, and idle hardware is the most expensive kind. The same GPU running a continuous batch classification job over a document backlog is a very different proposition.
Bursty workloads favour APIs strongly. Steady, high, predictable throughput favours self-hosting. If you cannot describe your load pattern yet, you are not ready to make this decision — which is a legitimate finding rather than an evasion.
Capability, described honestly
Open-weight models have closed much of the gap and are entirely sufficient for a large share of production work: classification, extraction, structured output, summarisation, and retrieval-grounded question answering. If that is your workload, capability is not the deciding factor.
On the hardest reasoning, long-context synthesis and agentic multi-step tasks, the frontier hosted models remain ahead, and the gap is real rather than a marketing artefact. Anyone telling you an open 8B model matches a frontier model across the board is either benchmarking narrowly or not benchmarking.
The way to settle it for your case is an evaluation set built from your own data, run against both. It costs a fraction of the deployment it informs, and it converts an argument into a measurement.
The hybrid answer, which is often the right one
These are not mutually exclusive, and treating the choice as binary is the most common error. Sensitive documents get processed by a local model; everything else goes to an API. Straightforward requests route to a small self-hosted model; hard ones escalate to a frontier API.
This needs an abstraction over the model layer, which you want anyway — providers change pricing and deprecate versions, and a system that can swap models on the strength of an evaluation run treats that as routine maintenance rather than a crisis.
How to actually decide
Start with the constraint question: is there an obligation that rules out sending this data to a third party? If yes, self-host and stop optimising for cost.
If no, build the evaluation set, run it against both options, measure your real load pattern for a few weeks on an API, then compute the total cost of ownership including operational time. The answer will be specific to you, and it will be a number rather than a preference.
If someone gives you that answer before knowing your volume, your data sensitivity and your load shape, they are guessing.