The Hidden Costs of Self-Hosting an LLM — and When Not to Do It

We deploy private LLMs for a living, which is exactly why you should believe us when we say: a lot of people shouldn't do this. The failure pattern is consistent — the plan is priced off the GPU quote, the GPU turns out to be the cheap part, and eighteen months later there's a very expensive space heater running last year's model because nobody owned the upgrades.

Here is the full bill, and the decision flowchart.

The model is the cheap part

Split a realistic self-hosting budget into its parts and the thing everyone obsesses over — the model — is a rounding error. Downloaded weights and their storage run 2–5% of total deployment cost. The hardware you carefully specced is only about a third. The rest is the part that doesn't appear on any quote: the infrastructure wrapped around the hardware, and the humans wrapped around the infrastructure.

Worked mid-points from the 2026 cost analyses cited below; your ratios will vary, the ranking rarely does.

Two of those bars deserve their own sentences.

The 2.5–3× multiplier. Whatever the GPUs cost, plan for two and a half to three times that once power, cooling, networking, redundancy and hosting arrive. This isn't pessimism; it's the consistent finding across the 2026 TCO analyses, and it's the line item that turns "we found a great deal on an A100" into a budget review.

The forever engineering. A production inference stack consumes, conservatively, 10–20 hours of senior engineering a month — driver and CUDA regressions, inference-server updates, security patches, model migrations when publishers retire versions, and eval runs so the migration doesn't silently change your product. At $75–150/hour that's $750–$3,000 a month, indefinitely. If nobody in your organisation wants that job, you don't have a self-hosting plan; you have a self-hosting incident scheduled for later.

The costs that aren't on any chart

Three more, harder to draw. Opportunity cost — those engineering hours came from your product. Capability lag — the API gives you the newest frontier models the day they ship; your rack gives you what you installed, until someone does the migration work. Utilisation risk — the economics of ownership assume the cluster stays busy; if usage disappoints, the API bill would have shrunk with it, but depreciation doesn't.

When not to self-host

Plainly: don't self-host if your sustained volume is below the low millions of tokens a day; if your workload is spiky and interactive rather than steady; if no one owns the 10–20 monthly hours; if your use case genuinely needs frontier-model quality against your acceptance criteria; or if the motivation is vibes — "we should own our AI" is a slogan, not a business case. The 2026 consensus is not subtle: for the majority of teams, once the full picture is costed, the API is cheaper.

Sovereignty is the great override — if your data legally can't transit a third-party API, the cost debate is moot and the question becomes how to self-host well, not whether. And genuine sustained volume past the break-even band is the other real trigger.

The flowchart

Four questions, in order: is the data barred from APIs? Are you past ~2–5M tokens a day, sustained? Can you fund the engineering hours without resenting them? Would an open-weight model pass your written acceptance criteria? Sovereignty alone can justify a deployment. Everything else needs at least three yeses.

Why we tell you this

Because our readiness assessment is worth its fee precisely because it sometimes says no. You bring your volumes, your data constraints and your use case; we run the numbers and the evals and hand you a written answer — a module-priced deployment spec if ownership is justified, and "stay on the API, here's what to do instead" if it isn't. A written "don't" is the cheapest thing we sell; it regularly saves a full private deployment and the space heater that follows. And if the answer is build, Maintain exists so the forever-engineering is somebody's actual job — with a monthly report to prove it.

[Book a consultation →](/book) — a 30-minute call, no commitment.

Sources - SitePoint — Self-Hosted LLM Costs 2026 (2.5–3× multiplier; engineering hours and rates) - SitePoint — Local LLMs vs Cloud APIs: 2026 TCO Analysis (weights 2–5% of cost; API-wins-for-most conclusion) - GIGAGPU — Is Self-Hosting LLMs Cheaper Than APIs in 2026? - Alpacked — Self-Hosted LLM Guide: Costs, Architecture & Breakeven

Worked figures are mid-points from the cited analyses; run your own numbers before buying anything with a fan.