The Hidden Costs of Self-Hosting an LLM — and When Not to Do It

We deploy private LLMs for a living, which is exactly why you should believe us when we say: a lot of people shouldn't do this. The failure pattern is consistent — the plan is priced off the GPU quote, the GPU turns out to be the cheap part, and eighteen months later there's a very expensive space heater running last year's model because nobody owned the upgrades.

Here is the full bill, and the honest flowchart.

The model is the cheap part

Split a realistic self-hosting budget into its parts and the thing everyone obsesses over — the model — is a rounding error. Downloaded weights and their storage run 2–5% of total deployment cost. The hardware you carefully specced is only about a third. The rest is the part that doesn't appear on any quote: the infrastructure wrapped around the hardware, and the humans wrapped around the infrastructure.

Worked mid-points from the 2026 cost analyses cited below; your ratios will vary, the ranking rarely does.

Two of those bars deserve their own sentences.

The 2.5–3× multiplier. Whatever the GPUs cost, plan for two and a half to three times that once power, cooling, networking, redundancy and hosting arrive. This isn't pessimism; it's the consistent finding across the 2026 TCO analyses, and it's the line item that turns "we found a great deal on an A100" into a budget review.

The forever engineering. A production inference stack consumes, conservatively, 10–20 hours of senior engineering a month — driver and CUDA regressions, inference-server updates, security patches, model migrations when publishers retire versions, and eval runs so the migration doesn't silently change your product. At $75–150/hour that's $750–$3,000 a month, indefinitely. If nobody in your organisation wants that job, you don't have a self-hosting plan; you have a self-hosting incident scheduled for later.

The costs that aren't on any chart

Three more, harder to draw. Opportunity cost — those engineering hours came from your product. Capability lag — the API gives you the newest frontier models the day they ship; your rack gives you what you installed, until someone does the migration work. Utilisation risk — the economics of ownership assume the cluster stays busy; if usage disappoints, the API bill would have shrunk with it, but depreciation doesn't.

When not to self-host

Plainly: don't self-host if your sustained volume is below the low millions of tokens a day; if your workload is spiky and interactive rather than steady; if no one owns the 10–20 monthly hours; if your use case genuinely needs frontier-model quality against your acceptance criteria; or if the motivation is vibes — "we should own our AI" is a slogan, not a business case. The 2026 consensus is not subtle: for the majority of teams, once the full picture is costed, the API is cheaper.

Sovereignty is the great override — if your data legally can't transit a third-party API, the cost debate is moot and the question becomes how to self-host well, not whether. And genuine sustained volume past the break-even band is the other honest trigger.

The flowchart

Four questions, in order: is the data barred from APIs? Are you past ~2–5M tokens a day, sustained? Can you fund the engineering hours without resenting them? Would an open-weight model pass your written acceptance criteria? Sovereignty alone can justify a deployment. Everything else needs at least three yeses.

Why we tell you this

Because our readiness assessment is worth $9,500 precisely because it sometimes says no. You bring your volumes, your data constraints and your use case; we run the numbers and the evals and hand you a written answer — a module-priced deployment spec if ownership is justified, and "stay on the API, here's what to do instead" if it isn't. A $9,500 "don't" is the cheapest thing we sell; it regularly saves a $45,000 build and the space heater that follows. And if the answer is build, Maintain exists so the forever-engineering is somebody's actual job — with a monthly report to prove it.

[Book a consultation →](/book) — free 30-minute call, no commitment.

Sources - SitePoint — Self-Hosted LLM Costs 2026 (2.5–3× multiplier; engineering hours and rates) - SitePoint — Local LLMs vs Cloud APIs: 2026 TCO Analysis (weights 2–5% of cost; API-wins-for-most conclusion) - GIGAGPU — Is Self-Hosting LLMs Cheaper Than APIs in 2026? - Alpacked — Self-Hosted LLM Guide: Costs, Architecture & Breakeven

Worked figures are mid-points from the cited analyses; run your own numbers before buying anything with a fan.