Guide
LLM API vs self-hosting: the break-even math
The question is not whether a GPU is cheaper than an API. It is whether you can keep a GPU busy enough. A per-token API bills only for what you use. A dedicated GPU bills for every hour it is switched on, busy or idle. Self-hosting wins only once your steady monthly volume is large enough to spread that fixed cost thinner than the API's per-token price.
1. Per-token APIs
Cost scales linearly with volume: monthly tokens × blended price per 1M, where the blended price weights input and output by your traffic mix. With 75% input tokens, GPT-6.2 Sol works out to about $4.00 per 1M tokens. No minimum, no idle cost, and the provider handles scaling and uptime. The LLM pricing calculator adds caching, batch and agent loops to this side of the comparison.
2. Hosted open-weight APIs
Inference providers serve open models such as Llama 3.3 70B Instruct on shared GPUs and still bill per token, here about $0.155 per 1M blended. Because they keep their GPUs saturated across thousands of customers, they are usually far cheaper than both a frontier API and a GPU you rent yourself. Compare hosted open-weight prices in the LLM API price table. For many teams, this, not self-hosting, is the real alternative to a closed API.
3. Dedicated cloud GPUs
A self-hosted deployment costs hourly rate × 730 hours × GPUs per replica × replicas per month, regardless of traffic. One NVIDIA H100 80GB at $2.39/hour is $1,745 a month. Its capacity depends on decode throughput: how many output tokens per second the GPU can generate with continuous batching in vLLM, SGLang or TGI. At full utilization, the setup above produces tokens for about $0.8555 per 1M. That is the best case. At 30% average utilization the effective price rises to $2.85 per 1M.
How to find your break-even point
Break-even is the monthly volume where the two cost lines cross: break-even tokens = GPU monthly cost ÷ API price per token. Below it, the API is cheaper; above it, the GPU is, until you exceed one replica's capacity and need another. For GPT-6.2 Sol against Llama 3.3 70B Instruct on a NVIDIA H100 80GB, the crossover is about 872.35M tokens per month. The chart above plots all three options so you can see where the lines meet.
Four inputs move the break-even point the most:
- Utilization. Traffic that peaks in office hours and drops at night leaves rented GPUs idle. Bursty workloads favour APIs.
- Model size and quantization. A 70B model at FP16 needs several 80 GB GPUs per replica; at 4-bit it fits on one. Size the hardware with the LLM VRAM calculator.
- Input/output mix. Prefill (input) is several times faster than decode (output), so input-heavy workloads such as RAG and classification get more tokens per GPU-hour.
- Which API you compare against. Against a flagship model, a GPU can pay off at hundreds of millions of tokens a month. Against a budget or hosted open-weight API, it may never pay off.
When to self-host an LLM
Price is rarely the only reason. Self-hosting makes sense when you need data to stay inside your own infrastructure, want to serve a custom fine-tune, need guaranteed capacity without rate limits, or have high, steady volume that keeps GPUs above roughly 50% utilization. Budget for at least two replicas in production, plus the engineering time for deployment, monitoring and upgrades, which this calculator leaves out. If those reasons don't apply, a per-token API, often a hosted open-weight one, is usually the cheaper and simpler choice. Voice workloads follow the same logic; see the AI voice agent cost calculator for per-minute costs.