Estimates, not quotes

Every number on this site is a planning estimate: good enough to shortlist GPUs, compare configurations and put a first figure in a proposal, not a measurement of your deployment and not a price anyone has offered you. This page says how far each kind of number can be trusted and what to do before you commit money.

Memory is a byte budget

VRAM is the sum of explicit terms — weights, KV cache, activations, runtime overhead, NCCL workspace — each computed from the model geometry and the runtime. Against 11 public memory logs, 11 land within −10…+20 % with a median absolute error of 2.1 %. The band is asymmetric on purpose: an underestimate is the dangerous error, because it tells you a configuration fits when it runs out of memory. Runtimes still differ by version in how much they reserve for CUDA graphs and allocator slack, which is why anything above 95 % of the card is treated as not fitting.

Speed is a calibrated roofline

Throughput, TTFT and TPOT come from a roofline model — each stage bounded by compute or memory bandwidth — with per-runtime efficiency factors fitted on published benchmarks: 20 of 29 throughput points and 107 of 167 low-concurrency measurements (batch 1–16) land within ±20 %. Your deployment will differ with the framework version, driver, kernels, scheduler settings and the real prompt mix, so expect ±15–30 % against your own load test, more on the newest GPUs and on AMD, where the calibration set is thinner.

Prices are indicative

Hourly rates are the current median of on-demand list prices that we collect daily from public provider price lists and APIs, or a catalog estimate where no market price exists. They exclude regional variation, committed-use and enterprise discounts, spot volatility, storage, egress and support. Swap in your contracted rate before budgeting; the cost-per-token figures scale linearly with it.

Before you commit

• Take the shortlist, not the single top row, into a load test on the runtime and version you will deploy.
• Measure at your real prompt and answer lengths and at peak concurrency; TTFT under load includes queueing that the estimate leaves out.
• Leave headroom: a tight fit today becomes an out-of-memory error after a context increase or a runtime upgrade.
• Check what is deliberately left out in What is not modeled.