Reading a result: VRAM, fit band, $/1M

A calculator result has three layers: a one-sentence answer with its memory bar, a speed and cost readout for that answer, and a table of every configuration that fits. This article goes through each number in the order worth reading them — memory first, because a configuration that does not fit has no speed.

The answer sentence

The headline names the smallest configuration that holds the model at your users and context: "fits on one H100", "needs 2× H100 (TP2)", "runs across 2 nodes of 8× H200" or "runs on a GB200 NVL72 rack". It is the top row of the table, picked by the recommendation order — single node before multi-node before rack-scale, live cards before end-of-life ones, then fewest GPUs, then fit band, then price (why that order). Use as answer on any row replaces it; the share actions then follow your choice.

When nothing fits on one node, the sentence says so and suggests the usual levers: an 8-bit format for 16-bit weights, a shorter context or fewer users.

Memory: the bar and the fit band

The memory bar is per GPU of the answer. Its segments are the terms of the byte budget:

• weights — parameters × bytes per parameter of the chosen precision, divided across tensor- or expert-parallel GPUs;
• kv — the KV cache: one full context window per concurrent user, sharded by attention heads (an MLA latent cache is replicated instead);
• runtime — the serving engine's fixed overhead: CUDA context, allocator slack and, on vLLM and SGLang, CUDA-graph buffers (per runtime);
• act — activations for one scheduler step, bounded by the prefill chunk;
• hatched extras when they are more than a sliver: the NCCL workspace on multi-GPU units, LoRA adapters, a speculative draft model and the fixed state of linear-attention layers.

The sum against the card's VRAM gives the utilization and the fit band: under 50 % underused, 50–70 % optimal, 70–80 % good, 80–95 % tight; above 95 % the configuration is not listed (Fit bands). If the bar is mostly KV cache, users and context are your levers; if it is mostly weights, precision and GPU count are.

Speed: four numbers that answer different questions

At batch 1, total output equals the decode rate and the prompt shows up only as TTFT. Above batch 1 a busy server prefills new prompts while it decodes, so each output token carries a share of prefill time; that is why total output is lower than "streams ÷ TPOT" for prompt-heavy traffic. The ✓ and ✗ marks next to TTFT, TPOT and total throughput appear only when you set targets under Advanced.

Cost: $/hr and $/1M tokens

$/hr is the unit price of one GPU × the GPU count. The unit price is the current market median across the providers we collect on-demand prices from, or the catalog estimate when the market has no price; the open row says which ("market median · N providers · as of …" or "catalog estimate"). $/1M tokens turns it into the cost of a million output tokens with the unit busy around the clock:

$/1M = $/hr ÷ (total output tok/s × 3600) × 10⁶

total output tok/s = 1 / ( 1/decode + (prompt ÷ answer) / batched prefill )   at batch > 1

Example with round numbers: decode at 2,000 tok/s and batched prefill at 40,000 tok/s. A chat turn (1K → 256) costs 1/2,000 + 4/40,000 s per output token, so total output is about 1,670 tok/s. A RAG turn (8K → 256) costs 1/2,000 + 32/40,000 s, about 770 tok/s — the same unit at the same $/hr is roughly twice as expensive per output token. $/1M is the most comparable number across GPUs because it folds GPU count, price and speed into one figure; it assumes full utilization, so divide by your expected average load for a budget.

Inside a row

Opening a row adds the tier and interconnect chips, an EOL marker where it applies, the price source, power draw in kW, the parallel layout (tensor, pipeline or expert parallel), short reasons why this GPU was chosen and, where wider tensor parallelism on the same card would cut TPOT by at least 1.25×, a Lowest latency line with that layout's numbers. Actions: Use as answer, Pin as baseline (every other row then shows its difference), Add to compare (up to four rows), the GPU page, live cloud prices and the 90-day cost history of that configuration.

What-if panels and notes

Under the answer, What if more people use it? re-runs the answer at other user counts and What if prompts get longer? at other context lengths, so you can see where the configuration tips into the next band or the next GPU count. Notes below the result flag what the estimate cannot see: a precision with no native path on the answer GPU, a runtime that does not batch, a context far longer than your typical request, the per-request nature of TTFT.