Fit bands: underused · optimal · good · tight
How VRAM utilization maps to the four fit tags and why more than 95% is excluded.
On this page · 4 sections
Every configuration the tools return carries a fit band: how much of each GPU's memory the model, its KV cache and the runtime's overhead take. The band is a quick read of headroom — not a quality score — and the same thresholds apply in the calculator, What fits, Optimize, the fine-tuning calculator and Edge.
The bands
| Band | VRAM used per GPU | What it means |
|---|---|---|
| Underused grey | < 50 % | More GPU than the workload needs; a smaller card would do. It runs, with room for a longer context or more users. |
| Optimal green | 50–70 % | Headroom for longer context, no paying for empty memory — room for traffic spikes and runtime growth too. |
| Good lime | 70–80 % | Well used, with some room to grow; comfortable for production. |
| Tight amber | 80–95 % | Fits, little room for longer prompts or more users — a runtime update can push it over. Validate before you commit. |
| Excluded | > 95 % | Not shown. Above 95 % the estimate's error band overlaps the card's capacity. |
The memory bars use a smooth colour scale for the same thresholds (grey below 50 %), while the pills are discrete. Utilization is computed per GPU on the unrounded estimate: weights and KV cache divided across the serving unit, plus what every GPU repeats — activations, framework overhead and the NCCL workspace.
Why 95 % is the cut-off
Memory estimates are explicit byte budgets and land close to measurements — 11 of 11 public logs within −10…+20 %, median error 2.1 % — but runtimes also allocate CUDA graphs, fragmentation slack and allocator reserves that differ by version. Above 95 % of the card that difference decides whether the server starts, so the tools treat it as "does not fit" and move to the next GPU count instead.
Bands and ranking
A band is not the first ranking key. The calculator orders by fewest GPUs first, then fit band, so one card at a tight fit ranks above two cards at an optimal one (recommendation order). Underused is ranked last among the bands because it usually means a smaller or cheaper card exists. What fits lists underused models too — on a card you already own, they are the ones that leave room for long context. The fine-tuning calculator ranks by closeness to about 85 % instead, because a training run has a steady footprint.
Moving a configuration between bands
- Context and users grow the KV cache linearly; at long context this is usually the term that tips a model from good to tight. See Context length and what drops out.
- KV-cache precision — FP8 or INT8 halves the KV cache against FP16.
- Weight precision — FP8 halves the weights against BF16, 4-bit formats quarter them.
- More GPUs — tensor or expert parallelism divides the weights and most of the KV cache.
- GPU
- L40S
- Memory
- 48 GB
- Users
- 1
- Context
- 8K