Accuracy and error bands

Every estimate is checked against public measurements: memory logs, published throughput benchmarks, low-concurrency speed tests (batch 1–16) and time-to-first-token runs, each re-scored by the live calculator. This page summarises how close the numbers are and how to read the bands; the accuracy page lists every point with its source.

The numbers today

Beyond these points the prefill model is fitted on the TensorRT-LLM v0.14 performance table: 56 % of its 534 rows land within ±20 %. The last recalibration was 2026-10-24. Against your own deployment, expect ±15–30 % on throughput and latency — more on the newest GPUs and on AMD, where the calibration set is thinner.

Reading the error

Error = prediction ÷ measurement − 1. For memory, a negative error is the dangerous one: it would tell you a configuration fits when it runs out of memory. That is why the memory band is asymmetric, −10 … +20 %, and why anything above 95 % of the card is not shown as fitting. For throughput a negative error means the estimate is conservative — the hardware does better than predicted — and a positive one that it is optimistic.

The worst throughput points today are -66 % and +70 %; they are listed on the accuracy page next to their sources. A point far outside the band usually means a runtime feature the model does not see (an unusual scheduler setting, a kernel that a new release changed) rather than a wrong formula.

Calibration points and check points

Some points were used to tune a constant — an efficiency factor, a tensor-parallel tier, the per-step cost — and are marked calibration; their error shows how well the model reproduces public data. The others are check points that no constant was tuned on, which is the better guide to a setup nobody has measured. Both are pinned by regression tests. Memory points must stay inside their −10 … +20 % band. Every throughput prediction is pinned to within 3 % of its recorded value — a code change that moves any of them by more than 3 % fails the build until the pin is deliberately updated — and a minimum number of them must stay within ±20 %; a single throughput point outside its band is shown on the accuracy page, not treated as a failure. The low-concurrency set is guarded as a whole: its count within ±20 % and its mean error must stay on the right side of fixed limits.

Where the error is largest

• Setups far from the calibration set — an unusual batch, prompt length or parallel layout on a GPU with few public benchmarks.
• New GPUs and AMD — few independent measurements; AMD multi-GPU over xGMI has none yet.
• Hybrid and linear-attention models — memory is exact, throughput has reduced calibration.
• TTFT under load — the estimate is per request on an idle replica; queueing adds to it.
• Agentic and capacity sizing — they chain several estimates (occupancy, hit rate, prefill load), so their uncertainty is wider than a single calculator row.