Capacity Planner vs Inference calculator

The two tools share one memory and throughput model, but they answer opposite questions. Pick the one whose input you actually know.

Two questions, one model

The Inference calculator answers "for this batch, what fits and how fast?" You give a model, a precision, a context window, a number of users and a batch; it returns every GPU configuration that holds the model with its VRAM fit band, throughput, TTFT, TPOT and $/1M tokens. The batch is an input: you decide how hard each GPU is driven.

The Capacity Planner (TCO in the navigation) inverts it: "how many of which GPUs do I need to serve N users at this latency?" You give a user count and an SLA — a TPOT target, optionally a TTFT target — and it solves for the largest batch per replica that still meets the SLA, then adds data-parallel replicas until all users are covered. The batch is an output.

What differs in the answer

When to use which

• Calculator first when the question is about a model: will it run on the cards we have, which precision, what TP shape, how much headroom at 32K context. It is also the faster way to compare precisions — the Precision cell lists every format for the runtime with its VRAM, speed and smallest GPU.
• Planner first when the question is about a service: a user count from product metrics and a latency target from the product team, and a budget line to fill. Its "You save" card shows what occupancy-aware sizing is worth against reserving the full window.
• Both for a proposal: shortlist GPUs in the calculator, size the fleet in the planner, then load-test the one or two candidates. The calculator's best card has a "Size it in TCO" link that carries the model across.

Latency-bound sizing carries wider uncertainty than VRAM sizing, so treat the planner's fleet as a starting estimate. The assumptions are listed in Capacity Planner — modeling assumptions.