Quick start — three steps

Most sizing questions take three steps in the Inference calculator: describe the model, describe the load, read the answer. Every change re-runs the estimate, so there is no submit button — the result on the right always matches the inputs on the left.

1Pick a model

Open the Model cell and choose one of 141 catalog models, paste a HuggingFace repo ID (org/model), or switch to Manual and enter the architecture: parameters, layers, hidden size, attention and KV heads, head size, vocabulary and, for MoE models, experts and active experts. The catalog and HuggingFace paths fill in the geometry, including MLA and hybrid-attention layers, and suggest the model's native precision. Details: Model input.

2Set the workload

The deck cells next to the model hold the rest: Precision and KV cache (see precision), Users — concurrent requests, each reserving a full context of KV cache — Context, Traffic (typical prompt and answer length) and Runtime. Advanced adds batch, latency targets, LoRA adapters and speculative decoding. What each one moves is in Workload: context, users, batch, traffic.

If your question starts from a latency target rather than a configuration — "how many GPUs for 400 users at 50 ms per token" — open TCO instead: it sizes the fleet against the SLA rather than checking it.

3Read the answer

The answer panel states the smallest configuration that holds the model, with its memory bar (weights, KV cache, activations, runtime overhead) and fit band. Below it, the table lists every GPU that fits with tokens per second per user, TTFT, $/hr and $/1M tokens; open a row for TPOT, total throughput, the parallel layout and live prices. How to read each number: Reading a result.

To keep the answer, copy the address bar — it encodes the whole configuration — or use the share actions (Sharing a configuration). Your last inputs are also kept in this browser: opening the calculator from the site menu restores them, and recent calculations are listed under Tools.