TCO · Capacity Planner
GPUs to serve N users at a latency SLA, chat or agentic.
-
01
Capacity Planner — step-by-step sizing guide
Size GPUs for a user count and a latency SLA: inputs, the anchor card, alternatives and how to read the result.
6 min read · updated 2026-10-26 -
02
Agentic mode — sizing session-based workloads
Turn a session profile (turns, prefix, sub-agents, compaction, cache) into active streams, prefill load and cost per session.
10 min read · updated 2026-10-26 -
03
Pareto explorer — the idea, on an example
Every feasible configuration plotted by cost and latency, with the frontier marked, explained on a worked example.
4 min read · updated 2026-10-26 -
04
Capacity Planner — modeling assumptions
The assumptions behind the sizing: statistical multiplexing, shared prefix, prefill load and the risk quantile.
4 min read · updated 2026-10-26 -
05
KV risk gauge and occupancy trend
What the KV risk gauge and the occupancy trend show and how the preemption-risk setting changes the reserve.
4 min read · updated 2026-10-26 -
06
Capacity Planner vs Inference calculator
The calculator answers what fits at a batch size; the planner inverts it from a user count and a latency SLA.
3 min read · updated 2026-10-26