B200 SXM 180GB vs GB200 NVL72 for LLM inference

B200 SXM 180GB vs GB200 NVL72: verdict

Choosing between a B200 SXM 180GB and a GB200 NVL72 for LLM inference comes down to three questions: how much VRAM do you actually need, how fast does each card move tokens once the model is loaded, and what does it cost to rent either one today. On raw capacity, GB200 NVL72 has the edge — 180GB on the B200 SXM 180GB against 186GB on the GB200 NVL72, which matters directly for how large a model or how many concurrent users you can serve on a single card before reaching for tensor parallelism. On raw memory bandwidth, which is usually the real bottleneck for token-by-token decoding rather than compute, GB200 NVL72 is ahead at 7700 GB/s on the B200 SXM 180GB versus 8000 GB/s on the GB200 NVL72. Live pricing wasn't fresh enough for a head-to-head comparison today, so we're showing catalog estimates instead, which is worth weighing against the raw specs if you're optimizing for cost per token rather than peak capability. Neither card is a strictly better choice in every dimension — the right pick depends on whether your deployment is VRAM-bound (a large model, long context, or many concurrent users) or throughput-bound (a smaller model where decode speed dominates unit economics). The spec table above breaks down the full comparison — VRAM, bandwidth, compute throughput at bf16 and fp8, NVLink support, and power draw — and the models table above shows which currently-popular models actually fit on each card.