How much VRAM does Nemotron-3 Super 120B-A12B need?

About Nemotron-3 Super 120B-A12B

At 120B total parameters, with only 12.0B active per token via mixture-of-experts routing, Nemotron-3 Super 120B-A12B sits firmly in the territory where quantization choice, not just raw parameter count, decides which GPU you end up buying or renting. In bf16 at a reference 8K context and batch size 1, the model needs 244.2GB of VRAM — computed the same way our interactive calculator computes it, from Nemotron's published architecture rather than a generic per-parameter estimate — and the smallest configuration that clears that bar is 1× AMD MI355X 288GB. Compare that to fp8, which roughly halves the weight footprint at essentially no measurable quality cost on most tasks, or AWQ and NVFP4, which push weight-only compression to 4 bits and can drop the requirement by roughly three-quarters relative to bf16 — often enough to move the model down an entire GPU tier or free up room for a much larger batch of concurrent requests. The full breakdown for all four dtypes is in the table above, alongside the exact GPU configuration each one unlocks. None of this accounts for context length, though: KV cache scales with how many tokens you keep resident, so a long-context deployment needs meaningfully more headroom than this 8K reference — the context-sensitivity table shows the same bf16 model at 32K and 128K wherever its native window supports it, so you can see the real cost of long-context serving before you commit to hardware.