How much VRAM does Nemotron-3 Nano 30B-A3B need?

About Nemotron-3 Nano 30B-A3B

Running an Nemotron-3 Nano 30B-A3B in bf16 at a reference 8K context and batch size 1 takes 67.3GB of VRAM — a number our calculator derives from the model's published architecture (layer count, hidden size, attention heads), not a rough rule of thumb. Nemotron-3 Nano 30B-A3B is a Nemotron model with 32B total parameters, with only 3.2B active per token via mixture-of-experts routing, and at this batch size that VRAM figure is dominated by the weights themselves, with KV cache and framework overhead adding a smaller but still meaningful slice on top. Which GPU you actually need depends heavily on quantization: bf16 is the reference precision with no quality loss, but fp8 roughly halves the weight footprint, AWQ (weight-only int4) goes further still, and — on Blackwell hardware — NVFP4 offers a similar 4-bit footprint with hardware-native throughput. At bf16, the smallest configuration that fits this model is 1× H100 PCIe 80GB; the dtype table above shows exactly how that changes as you quantize down, including whether a smaller or cheaper GPU becomes viable. Context length is the other lever: KV cache grows roughly linearly with how many tokens you keep resident, so a deployment serving long documents or long conversations needs meaningfully more headroom than this 8K reference number suggests — see the context-sensitivity table for how VRAM scales at 32K and 128K where this model's native context allows it. If you're sizing a real deployment rather than just checking whether a model fits, feed these same architecture numbers into the full calculator to model your actual concurrency and serving framework.