About howmanygpus.ai

howmanygpus.ai answers one question for LLM deployments: how many GPUs, which ones, and what they cost to run — for inference, fine-tuning and edge devices.

GPU catalog48 GPUs · 10 Jetson modules

Pre-filled models141

Cloud providers, last 7 days14

Calibrated on 11 VRAM measurements · 29 throughput benchmarks · 167 low-batch points · How accurate is this?

What it is

A sizing tool for ML and infrastructure engineers, presales and architects. You describe a model, a precision, a runtime, a context length and a load; it returns a VRAM breakdown, expected speed (tokens/s, time to first token, time per output token) and cost per hour and per 1M tokens.

Calculator — the GPUs that fit a model at a given context and load, ranked fewest-GPUs-first.
What fits my GPU — the reverse question: which models run on a card you already have.
Optimize — every GPU × precision × parallelism configuration that fits, across all context lengths, on one cost/speed map.
TCO — GPUs to serve N concurrent users (or agentic sessions) at a latency target, with replicas and cost per session.
Fine-tuning — training memory for full fine-tuning, LoRA and QLoRA, with optimizer choice and ZeRO / FSDP sharding.
Edge / Jetson — fit and per-power-mode speed on NVIDIA Jetson Orin and Thor modules, for text and camera (vision) workloads.

The catalog covers 48 GPUs · 10 Jetson modules — data-center, workstation and consumer GPUs (NVIDIA and AMD) and NVIDIA Jetson edge modules — with 141 pre-filled models; any other model can be loaded from its Hugging Face config.

Cloud prices are collected once a day from the public price lists and pricing APIs of cloud providers — hyperscalers and GPU clouds; 14 providers had prices in the last 7 days — and kept as history: per-GPU charts, a price index and RSS feeds.

What it is not

Not a quote and not a benchmark. Every number is an estimate from published specifications and a calibrated model. Validate a shortlisted configuration with a load test on the target stack before you buy or sign.
Not affiliated with NVIDIA, AMD or any cloud provider shown. Product and company names belong to their owners.
No ads, no paid placements, no affiliate links. Nobody pays to appear in a ranking: recommendations are ordered by GPU count, fit and price, computed the same way for every card.
No paywall. The free tool is fully functional, without an account.

How the numbers are made

Memory is an explicit byte budget: weights at the chosen precision + KV cache + activations + runtime overhead + communication workspace. The KV model handles MLA (DeepSeek, Kimi), sliding-window and linear-attention hybrids, so long contexts are not overestimated.
Speed is a calibrated roofline: generation is bounded by memory bandwidth (about 80% of the datasheet figure is treated as achievable), prompt processing by compute, with per-runtime efficiency, tensor-parallel communication and a per-step scheduler cost. The formulas follow NVIDIA's open-source aiconfigurator.
Calibration. The model is recalibrated against public measurements: 11 VRAM measurements · 29 throughput benchmarks · 167 low-batch points (single users and batches up to 16), plus 4 llama.cpp time-to-first-token runs. Every point, its source and our error is on the Accuracy page, recomputed by the live calculator — How accurate is this?
Measured error. VRAM: median absolute error 2.1 %, 11/11 within −10…+20 %. Throughput: 20/29 within ±20 % (median absolute error 15.5 %); single-user speed: 107/167 within ±20 %. Against your own deployment expect ±15–30 % on throughput and latency, more on the newest and non-NVIDIA GPUs.
Prices are on-demand list prices per GPU-hour. Euro prices are converted to US dollars at the day's reference rate. Each price shows its provider count and update date; stale data is flagged.

How accurate is this? · Methodology and data sources · Knowledge base · Engineering notes

Who builds it

Siarhei Harlinski

I'm Siarhei Harlinski, based in Warsaw. I design AI infrastructure end to end — from GPU servers and high-speed networking to a platform an enterprise can run in production. My stack: NVIDIA Hopper and Blackwell and AMD Instinct GPUs, NVLink and InfiniBand; VMware and Kubernetes with GPU sharing (MIG, vGPU), Slurm for training; NVIDIA AI Enterprise, NIM and NeMo; vLLM, TensorRT-LLM and Triton for serving. Then I tune it for performance and cost — throughput, latency and price per token. howmanygpus.ai is my independent project to make that sizing fast and honest for everyone. No employer, GPU vendor or cloud is behind it.

Contact

Found a wrong spec, a missing GPU or a price that looks off? Send the page link and what you expected.