Fine-tuning
Training memory and cost for full, LoRA and QLoRA runs.
-
01
Fine-tuning calculator — full, LoRA, QLoRA
Training memory for the three methods: what lives on the GPU in each and how much VRAM it takes.
4 min read · updated 2026-10-26 -
02
Optimizers and ZeRO / FSDP sharding
Optimizer state bytes per parameter and how ZeRO stages and FSDP shard training state across GPUs.
4 min read · updated 2026-10-26 -
03
Training tokens → wall-clock time and cost-to-train
From training tokens and MFU to tokens per second, wall-clock time and the cost to train on each recommended GPU.
4 min read · updated 2026-10-26