Training tokens → wall-clock time and cost-to-train

Memory decides which GPUs can run a training job; compute decides how long it takes. Enter the number of training tokens in the fine-tuning calculator — under Training time (optional) — and every recommended GPU also gets a throughput, a wall-clock time and a cost to train.

The two inputs

• Training tokens — the total number of tokens the run processes, in millions: dataset tokens × epochs. A 200M-token instruction set trained for 3 epochs is 600M. Leave it at 0 to skip the estimate.
• MFU — model FLOPs utilization, the share of the GPU's peak BF16 compute the training loop actually achieves. The default is 40 %; published runs typically land between 30 and 50 %, lower on small batches, PCIe cards and short sequences.

How the estimate is built

The estimate counts the floating-point work per token and divides it by the compute the cluster delivers:

FLOPs per token = f × N + 12 × L × h × s
cluster FLOP/s  = GPUs × peak BF16 TFLOPS × 10¹² × effective MFU
tokens / s      = cluster FLOP/s ÷ FLOPs per token
wall-clock      = training tokens ÷ tokens / s
cost            = unit $/hr × wall-clock hours

f = 6 for full fine-tuning, 4 for LoRA / QLoRA
N = parameters (active parameters for MoE), L = layers, h = hidden size, s = sequence length
effective MFU = MFU × 0.75 with gradient checkpointing

• 6N is the standard estimate for a training step: 2N FLOPs per token for the forward pass and 4N for the backward pass (EleutherAI's Transformer Math). With a frozen base, LoRA and QLoRA skip the weight-gradient half of the backward pass, so the model uses about 4N.
• 12·L·h·s is the attention work, which grows with sequence length and matters at long context.
• MoE models count active parameters: only the routed experts run for a token.
• Gradient checkpointing recomputes the forward pass once, about a third more FLOPs. The model keeps the 6N numerator and lowers the effective MFU by a factor of 0.75 instead, so the reported MFU is not double-counted.

Peak compute is the card's dense BF16 TFLOPS from the catalog. The GPU count is the one the memory step chose for that card — the training-time estimate never adds GPUs just to go faster.

An example

A LoRA run on Llama 3.1 8B at a 2,048-token sequence, 1B training tokens, MFU 40 % with checkpointing (effective 30 %): about 3.5 × 10¹⁰ FLOPs per token. On one L4 (about 121 dense BF16 TFLOPS) that is roughly 1,000 tokens per second and about 270 hours; a card with twice the compute halves the time. The GPU table shows tokens per second, wall-clock hours and the cost of the run per option, so a cheaper card that takes three times longer can be compared with a faster one on total cost.

Reading the cost

Cost to train is the unit's hourly price — the market median per GPU, or the catalog price when no market price exists, times the GPU count — multiplied by the wall-clock hours. It assumes the GPUs are rented for exactly the training time. Data loading, evaluation, checkpoint saves, failed runs and idle time between jobs are not included; budget a margin on top.

The options table is ranked by memory fit, not by cost: the cheapest run is often not the first row. Compare the cost column directly.