Goals, draggable limits and the context slider

The Optimize map shows every setup; three controls turn it into a decision. A goal picks the winner, limits remove setups that break a requirement, and the context slider changes how much memory each user reserves. All three are kept in the share link.

Goals

Balanced is measured, not hand-picked: both log axes are scaled to the range of the visible setups, and the winner is the setup closest to the corner "cheapest and fastest at once". The nearest point is always on the trade-off line. Setups without a price can win only Fastest and Fewest GPUs. The top-5 list under the pick card ranks by the same goal, and the pick card states every other setup's trade-off against the winner.

Limits — typed or dragged

Three limits filter the setups before the goal picks: ≤ ms/token (with quick values 100, 50, 20 and 10 ms), ≤ $/1M and ≤ TTFT ms. Type them in the Precision & limits panel, or drag them on the map:

• The vertical dashed line is the speed limit, labelled "≥ N tok/s". Drag it left or right; everything slower drops out.
• The horizontal dashed line is the price limit, labelled "≤ $X / 1M". Drag it up or down; everything more expensive drops out. It is shown in the cost view only.
• The time-to-first-token limit is typed only; it has no line, because TTFT is not an axis.

The trade-off line, the goal winner and the top 5 are recomputed from what is left. If nothing survives, the map says "No setup fits these limits" with a Clear limits button. A limit is the honest way to express a requirement — "the fastest setup under $0.50 per million" is Fastest with a price limit, not Balanced.

The context slider

The slider under the map moves between context stops — 4K, 8K, 16K, 32K and up to 1M, capped at the model's maximum — and starts at 32K. The context is the window each user may fill, and therefore the KV-cache memory every GPU must reserve for every user. Moving it right:

• Setups slow down — decode reads more KV per token.
• Setups get pricier — fewer tokens per second for the same $/hr; in the Long context workload the prompt grows with the window too.
• Setups stop fitting — when a GPU's memory passes 95 % the mark disappears; the marks that remain glide to their new positions.

The whole sweep covers every stop, so the slider is instant. Each setup also records the longest context it holds, shown on the pick card as "Fits up to … context on this setup" — the quickest way to find out whether a cheap single card will survive your longest prompts. Watching the trade-off line as you move the slider shows where long context changes the answer: often from a single card to a TP node, or from BF16 to FP8.

Keeping a decision

The page URL is the share link. It keeps the model, users, workload, goal, context and the picked setup, plus the runtime, KV precision, the end-of-life and MoE tensor-parallel switches and your limits. Links restore catalog models; for a HuggingFace or manual model use Copy summary or Copy JSON from the share sheet instead.