Local inference
AI LLM and GPU calculator
Explore model fit, context length and a theoretical memory-bandwidth ceiling for decoding.
Reviewed · Free to use · Calculations run in your browser
How this calculator works
Weight storage = total parameters × bits per weight ÷ 8. KV-cache storage = 2 × layers × KV heads × head dimension × stored tokens × concurrent sequences × bytes per KV element. The runtime allowance is then added to their sum.
Results show decimal GB (1 billion bytes) and binary GiB (1,073,741,824 bytes). GPU capacity uses decimal GB throughout. The GPU count is a capacity lower bound assuming ideal sharding; divisibility, replication, interconnects and workspace can require more hardware.
These presets model full-attention dense inference. Sliding-window, hybrid, MLA and other cache architectures need their own accounting. For MoE weights use total stored parameters, not active parameters. Training optimizer states and gradients are not included. Qwen3 presets have a native 32,768-token context; longer sequences need compatible configuration.
The decode figure is an illustrative bandwidth ceiling for one sequence: usable memory bandwidth ÷ (weight bytes + KV-cache bytes). It excludes compute limits, kernel overhead and interconnect traffic. It is not measured tokens/second, does not predict prefill speed and is withheld when the model does not fit one GPU or concurrency exceeds one.
Sources & review date
Checked September 11, 2026. Providers can change rates, model availability and specifications. Open the source for the selected preset before budgeting or buying hardware.