Local inference
LLM inference memory calculator
Plan GPU memory for model weights, KV cache and runtime overhead, with clear GB and GiB units.
Reviewed · Free to use · Calculations run in your browser
How this calculator works
Weight storage = total parameters × bits per weight ÷ 8. KV-cache storage = 2 × layers × KV heads × head dimension × stored tokens × concurrent sequences × bytes per KV element. The runtime allowance is then added to their sum.
Results show decimal GB (1 billion bytes) and binary GiB (1,073,741,824 bytes). GPU capacity uses decimal GB throughout. The GPU count is a capacity lower bound assuming ideal sharding; divisibility, replication, interconnects and workspace can require more hardware.
These presets model full-attention dense inference. Sliding-window, hybrid, MLA and other cache architectures need their own accounting. For MoE weights use total stored parameters, not active parameters. Training optimizer states and gradients are not included. Qwen3 presets have a native 32,768-token context; longer sequences need compatible configuration.
Sources & review date
Checked September 11, 2026. Providers can change rates, model availability and specifications. Open the source for the selected preset before budgeting or buying hardware.