“How much VRAM does this model need?” sounds like a single-number question. It is really two questions: how much memory loads the model, and how much additional memory serves the workload you expect. A useful estimate keeps those parts visible.
This online LLM VRAM calculator is designed for both local evaluation and cloud inference planning. It does not claim benchmark accuracy; it gives you an auditable starting envelope that you can validate on the intended runtime.
Calculate the deployment envelope.
Select an open model, weight format, context length, concurrent sequences, and GPU count. The result shows each memory bucket and suitable hardware profiles.
Includes 15% operating headroom for the selected runtime profile.
Sample planning rates only—not live Flatkey quotes or availability.
vllm serve YOUR_COMPATIBLE_QUANTIZED_MODEL_ID --tensor-parallel-size 1 --max-model-len 8192 --quantization YOUR_MATCHING_BACKEND4-bit is a memory assumption, not a checkpoint choice. Replace both placeholders with a compatible quantized artifact and its vLLM backend.
Start with four memory buckets.
A practical LLM memory estimate can be expressed as:
recommended VRAM = (weights + KV cache + runtime workspace) / target utilizationModelFit uses an 85% target utilization. Dividing the workload subtotal by 0.85 converts it into recommended capacity with 15% operating headroom. That margin is a planning choice, not a universal runtime constant.
Parameter count multiplied by bytes per parameter, plus format overhead.
Attention keys and values retained for active tokens and sequences.
Kernels, temporary buffers, graph capture, and allocator needs.
Capacity intentionally left outside the expected workload.
Quantization moves the largest block.
For a dense model, a quick raw-weight estimate multiplies parameters by the number of bytes stored for each parameter. BF16 and FP16 use two bytes; 8-bit formats use roughly one byte; 4-bit formats begin around half a byte. Real loaded memory is usually higher because formats store scales and metadata, and some layers may remain at higher precision.
Raw binary conversions shown before quantization metadata, cache, workspace, or headroom.
A downloaded file is not a serving plan.
Model file size can approximate weight storage, but it does not include the KV cache created while generating tokens. Cache usage grows with the model architecture, active context, number of concurrent sequences, and cache precision. Runtime allocations add another layer that can change between vLLM, llama.cpp, TensorRT-LLM, Transformers, and version-specific kernels.
For a local LLM, start with one concurrent sequence and the context you realistically use—not the model’s advertised maximum. For an API workload, use the longest accepted request plus the concurrency your scheduler may keep active. If the runtime supports FP8 KV cache, the cache component can be materially smaller, but support and output quality must be validated. Hybrid and sliding-attention models also depend on whether the selected runtime honors their local-cache window or reserves a larger static pool.
Planning a 70B model at 4-bit.
Take a 70B dense model in a 4-bit format. Raw weights begin around 32.6 GiB. Quantization metadata lifts the loaded weight estimate, then an 8K sequence adds model-specific KV cache and the runtime reserves workspace. Applying headroom can move the recommendation beyond a 48 GB card even when the downloaded weight files appear smaller.
That is why “70B at Q4 fits on 48 GB” is not a universal promise. Context, cache precision, runtime, adapter use, and the quantization implementation can change the result. Use the tool as a shortlist, then verify memory and generation behavior on the exact stack.
See how tokens and concurrent sequences change the KV cache.
LLM VRAM calculator questions.
How much VRAM does an LLM need?+
It depends on parameter count, weight precision, KV cache, runtime workspace, context length, and concurrency. Raw weight memory is only the starting point; a stable serving plan also needs operating headroom.
How do I estimate VRAM from a Hugging Face model card?+
Find the parameter count and architecture on the model card, choose the weight format you will actually load, then add model-specific KV cache and runtime overhead. The parameter count alone cannot determine production memory.
Can I use this calculator for a local LLM?+
Yes. Set one GPU, the intended quantization, context length, and one concurrent sequence for a local starting point. CPU offload and shared-memory behavior are not included, so validate with your local runtime.
Why does a 4-bit model use more than 0.5 bytes per parameter?+
Quantized formats commonly store scales, metadata, and some modules at higher precision. Runtimes also allocate cache and workspace beyond the weight file itself.
Should I plan to use all advertised GPU memory?+
No. ModelFit targets an 85% ceiling so kernels, allocators, graph capture, and workload variation have room. The appropriate margin can differ by runtime and provider.
Compare the hardware profiles.
Use the estimate to shortlist GPUs, then validate runtime support and your workload assumptions on the exact stack.