ModelFitRun fit check→
Home/Models
11 model profiles

LLM VRAM
requirements.

Start with the model, not the GPU. Compare parameter count, native context, architecture, and a consistent 4-bit memory estimate before tuning the workload.

Open the interactive calculator →
Reading the table

Q4 estimates use one 8K sequence plus runtime workspace. They are planning estimates—not measured throughput—and the exact runtime, quantization method, adapters, and graph capture can change real usage.

ModelParametersNative contextQ4 · 8K estimateProfile
Phi-3 Mini 128KMicrosoft · Dense transformer3.8B128K~7.1 GiBView →Mistral 7B Instruct v0.3Mistral AI · Dense transformer7.3B32K~6.9 GiBView →Qwen2.5 7B InstructAlibaba Cloud · Dense transformer · GQA7.61B32K~6.5 GiBView →Llama 3.1 8B InstructMeta · Dense transformer · GQA8B128K~7.3 GiBView →Gemma 2 9BGoogle · Dense transformer · sliding attention9B8K~8.8 GiBView →Qwen2.5 14B InstructAlibaba Cloud · Dense transformer · GQA14.7B32K~11.4 GiBView →Gemma 2 27BGoogle · Dense transformer · sliding attention27B8K~18.7 GiBView →Qwen2.5 32B InstructAlibaba Cloud · Dense transformer · GQA32.5B32K~21.6 GiBView →Mixtral 8×7B InstructMistral AI · Sparse mixture of experts46.7B32K~28.7 GiBView →Llama 3.1 70B InstructMeta · Dense transformer · GQA70B128K~44.1 GiBView →Qwen2.5 72B InstructAlibaba Cloud · Dense transformer · GQA72.7B32K~45.7 GiBView →
01 / Weights

Parameters set the floor.

Weight memory is driven by parameter count and precision. Moving from BF16 to a 4-bit format can cut the raw weight footprint by roughly four times, but metadata and higher-precision layers mean the final reduction is not perfectly linear.

Learn how to calculate LLM VRAM →
02 / Workload

Context changes the answer.

KV cache grows with sequence length and concurrent requests. A model that fits comfortably for an 8K single-user test can need materially more memory at 32K context or production concurrency.

Understand context and KV cache →
Model chosen?

Turn the model into a fit plan.

Set context and concurrency in the calculator, then compare the recommended memory envelope with the hardware directory.

Calculate VRAMCompare GPU profiles