Useful estimates
show their work.
ModelFit separates the three memory buckets that dominate inference planning, then applies explicit headroom instead of pretending the board’s advertised capacity is fully usable.
Model weights
Raw weight memory is parameter count multiplied by bytes per parameter. An overhead multiplier accounts for quantization scales, metadata, and modules retained at higher precision.
weights = parameters × bits / 8 × format_overheadKV cache
KV cache grows with context, concurrent sequences, layers, KV heads, head dimension, and cache precision. ModelFit derives each FP16 full-attention coefficient from the model configuration: two tensors (keys and values) multiplied by layers, KV heads, head dimension, and two bytes per element. For Gemma 2, the estimate also models its alternating local/global attention by capping the local-layer share at the 4K sliding window while global layers continue to grow.
kv = architecture_bytes_per_token × effective_cached_tokens × concurrency × precision_factorRuntime workspace
The launch estimator reserves the larger of 2 GiB or 10% of weight memory for kernels, graph capture, temporary buffers, and allocator behavior. Exact requirements vary by runtime and version.
Fit verdict
The directory’s planning ceiling targets no more than 85% of advertised VRAM. The interactive worksheet also shows nearby options: below 80% is labeled comfortable, 80–92% is a tight fit, and above 92% is rejected. A tight fit can fall above the recommended 85% target and should not be read as equivalent to the directory ceiling.
What the estimate does not promise
It is not a speed benchmark and does not yet model multimodal encoders, adapters, speculative decoding, CUDA graph profiles, pipeline parallelism, CPU offload, fragmentation, or provider-reserved memory. Cache allocation is runtime-specific: some engines reserve blocks or a full static pool differently, and a runtime that does not honor hybrid/sliding cache behavior can use more memory than the Gemma 2 estimate. Production decisions should be validated on the intended runtime and exact hardware.