01

Open-model deployment planner

Will this
model fit?

Turn model size, quantization, context, and concurrency into a practical GPU plan—before you reserve a machine.

Open Flatkey Compute ↗Check a model ↓

INPUT Model + workload→CALCULATE Weights + cache + overhead→OUTPUT GPU plan + command template

02 / Calculator

Build a deployment worksheet.

Start with an open model, then adjust the details that actually change memory: precision, context length, concurrent sequences, and tensor-parallel GPU count.

Deployment worksheetMODEL / GPU FIT

Calculated estimate

Recommended VRAM

8.6 GiBper GPU × 1

Includes 15% operating headroom for the selected runtime profile.

Deployment next stepCheck live capacity on Flatkey Compute.

Carry this workload estimate into the product and review currently available GPU options.

Continue to Flatkey Compute ↗

Estimated allocation per GPU7.3 GiB each

Weights 4.3 GiBKV cache 1.0 GiBRuntime 2.0 GiB

Example cloud fitsBrowse profiles →

1× NVIDIA T416 GB each · Turing

$0.20sample / hour

1× RTX 409024 GB each · Ada Lovelace

$0.32sample / hour

1× NVIDIA L424 GB each · Ada Lovelace

$0.43sample / hour

Sample planning rates only—not live Flatkey quotes or availability.

Command template · vLLM

vllm serve YOUR_COMPATIBLE_QUANTIZED_MODEL_ID --tensor-parallel-size 1 --max-model-len 8192 --quantization YOUR_MATCHING_BACKEND

4-bit is a memory assumption, not a checkpoint choice. Replace both placeholders with a compatible quantized artifact and its vLLM backend.

Compare GPU profiles →

03 / Model index

Choose the model.
We’ll map the hardware.

Curated model cards keep parameter counts, context limits, architecture notes, and source links separate from the calculator itself.

Explore the full model directory →

Phi-3 Mini 128KMicrosoft · Phi-3

Q4 · 8K estimate~7 GiB

Mistral 7B Instruct v0.3Mistral AI · Mistral 7B

Q4 · 8K estimate~7 GiB

Qwen2.5 7B InstructAlibaba Cloud · Qwen2.5

Parameters7.61B

Q4 · 8K estimate~7 GiB

Llama 3.1 8B InstructMeta · Llama 3.1

Q4 · 8K estimate~7 GiB

Gemma 2 9BGoogle · Gemma 2

Q4 · 8K estimate~9 GiB

Qwen2.5 14B InstructAlibaba Cloud · Qwen2.5

Parameters14.7B

Q4 · 8K estimate~11 GiB

Gemma 2 27BGoogle · Gemma 2

Q4 · 8K estimate~19 GiB

Qwen2.5 32B InstructAlibaba Cloud · Qwen2.5

Parameters32.5B

Q4 · 8K estimate~22 GiB

Mixtral 8×7B InstructMistral AI · Mixtral

Parameters46.7B

Q4 · 8K estimate~29 GiB

Llama 3.1 70B InstructMeta · Llama 3.1

Q4 · 8K estimate~44 GiB

Qwen2.5 72B InstructAlibaba Cloud · Qwen2.5

Parameters72.7B

Q4 · 8K estimate~46 GiB

04 / Decision path

One estimate.
Three useful answers.

01

Does it fit?

Break total memory into weights, KV cache, and runtime workspace instead of relying on parameter count alone.

02

What should I rent?

Match the per-device estimate to practical GPU profiles and keep enough headroom for stable serving.

03

$

How do I launch?

Carry context, cache precision, and parallelism into a command template. Quantized plans still need a compatible checkpoint and backend.

05 / Planning library

Go deeper than
a single number.

Use the hardware profiles and field guides to understand why the estimate changes—and what to validate before renting a machine.

Hardware directory

Compare GPU VRAM profiles

See which starter models fit across 16 GB to 192 GB hardware envelopes.

9 profiles →Guide 01

How to calculate LLM VRAM

Break memory into weights, KV cache, runtime workspace, and operating headroom.

Read guide →Guide 02

Context windows and KV cache

Understand how active tokens, concurrent sequences, and cache precision use memory.

Read guide →

Ready when you are

Stop guessing.
Run the fit check.

Open the full calculator →