Local performance. This profile uses an 85% planning ceiling, leaving approximately 3.6 GB outside the recommended workload envelope.
These estimates use one 8K sequence and include runtime workspace. “Multi-GPU” indicates the minimum memory-equivalent device count, not a guarantee that the model or runtime supports that topology efficiently.
RTX 4090 is positioned as local performance hardware with 24 GB of memory. It is a practical candidate when the full serving envelope stays below 20.4 GB per device and the runtime supports the selected precision. For latency or throughput planning, memory fit is only the first gate: benchmark the exact model, batch shape, framework, and provider topology.
Advertised capacity is not always fully available to model serving. Drivers, runtime kernels, CUDA or ROCm graphs, fragmentation, adapters, multimodal encoders, speculative decoding, and provider reservations can all move the real boundary. Treat the matrix as a shortlist, then validate on the target stack.
The sample rate on this page is a planning input, not a live offer. Recheck context, precision, concurrency, and GPU count before committing capacity.