Budget inference. This profile uses an 85% planning ceiling, leaving approximately 2.4 GB outside the recommended workload envelope.
These estimates use one 8K sequence and include runtime workspace. “Multi-GPU” indicates the minimum memory-equivalent device count, not a guarantee that the model or runtime supports that topology efficiently.
NVIDIA T4 is positioned as budget inference hardware with 16 GB of memory. It is a practical candidate when the full serving envelope stays below 13.6 GB per device and the runtime supports the selected precision. For latency or throughput planning, memory fit is only the first gate: benchmark the exact model, batch shape, framework, and provider topology.
Advertised capacity is not always fully available to model serving. Drivers, runtime kernels, CUDA or ROCm graphs, fragmentation, adapters, multimodal encoders, speculative decoding, and provider reservations can all move the real boundary. Treat the matrix as a shortlist, then validate on the target stack.
The sample rate on this page is a planning input, not a live offer. Recheck context, precision, concurrency, and GPU count before committing capacity.