Long-context memory. This profile uses an 85% planning ceiling, leaving approximately 21.2 GB outside the recommended workload envelope.
These estimates use one 8K sequence and include runtime workspace. “Multi-GPU” indicates the minimum memory-equivalent device count, not a guarantee that the model or runtime supports that topology efficiently.
NVIDIA H200 is positioned as long-context memory hardware with 141 GB of memory. It is a practical candidate when the full serving envelope stays below 119.8 GB per device and the runtime supports the selected precision. For latency or throughput planning, memory fit is only the first gate: benchmark the exact model, batch shape, framework, and provider topology.
Advertised capacity is not always fully available to model serving. Drivers, runtime kernels, CUDA or ROCm graphs, fragmentation, adapters, multimodal encoders, speculative decoding, and provider reservations can all move the real boundary. Treat the matrix as a shortlist, then validate on the target stack.
The sample rate on this page is a planning input, not a live offer. Recheck context, precision, concurrency, and GPU count before committing capacity.