Large HBM capacity. This profile uses an 85% planning ceiling, leaving approximately 28.8 GB outside the recommended workload envelope.
These estimates use one 8K sequence and include runtime workspace. “Multi-GPU” indicates the minimum memory-equivalent device count, not a guarantee that the model or runtime supports that topology efficiently.
AMD MI300X is positioned as large hbm capacity hardware with 192 GB of memory. It is a practical candidate when the full serving envelope stays below 163.2 GB per device and the runtime supports the selected precision. For latency or throughput planning, memory fit is only the first gate: benchmark the exact model, batch shape, framework, and provider topology.
Advertised capacity is not always fully available to model serving. Drivers, runtime kernels, CUDA or ROCm graphs, fragmentation, adapters, multimodal encoders, speculative decoding, and provider reservations can all move the real boundary. Treat the matrix as a shortlist, then validate on the target stack.
The sample rate on this page is a planning input, not a live offer. Recheck context, precision, concurrency, and GPU count before committing capacity.