Short answer: AMD Instinct MI300X is a 192GB HBM3 datacenter accelerator, not a desktop graphics card. Its capacity can hold the complete Qwen3.5-122B-A10B official FP8 checkpoint on one accelerator with substantially more file headroom than a 141GB device. Qwen3.5-397B-A17B still requires multiple MI300X accelerators: two for practical 4-bit aggregate capacity, three for FP8/Q8, and at least six for the 1TB-class BF16 planning target.
Capacity is not compatibility or speed. Confirm the exact ROCm, runtime, model architecture, quantization, multimodal kernels and distributed-placement path. An aggregate of several 192GB devices is not one automatically pooled memory space.
MI300X specifications that matter for local AI
| Official specification | MI300X | Planning meaning |
|---|---|---|
| Architecture / target | AMD CDNA 3 / gfx942 | Use the matching ROCm software and container matrix |
| Dedicated memory | 192GB HBM3 with full-chip ECC | Weights, cache, runtime buffers and concurrency share one local pool |
| Peak memory bandwidth | 5.3TB/s peak theoretical | Published ceiling, not an application-independent token rate |
| Compute units | 304 | Does not replace measured model/runtime performance |
| Form factor | OAM module; passive cooling | Requires a qualified server/platform, not a normal PCIe workstation slot |
| Host interface | PCIe 5.0 x16 | Host and scale-out path still affect offload and distributed execution |
| Scale-up links | Infinity Fabric | Record topology and placement for every multi-accelerator result |
| Maximum TBP | 750W | Platform power and cooling are system-level requirements |
What fits in one 192GB MI300X?
| Qwen3.5 path | Complete files | One MI300X? | Boundary |
|---|---|---|---|
| 122B-A10B official GPTQ-Int4 | 78.864GB | Yes, by file size | Leaves capacity for runtime/workload allocations; measure the actual path |
| 122B-A10B community Q8_0 + F16 projector | 130.781GB | Yes, by file size | Separately labeled community conversion; exact runtime support required |
| 122B-A10B official FP8 | 127.163GB | Yes, by file size | 192GB is the practical single-device capacity class |
| 122B-A10B official BF16 | 250.173GB | No | Requires multiple devices or explicit offload; 320GB is the practical planning class |
| 397B-A17B official GPTQ-Int4 | 235.708GB | No | Two MI300X provide 384GB aggregate capacity |
| 397B-A17B official FP8 | 406.152GB | No | Three MI300X provide 576GB aggregate capacity |
| 397B-A17B official BF16 | 806.796GB | No | Five provide 960GB aggregate but remain below the 1TB planning class; six provide 1,152GB |
Eight-MI300X platform context
AMD’s MI300X platform documentation describes eight 192GB OAM accelerators with 1.5TB aggregate HBM3 and fully meshed Infinity Fabric connectivity. That capacity clears the Qwen3.5-397B official BF16 file total and 1TB planning tier, but the deployment still needs a supported runtime, expert/tensor/pipeline placement, per-device headroom, host resources and a representative benchmark.
- Record every accelerator and its assigned layers or experts.
- Measure memory on each accelerator, not only a summed number.
- Report interconnect traffic, host RAM, storage and any CPU/offload path.
- Fix context, visual workload, batch size, concurrency and output length.
- Capture time to first token, prompt processing, generation rate, fallback and errors separately.
ROCm and runtime checks
AMD’s current ROCm compatibility matrix identifies the MI300 series as CDNA 3/gfx942 and ties support to specific operating-system, kernel, firmware, driver and user-space combinations. AMD’s LLM Extension matrix separately lists tested MI300X combinations for components including llama.cpp, FlashInfer, Triton Inference Server, Ray and other stacks. Treat those as versioned compatibility records—not a promise that every Qwen3.5 quantization or multimodal kernel works in every container.
- Choose the exact ROCm release and supported operating system for the platform.
- Select a runtime/container combination that explicitly lists MI300X support.
- Verify Qwen3.5 MoE, hybrid attention, MTP, quantization and vision support at the exact version.
- Run platform health, memory and link checks before model benchmarking.
- Preserve container digest, driver, firmware, runtime, model revision and command line in the result.
MI300X versus 96GB and 141GB paths
Compared with a 96GB RTX PRO workstation card, MI300X doubles nominal local capacity but moves into a qualified OAM server platform and ROCm software path. Compared with 141GB H200, MI300X adds 51GB of local capacity: that distinction matters for the 127.163GB Qwen3.5-122B FP8 checkpoint because 141GB is a tight file-fit boundary while 192GB provides more workload headroom. Capacity alone does not establish which platform is faster, easier to deploy or better value.
Continue with the Qwen3.5-122B exact guide, Qwen3.5-397B exact guide, 96GB-to-1TB hardware comparison, H200 guide, workload test, benchmark submission, private watchlist/RSS page or all guides and tools.
AMD sources and limits
- AMD Instinct MI300X product specifications
- AMD MI300X data sheet
- AMD MI300X system acceptance guide
- AMD ROCm compatibility matrix
- AMD LLM Extension compatibility matrix
- HighVRAM methodology
Reviewed October 25, 2026. Specifications and compatibility claims are manufacturer-published and version-specific. HighVRAM has not independently benchmarked MI300X with every model, quantization, runtime, container, topology or multimodal workload. File fit does not establish compatibility, speed, quality, reliability or value.
512GB unified-memory comparison: the M5 Ultra Mac Studio guide contrasts shared-memory fit with dedicated 192GB HBM3 without inferring performance.
\n\nChoosing between 192GB AMD and 141GB NVIDIA HBM? Read AMD MI300X vs NVIDIA H200 for local AI for exact Qwen3.5 fit, ROCm/CUDA, topology and complete-server requirements.
\n\nNeed the newer 256GB CDNA 3 tier? Read the MI325X local-AI guide for exact Qwen3.5 fit, 6TB/s HBM3E specifications, 1000W platform requirements and MI300X differences.
Need more single-accelerator headroom? See how the 288GB MI350X and MI355X tier changes these model-fit boundaries.
Need the full AMD capacity ladder? Compare MI300X, MI325X, MI350X and MI355X across 192GB through 288GB, ROCm generations and platform requirements.
Choosing an AMD path across consumer, workstation and data-center hardware? Use the Best AMD GPU for Local AI guide to compare 24GB through 288GB capacity, physical deployment and ROCm validation gates.