AMD Instinct MI325X 256GB for Local AI: What Fits?

Short answer: AMD Instinct MI325X provides 256GB HBM3E on one CDNA 3 accelerator. That is enough for every listed Qwen3.5-122B artifact by file size, although the 250.173GB BF16 checkpoint leaves only 5.827GB of nominal decimal margin and is not a practical one-device conclusion. For Qwen3.5-397B, the 235.708GB GPTQ and 245.012GB community Q4/projector pair fit by files but remain tight boundaries; FP8, Q8 and BF16 require multiple accelerators.

MI325X specifications that matter for local AI

Official specification	AMD Instinct MI325X	Planning meaning
Architecture	AMD CDNA 3; gfx942	Requires exact ROCm, framework and kernel support
GPU memory	256GB HBM3E with full-chip ECC	Dedicated accelerator memory; still reserve runtime headroom
Peak memory bandwidth	6TB/s maximum peak theoretical	Not a universal prompt or generation rate
Compute resources	304 compute units; 1,216 matrix cores	Peak hardware counts do not establish application throughput
Host interface	PCIe 5.0 x16	Does not make the OAM module a desktop PCIe add-in card
Form factor and cooling	Passive OAM module	Needs qualified chassis airflow or platform cooling
Maximum TBP	1000W	Accelerator limit, not complete-server wall power

Qwen3.5-122B on one MI325X

Complete artifact	Files	Nominal decimal margin in 256GB	Fit interpretation
Official GPTQ-Int4	78.864GB	177.136GB	Fits by files with substantial capacity margin
Community Q4_K_M + F16 projector	77.446GB	178.554GB	Fits by files; verify projector, modality and peak allocation
Official FP8	127.163GB	128.837GB	Fits by files with substantial capacity margin
Community Q8_0 + F16 projector	130.781GB	125.219GB	Fits by files with substantial capacity margin
Official BF16	250.173GB	5.827GB	File-only boundary; insufficient evidence for practical runtime fit

Qwen3.5-397B: two 4-bit boundaries and multi-device paths

Complete artifact	Files	One MI325X	Minimum nominal aggregate
Official GPTQ-Int4	235.708GB	Fits by files; 20.292GB nominal margin	1×256GB, but measured peak decides practicality
Community Q4_K_M + F16 projector	245.012GB	Very tight file boundary; 10.988GB nominal margin	1×256GB file-only boundary
Official FP8	406.152GB	Does not fit	2×256GB = 512GB
Community Q8_0 + F16 projector	422.426GB	Does not fit	2×256GB = 512GB
Official BF16	806.796GB	Does not fit	4×256GB = 1,024GB

MI325X versus MI300X: capacity changes more than the label

Specification	MI300X	MI325X
Architecture	CDNA 3 / gfx942	CDNA 3 / gfx942
Memory	192GB HBM3	256GB HBM3E
Peak bandwidth	5.3TB/s	6TB/s
Maximum TBP	750W	1000W
Eight-accelerator platform	1.5TB aggregate HBM3	2.048TB aggregate HBM3E
Qwen3.5-397B 4-bit	Does not fit on one accelerator	Fits by files at a tight one-device boundary

The eight-accelerator platform is not one 2TB GPU

AMD’s MI325X platform combines eight OAM accelerators for 2.048TB aggregate HBM3E. The devices are connected through Infinity Fabric, but software must still distribute weights, cache and work. Tensor, pipeline, expert or layer placement determines local memory pressure and communication. Report the split and peak memory on every device rather than quoting only the aggregate total.

ROCm compatibility and reproducibility checks

1. Choose the exact ROCm-supported operating system, container and driver combination.
2. Verify the runtime build includes gfx942 kernels and the intended precision/quantization path.
3. Confirm the exact Qwen3.5 revision, MoE routing, hybrid attention, MTP and multimodal components.
4. Fix context, input modality, output length, batch size and concurrency.
5. For multiple accelerators, record topology, placement and CPU/system-memory offload.
6. Measure peak device and system memory, time to first token, prompt processing, generation, errors, throttling and wall power.

Use the benchmark builder to capture topology, placement, offload, modality and separate peak device/system memory. A model launching once is not a sufficient deployment result.

Purchase and deployment safeguards

• Require the exact MI325X label, part identity and health evidence.
• Confirm whether the quote is for a bare OAM module, Universal Baseboard or complete tested server.
• Validate chassis, airflow or liquid-cooling design, 54V delivery, circuit capacity and management firmware.
• Check the platform’s warranty, support responsibility, return terms and replacement path.
• Compare complete acquisition or hosting cost, not accelerator-only pricing.

Continue with the MI300X guide, MI300X vs H200 comparison, Qwen3.5-122B guide, Qwen3.5-397B guide, 96GB-to-1TB hardware paths, multi-GPU guide, workload test or private watchlist/RSS page.

Primary AMD sources

• AMD Instinct MI325X specifications
• AMD MI325X datasheet
• AMD MI325X platform specifications
• AMD MI325X customer acceptance guide
• AMD ROCm Instinct workload optimization guide
• HighVRAM methodology

Reviewed October 25, 2026. Specifications are AMD-published. HighVRAM has not independently benchmarked every platform, runtime or workload. File fit and nominal margin do not establish compatibility, speed, quality, reliability or value.

Comparing across vendors? The MI325X 256GB versus NVIDIA B200 180GB guide shows which Qwen3.5 boundaries change, and separates ROCm/gfx942 capacity from CUDA/Blackwell software and platform requirements.

For the next 288GB CDNA 4 tier, compare MI350X and MI355X model fit, power and cooling.

The newer 288GB-class cross-vendor decision is covered in NVIDIA B300 vs AMD MI355X.

Need the full AMD capacity ladder? Compare MI300X, MI325X, MI350X and MI355X with exact large-model fit and system tradeoffs.

Choosing an AMD path across consumer, workstation and data-center hardware? Use the Best AMD GPU for Local AI guide to compare 24GB through 288GB capacity, physical deployment and ROCm validation gates.