Mixtral 8x7B VRAM Requirements: 24GB, 32GB, 48GB or 64GB?

Short answer: Mixtral 8x7B is a sparse mixture-of-experts model, but its stored weights are not the size of the two experts used for each token. Mistral’s official BF16 checkpoint contains 93.4GB of safetensor shards. A widely used community GGUF repository displays 15.6GB Q2_K, 20.4GB Q3_K_M, 26.4GB Q4_K_M, 32.2GB Q5_K_M, 38.4GB Q6_K and 49.6GB Q8_0. In practice, 24GB is a low-bit path, 32GB is a constrained Q4 path, 48GB is the Q5/Q6 tier, and 64GB is the Q8 tier. Long FP16/BF16 KV caches can move every boundary upward.

Mixtral 8x7B file sizes

Artifact	Displayed size	Memory implication
Q2_K GGUF	15.6GB	16GB is a file-size trap; target 24GB for working room
Q3_K_M GGUF	20.4GB	24GB is constrained, especially as context grows
Q4_K_M GGUF	26.4GB	Does not fit wholly on 24GB; 32GB is a constrained starting point
Q5_K_M GGUF	32.2GB	Does not fit wholly on 32GB; target 48GB+
Q6_K GGUF	38.4GB	48GB tier, subject to context and runtime margin
Q8_0 GGUF	49.6GB	Does not fit wholly on 48GB; target 64GB+
Official BF16 checkpoint	93.4GB across 19 shards	96GB is narrow before cache/runtime state; 128GB is a safer planning tier

24GB, 32GB, 48GB or 64GB?

• 24GB: Q2_K has useful nominal room; Q3_K_M leaves only about 3.6GB before cache and runtime allocations. Test quality as well as fit.
• 32GB: Q4_K_M leaves about 5.6GB on the decimal labels, but an 8K FP16/BF16 cache payload alone is 8GiB under the official configuration. Reduce context, quantize/offload cache where supported, or move up a tier.
• 48GB: Q5_K_M and Q6_K become plausible, with very different remaining margins. Q8_0 already exceeds the label before cache.
• 64GB: the first listed tier above the 49.6GB Q8_0 file, leaving room that still must be tested against context and runtime allocations.
• 96GB/128GB: the official 93.4GB BF16 checkpoint is too close to a nominal 96GB pool for a blanket recommendation. A long full-precision cache makes the gap much larger.

Compare the 24GB/32GB/48GB tiers, the 48GB-through-96GB tiers, and calculate your own workload with the VRAM calculator.

Why “2 active experts” does not mean 2 × 7B weight memory

The official configuration declares 8 local experts and routes each token to 2 experts. Routing sparsity reduces the expert computation performed for that token. It does not discard the other experts’ parameters from the checkpoint. The official 19-shard total is direct evidence: the BF16 files occupy about 93.4GB.

The MoE VRAM guide explains total versus active parameters and why memory planning begins with the artifact rather than the active count.

KV cache is unusually important here

Mistral’s official configuration records 32 layers, 8 key/value heads, 32 attention heads, a 4,096 hidden size and 32,768 maximum positions. The head dimension is therefore 4,096 ÷ 32 = 128. For one sequence with a two-byte FP16/BF16 cache and every layer on the accelerator:

bytes per token = 2 × 32 layers × 8 KV heads × 128 head dimension × 2 bytes
                = 1,048,576 bytes
                = 1 MiB per token

Allocated token slots	Illustrative FP16/BF16 KV payload
4,096	4GiB
8,192	8GiB
16,384	16GiB
32,768	32GiB

This is why a Q4 file can exceed a 32GB workload at longer context even though the file itself is 26.4GB. Maximum context is not a promise that every hardware tier can allocate it.

Official evidence versus community conversion

Mistral’s repository is authoritative for the model identity, BF16 files and configuration. The listed GGUF sizes come from a third-party conversion of that model. Record the converter, filename and checksum; do not imply that Mistral published or validated those quantizations. Use the quantization guide to compare formats without treating a format name as a universal quality score.

When the model exceeds one GPU

CPU offload and multi-GPU placement can make a larger artifact load, but neither is free. Offload shifts part of the workload into system RAM and across the host-device link. Multiple GPUs retain separate memory pools and require explicit model placement. Review CPU offload and multi-GPU VRAM placement before buying around either assumption.

A reproducible Mixtral fit test

1. Record the exact official revision or community GGUF repository, filename, size and checksum.
2. Record runtime/version, device placement and offloaded layers.
3. Record GPU or unified-memory capacity, available system RAM, OS and driver.
4. Record allocated context slots, actual prompt length, output target, concurrency and cache precision.
5. Measure available memory before load, after load and at peak generation.
6. Capture time to first token, throughput, host-device traffic and any fallback or out-of-memory error.
7. Evaluate task quality at the selected quantization; loading successfully is not proof of useful output.

Run the same repeatable workload and share a reproducible result.

Sources and limits

• Mistral official model card
• Mistral official checkpoint files
• Mistral official configuration
• TheBloke community GGUF conversion
• HighVRAM methodology

Reviewed October 25, 2026. Official shard sizes and configuration were read from Mistral’s repository; GGUF sizes are third-party artifacts and labeled accordingly. HighVRAM has not independently benchmarked every file or runtime. File fit does not establish speed or output quality.