Short answer: there is no universal best GPU for a 24B LLM. For the official Mistral 24B examples used here, 16GB is a constrained Q4 target, 24GB is the practical Q4/Q5 tier, 32GB is the practical Q8 tier, and 48GB is still too tight for a roughly 47–48GB BF16 file as a complete workload. Plan 64GB+ for practical BF16 working room. Choose the least-expensive complete system that supports the exact runtime, model and task with measured headroom.
Fast decision
- Already own 16GB: test Q4 at the real context before upgrading; a 14.3GB file leaves little room.
- Buying for Q4/Q5: start with one 24GB CUDA GPU or a supported 24GB+ unified-memory system.
- Want Q8: target 32GB+; the cited 25.1GB files exceed one 24GB device before runtime memory.
- Want BF16: skip 48GB as a comfortable target; the cited 47.2–48.023GB files leave essentially no complete-workload margin.
Evidence from three 24B model paths
| Artifact tier | Displayed size | Planning implication |
|---|---|---|
| Q4_K_M | 14.3GB | 16GB leaves about 1.7GB before cache, buffers, kernels and display use; 24GB is the practical tier |
| Q5_K_M | 16.8GB | Does not fit wholly on 16GB; 24GB leaves about 7.2GB before other allocations |
| Q8_0 | 25.1GB | Does not fit wholly on 24GB; 32GB leaves about 6.9GB before other allocations |
| BF16 | 47.2GB–48.023GB | 48GB is a file-size boundary, not a practical complete-workload tier; start planning at 64GB+ |
Hardware paths by memory tier
| Path | Why consider it | Main constraint |
|---|---|---|
| 16GB RTX path | Use hardware you already own for a tightly tested Q4 workload | 14.3GB leaves little room; display use, context and runtime allocations can break fit |
| RTX 3090 24GB | Used CUDA value when condition, power and host requirements are acceptable | Used-card risk; 350W reference board power; no practical Q8 fit for the cited 25.1GB file |
| RTX 4090 24GB | Newer 24GB CUDA path when the complete Q4/Q5 workload fits | Same 24GB capacity ceiling as RTX 3090; higher price does not make Q8 fit |
| RTX 5090 32GB | One desktop CUDA device above the cited Q8 file boundary | Only about 6.9GB file-only margin before cache and runtime state; 575W reference graphics power |
| 48GB workstation/server paths | More context, concurrency or Q8 room on one device | The cited BF16 files still sit at the capacity boundary; active and passive cards have different integration needs |
| 64GB+ dedicated or unified-memory paths | Practical starting class for the cited BF16 files | Software support, memory model, host, cooling and total cost vary widely |
A 24B label does not define one workload
Mistral Small 3.2, Devstral Small 1.1 and Magistral Small 1.2 share similar artifact-size tiers but target different work. Small 3.2 is a general multimodal model on its official path, Devstral targets agentic coding, and Magistral targets reasoning with specific template guidance. Vision projectors, agent containers and tools, reasoning formatting, context behavior and host processes change the complete workload even when the model file is similar.
Context and concurrency can reverse the answer
The file is only the starting allocation. KV cache grows with context and concurrent sequences; runtimes need workspaces and kernels; desktop GPUs may also drive a display. Mistral Small 3.2’s official geometry produces 320KiB per token for an FP16/BF16 text cache: about 2.5GiB at 8K, 10GiB at 32K and 40GiB at 128K for one sequence before vision and runtime state. A card that loads Q4 may still fail the intended long-context or multi-user workload.
Do not combine two cards on paper
Two 24GB GPUs provide 48GB aggregate device-local capacity, not one automatic 48GB allocation. A supported runtime must split the model and other state across devices, some allocations may be duplicated, interconnect and placement affect performance, and each device’s peak must remain below its own limit. Use the multi-GPU guide before treating aggregate labels as a fit result.
Choose with a reproducible test
- Record the exact repository, revision, filename, displayed size and checksum.
- Record runtime, version, templates/tokenization and architecture support.
- Fix context, output length, concurrency, cache precision and device placement.
- Include vision inputs, agent tools or reasoning-format requirements when applicable.
- Measure peak accelerator memory, peak system RAM, time to first token and generation rate.
- Evaluate task quality and errors at the chosen quantization.
- Compare the complete system: host, power, cooling, storage, warranty and delivered cost.
Continue the decision
- Estimate a planning range · define the workload
- Compare hardware paths · inspect screened offers
- Check benchmark evidence · build a reproducible record
- Stage a private market watchlist
Sources and limits
- Mistral official Devstral Small 1.1 GGUF files
- Mistral official Magistral Small 1.2 GGUF files
- Mistral official Small 3.2 repository
- HighVRAM methodology
Reviewed October 25, 2026. HighVRAM has not independently benchmarked every listed hardware path with every 24B model and runtime. File fit does not establish compatibility, speed, task quality or reliability. Marketplace prices and availability change.