What size LLM can I run with 24GB, 32GB or 48GB VRAM?

Short answer: 24GB is a practical single-GPU tier for many 32B 4-bit workloads; 32GB adds room for larger 32B quantizations, context and runtime allocations; 48GB can support still larger 32B paths and selected 70B 4-bit configurations. These are capacity boundaries, not speed rankings. A 44GB Q4_K_M 72B artifact is still tight in 48GB after cache and runtime allocations, and two 24GB cards do not automatically behave like one 48GB card.

24GB vs 32GB vs 48GB at a glance

Question	24GB GPU	32GB GPU	48GB GPU
Model-size starting point	Many 32B 4-bit workloads	Broader 32B quantization and context choices	32B headroom; selected 70B 4-bit paths
Official artifact boundary	Qwen2.5-32B Q4_K_M is 19.9GB	Qwen2.5-32B Q6_K is 26.9GB	Qwen2.5-72B Q4_K_M is 44GB and tight
Example hardware path	RTX 3090 or RTX 4090	RTX 5090	RTX A6000, RTX 6000 Ada or W7900
Memory model	One 24GB dedicated pool	One 32GB dedicated pool	One 48GB dedicated pool
Main risk	Larger 32B quantizations or context exceed the margin	70B artifacts generally require offload or splitting	A nominal 70B file fit can leave too little cache/runtime room

Exact Qwen3.5 boundaries across 24GB, 32GB and 48GB

Complete artifact	Files	24GB	32GB	48GB
Qwen3.5-27B community Q4_K_M plus F16 projector	17.668GB	6.332GB margin	14.332GB margin	30.332GB margin
Qwen3.5-27B community Q8_0 plus F16 projector	29.523GB	Does not fit	2.477GB; unsafe boundary	18.477GB margin
Qwen3.5-27B official BF16	55.563GB	Does not fit	Does not fit	Does not fit
Qwen3.5-35B-A3B official GPTQ-Int4	24.420GB	Does not fit	7.580GB margin	23.580GB margin
Qwen3.5-35B-A3B community Q4_K_M plus F16 projector	22.915GB	1.085GB; unsafe boundary	9.085GB margin	25.085GB margin
Qwen3.5-35B-A3B community Q8_0 plus F16 projector	37.802GB	Does not fit	Does not fit	10.198GB margin
Qwen3.5-35B-A3B official BF16	71.904GB	Does not fit	Does not fit	Does not fit

The table shows why the tiers are not interchangeable labels. Moving from 24GB to 32GB changes the listed 27B Q8 and 35B GPTQ file outcomes, but the 2.477GB Q8 remainder is still unsafe before runtime and hybrid-state allocations. Moving to 48GB changes the listed 35B Q8 outcome, leaving 10.198GB by file arithmetic, while neither listed BF16 checkpoint fits. Read the 27B source guide and 35B-A3B source guide for provenance and hybrid-attention limitations.

Translate capacity into a product decision: compare RTX 3090, 4090 and 5090 across 24GB and 32GB, then compare 32GB RTX 5090 with either 48GB RTX A6000 or 48GB RTX 6000 Ada.

What fits in 24GB?

A 24GB card can fully hold many small and mid-sized quantized language models, but the “parameter count × bits ÷ 8” weight estimate is only the starting point. The model file, KV cache, runtime buffers and concurrent requests all share the pool. Longer context can push a configuration that loaded successfully into an out-of-memory error.

For exactly 32 billion parameters at exactly 4 bits, raw weights equal 16GB. That leaves nominal room in a 24GB pool, but it does not prove a specific 32B artifact and context will fit. For exactly 70 billion parameters at 4 bits, raw weights alone equal 35GB—already beyond 24GB.

What changes at 32GB?

Qwen’s official Qwen2.5-32B-Instruct-GGUF example displays Q4_K_M at 19.9GB, Q5_K_M at 23.3GB, Q6_K at 26.9GB and Q8_0 at 34.8GB. A 32GB pool therefore gives the 4-/5-/6-bit examples progressively different working margins, while the displayed Q8_0 artifact already exceeds the label before cache and runtime allocations.

That makes 32GB a meaningful 32B headroom tier, not a universal bridge to 70B. Qwen’s official 72B example displays Q3_K_M at 35.5GB and Q4_K_M at 44GB, both larger than 32GB before cache. Read the 32B requirements guide for artifact and cache calculations.

What does 48GB unlock?

One 48GB pool can avoid the cross-device and CPU-offload compromises required when a workload exceeds 24GB. It can also preserve room for a larger quantization or context on workloads that technically squeeze into less memory. But 48GB is not a universal 70B guarantee.

For example, Qwen’s official Qwen2.5-72B GGUF repository lists Q4_K_M at approximately 44GB. A 48GB card has about 44.7GiB of nominal capacity. An illustrative one-sequence FP16 KV payload for that model is 2.5GiB at 8,192 token slots, before runtime buffers. That makes this exact combination a tight target rather than a safe blanket recommendation.

RTX 3090 versus RTX A6000: capacity is not speed

NVIDIA specifies the Ampere-generation GeForce RTX 3090 with 24GB GDDR6X. NVIDIA specifies the RTX A6000 with 48GB GDDR6 ECC, a 300W maximum power figure and professional workstation positioning.

The RTX A6000’s advantage in this comparison is its single 48GB memory pool and professional feature set—not a promise that it beats every 24GB card on tokens per second. Architecture, memory bandwidth, compute, precision, kernels, cooling and runtime optimization all affect measured performance.

One 48GB GPU versus two 24GB GPUs

One 48GB GPU exposes one pool to a compatible workload. Two 24GB GPUs remain separate devices. Software must deliberately partition layers or tensors across them, and some workloads or applications do not support that path. Transfers, PCIe topology, motherboard lanes, power and cooling can affect the result.

llama.cpp documents layer, row and experimental tensor split modes plus per-device split controls. That proves a supported partitioning mechanism exists; it does not prove that every frontend, model or split performs well on a particular dual-GPU system.

Do you need ECC?

ECC can detect and correct certain memory errors and is present on specified professional cards such as the RTX A6000. Whether that matters enough to justify the price depends on workload duration, reliability requirements, support expectations and the cost of an incorrect or interrupted run. ECC does not make a model more accurate and does not replace output validation.

Choose 24GB when

• Your exact tested model, context and concurrency fit with usable headroom.
• You prioritize consumer-card value and broad used-market availability.
• You can accept CPU offload or a smaller model for occasional larger jobs.
• Your case, PSU and cooling support the exact board.

Choose 32GB when

• Your 32B artifact fits in 24GB only narrowly and you need more context or runtime headroom.
• You want selected 32B 5-/6-bit paths on one consumer GPU.
• Your complete workload remains below 32GB after cache, buffers and other GPU users are counted.
• You do not need the 48GB tier’s additional capacity or professional-card features.

Choose 48GB when

• Your measured workload exceeds 24GB and benefits from staying on one CUDA GPU.
• You need a larger quantization, context or concurrent-request budget than 24GB permits.
• Your application benefits from the professional card’s documented driver, ECC or support features.
• The complete-system and used-card risks still make sense at the delivered price.

Before buying any tier

1. Record the exact model artifact, quantization, context, cache precision and concurrency.
2. Check your runtime’s GPU and architecture support.
3. Estimate memory, then test the longest representative request.
4. Compare complete-system cost, not only the card price.
5. For used hardware, verify photos, stress-test evidence, return terms and seller history with the used-GPU checklist.

Compare current hardware paths

Browse the RTX 3090 page for a tracked 24GB path and the hardware guide for 48GB professional-card coverage. If 48GB still is not enough, compare large unified-memory Mac Studio configurations with a 32GB RTX 5090. Current listings are saved observations, not guarantees of availability or condition.

Current 32B vision example: the Qwen3-VL 32B guide maps a complete 20.535GB official Q4 pair to constrained 24GB and practical 32GB paths.

Current vision family: the Qwen3-VL VRAM hub connects 2B, 4B, 8B and 32B complete multimodal artifacts to 24GB–48GB boundaries.

Current MoE vision example: the Qwen3-VL 30B-A3B guide maps complete official Q4 and Q8 projector paths across 24GB, 32GB and 48GB tiers.

Sources

• Official Qwen2.5-32B-Instruct-GGUF repository
• Official Qwen2.5-72B-Instruct-GGUF repository
• NVIDIA GeForce RTX 3090 specifications
• NVIDIA RTX A6000 specifications
• llama.cpp server and tensor-split documentation
• HighVRAM methodology

Reviewed October 25, 2026. Manufacturer specifications, official model repositories and technical documentation are linked. HighVRAM has not benchmarked every tier. Capacity does not establish speed, output quality or value. Some marketplace links on related pages are affiliate links.

Evaluating a specific 48GB card? Read the RTX A6000 48GB local-AI guide for specifications, used-card checks and the current screened offer.

Considering AMD for the 48GB tier? Read the Radeon PRO W7900 local-AI guide and verify the exact ROCm, OS and application combination before comparing price.

Comparing one 48GB card with two 24GB cards? Use the multi-GPU VRAM guide to account for software partitioning, communication, duplicated allocations and per-device headroom.

Concrete 32GB-versus-48GB decision: read RTX 5090 vs RTX A6000 for local AI for model-fit, power, ECC and full-system tradeoffs.

Trying to stay below the 48GB tier? Read how CPU offload changes VRAM, RAM and speed before treating a model that starts as a usable workflow.

Looking at a low-cost 24GB server accelerator? The Tesla P40 local-AI guide explains why passive cooling, CPU 8-pin power and the CUDA 13 Pascal cutoff can outweigh the memory label.

Evaluating a newer 48GB workstation card? Read the RTX 6000 Ada local-AI guide for model fit, ECC, power, A6000 differences and product-name traps.

Choosing a 48GB Ada card? Read the L40S vs RTX 6000 Ada comparison before treating a passive server accelerator and an active workstation card as interchangeable.

Would 80GB materially change the workload? Continue with the 48GB vs 80GB VRAM guide to compare capacity headroom with the server-platform cost of common 80GB accelerators.

\n

Comparing a current 32GB consumer card with a 48GB workstation card? Use the RTX 5090 vs RTX 6000 Ada local-AI comparison for capacity, bandwidth, ECC, power and software tradeoffs.

\n

Choosing a tier for a 32B model? The 32B GPU decision hub compares 24GB, 32GB and 48GB paths using official artifact sizes and cache headroom.

Choosing between 27B and 32B? Read the official-artifact comparison before treating parameter count or a 15.9GB file as a complete fit result.

Sizing Qwen3-30B-A3B or gpt-oss? The MoE VRAM guide explains why the active-parameter label does not replace total-weight, cache and runtime memory.

Choosing Qwen3-30B-A3B across these tiers? The exact-model guide maps Qwen’s official 18.6GB through 32.5GB files to 24GB, 32GB and 48GB planning.

Shopping specifically for a 27B model? The 27B GPU buyer guide compares 24GB, 32GB, 48GB and 64GB–80GB paths using exact Gemma 3 and Qwen3.5 artifacts.

One concrete 24GB case: the Gemma 3 27B VRAM guide explains why Google’s 17.2GB QAT Q4_0 file makes 24GB the first common one-device starting tier.

Concrete 24B reasoning example: the Magistral Small 1.2 VRAM guide maps Mistral’s official Q4/Q5/Q8/BF16 files to 16GB-through-64GB paths.

Agentic coding model: the Devstral Small 1.1 24B VRAM guide maps Mistral’s official 14.3GB–47.2GB GGUF files to these memory tiers and budgets the agent scaffold separately.

MoE workload: the Mixtral 8x7B VRAM guide maps labeled Q2–Q8 files across 24GB, 32GB, 48GB and 64GB tiers and explains why active parameters do not set weight memory.

Vision-model path: use the Llama 3.2 11B Vision VRAM guide for Meta’s official 21.34GB BF16 checkpoint, exact cache costs and multimodal component checks.

24B multimodal path: use the Mistral Small 3.2 VRAM guide for official BF16, community Q4–Q8 and exact 4K–128K cache costs.

24B Mistral decision: compare Small 3.2, Devstral and Magistral before treating their similar Q4/Q5/Q8 sizes as interchangeable deployments.

Current hybrid 27B example: Qwen3.5-27B maps a 17.668GB community Q4-plus-projector pair to practical 24GB hardware with explicit provenance.

24GB boundary case: Qwen3.5-35B-A3B’s official Int4 files exceed 24GB, while its community Q4-plus-projector path leaves little runtime headroom.

One direct 32GB-to-48GB hardware decision: compare RTX 5090 with NVIDIA A40 using exact Qwen3.5 margins, complete-system requirements and runtime headroom.

32GB versus 40GB/80GB decision: use the RTX 5090 vs A100 local-AI comparison for exact Qwen3.5 file margins and host-platform safeguards.

When 32GB is too small: compare RTX 5090 32GB with DGX Spark 128GB unified memory, including exact model-file boundaries and shared-memory caveats.

Direct 24GB-versus-128GB hardware path: RTX 4090 vs DGX Spark for local AI.

Direct used-24GB-versus-128GB hardware path: RTX 3090 vs DGX Spark.

24B workload path: compare 16GB through 64GB+ hardware for a 24B LLM using exact Q4, Q5, Q8 and BF16 file boundaries.

Staying in the 24GB tier? Compare the RTX 3090 and RTX 4090 as 24GB local-AI paths before paying for speed without adding model capacity.

Comparing a 16GB throughput tier with a 32GB capacity tier? Use RTX 5080 vs RTX 5090 for local AI to keep model fit separate from bandwidth and core count.

Choosing directly between 24GB and 32GB GeForce cards? Read the RTX 4090 vs RTX 5090 comparison for official platform specifications, exact Qwen2.5-32B file-fit boundaries and matched-test controls.

Choosing directly between used 24GB and current 32GB GeForce cards? Read the RTX 3090 vs RTX 5090 comparison for official platform specifications, exact Qwen2.5-32B file-fit boundaries, used-card safeguards and matched-test controls.

Choosing directly between used 24GB and current 16GB GeForce cards? Read the RTX 3090 vs RTX 5080 comparison for official specifications, exact 14B and 32B file-fit boundaries, used-card safeguards and matched-test controls.

Evaluating a consumer Radeon at the 24GB tier? Use the RX 7900 XTX 24GB fit guide to separate model-file capacity from runtime headroom and verify the exact ROCm, OS, driver and framework path.

Need to choose between two 24GB cards? The RX 7900 XTX versus RTX 3090 guide keeps the file-fit boundary constant while separating ROCm/CUDA compatibility, condition and performance evidence.

Choosing between two current 24GB cards? The RX 7900 XTX versus RTX 4090 guide keeps file fit constant while separating CUDA/ROCm support, board constraints, power and measured performance.

Comparing 24GB Radeon against 32GB Blackwell? Use the RX 7900 XTX vs RTX 5090 local-AI comparison for exact 32B file margins, ROCm/CUDA checks, power planning and matched benchmark controls.

Choosing between 24GB Radeon capacity and a 16GB Blackwell CUDA path? Use the RX 7900 XTX vs RTX 5080 local-AI comparison for exact 32B file margins, software checks, power planning and matched benchmark controls.

Choosing between 24GB Radeon capacity and a 16GB Ada CUDA path? Use the RX 7900 XTX vs RTX 4080 SUPER local-AI comparison for exact 32B file margins, software checks and matched benchmark controls.

Choosing between 24GB Radeon capacity and a 16GB Ada CUDA path? Use the RX 7900 XTX vs RTX 4070 Ti SUPER comparison for exact 32B margins, software checks, power and matched-test controls.

Considering Intel Arc Pro for local AI? Use the Best Intel GPU for Local AI guide to compare B50, B60, B65 and B70 across 16GB, 24GB and 32GB capacity, deployment and oneAPI/OpenVINO/SYCL support gates.

Evaluating Intel Arc Pro B60 24GB? Read the Arc Pro B60 local-AI guide for exact Qwen2.5-32B file margins, official board specifications, Intel software gates and a reproducible test checklist.

Evaluating Intel Arc Pro B70 32GB? Read the Arc Pro B70 local-AI guide for exact 32B file margins, official board specifications, current SYCL caveats and output-correctness controls.

Starting below 24GB? The Arc Pro B50 16GB guide shows exact 14B file margins and why runtime overhead must be calculated before moving to larger capacity tiers.

Exploring a 32GB Intel capacity tier? The Arc Pro B65 guide maps official 32B artifacts to the pool and explains why host-memory staging can still affect loading.

Same 32GB capacity, different Intel hardware: compare Arc Pro B65 and B70 without confusing shared file fit with performance.

Crossing from 16GB to 24GB on Intel? The Arc Pro B50 versus B60 guide shows which official 32B files change from impossible or unsafe to plausible.

Intel 24GB-versus-32GB decision: the Arc Pro B60 versus B65 guide maps exact 32B file margins without inventing a speed ranking.

Two 24GB cards can still be different bets. The Arc Pro B60 versus RTX 3090 comparison holds capacity constant and isolates software, power and used-card risk.

Same-capacity cross-platform comparison: Compare Intel Arc Pro B70 32GB with GeForce RTX 5090 32GB by software support, power, system requirements and matched workload evidence.

Same-capacity cross-platform comparison: Compare Intel Arc Pro B60 24GB with GeForce RTX 4090 24GB by software support, power, system requirements and matched evidence.