What size LLM can I run with 8GB, 12GB or 16GB VRAM?

A specific 12GB example: the RTX 4070 SUPER local-AI guide maps official 7B and 14B files to the card’s capacity.

A specific 16GB path: the RTX 4070 Ti SUPER local-AI guide maps official model files to the card’s capacity.

One concrete 16GB example: the RTX 4080 SUPER local-AI guide maps official model files to the card’s 16GB capacity.

Short answer: with 8GB VRAM, investigate 4B models broadly and selected 7B/8B 4-/5-bit workloads; with 12GB, 7B and 8B become more practical and selected low-bit 14B configurations enter the range; with 16GB, larger 7B/8B quantizations and many 14B paths become practical. A typical 32B 4-bit artifact can still exceed 16GB. More VRAM does not automatically make inference faster. Choose the smallest tier that fits the exact model artifact, KV cache, runtime allocations and safety margin while supporting your software.

8GB vs 12GB vs 16GB at a glance

Dedicated VRAM	Reasonable starting use	Main constraint
8GB	Many 7B-class 4-/5-bit artifacts, moderate context, single interactive sequence	Larger quantizations, long context or other GPU allocations can exhaust the remaining margin
12GB	7B-class models with more headroom; selected low-bit 14B configurations	Many 14B 4-bit artifacts consume most of the pool before cache and buffers
16GB	Larger 7B quantizations; more practical 14B 5-/6-bit paths; greater context flexibility	It is still too small for many 32B artifacts without aggressive quantization or offload

A real 7B artifact example

Qwen’s official Qwen2.5-7B-Instruct-GGUF repository displays a 4.68GB Q4_K_M artifact, 5.44GB Q5_K_M, 6.25GB Q6_K and approximately 8.10GB Q8_0 after summing split files. That makes the practical difference visible:

• 8GB: Q4_K_M and Q5_K_M leave some nominal room; Q6_K needs careful total-memory validation; the displayed Q8_0 artifact is already larger than the memory label.
• 12GB: all four artifact sizes are below the label, leaving progressively different margins for cache and runtime allocations.
• 16GB: gives the Q8_0 artifact substantially more working room and can make longer context or additional allocations easier to accommodate.

Download size is not peak resident memory. Read the complete 7B VRAM requirements guide for the official file table and explicit KV-cache arithmetic.

What size 8B LLM can each tier run?

Qwen’s official Qwen3-8B-GGUF example displays a 5.03GB Q4_K_M artifact and an 8.71GB Q8_0 artifact. Those files are distinct from the 7B example and use a different published cache geometry.

• 8GB: the 5.03GB Q4_K_M example leaves nominal room for cache and runtime allocations, but the 8.71GB Q8_0 file already exceeds the label.
• 12GB: both displayed artifact examples are below the label, with materially different remaining margins.
• 16GB: adds context, concurrency and runtime headroom; unused capacity still does not guarantee higher speed.

Read the sourced 8B VRAM requirements guide for the full official artifact table and explicit cache arithmetic.

What changes for a 14B model?

The official Qwen2.5-14B-Instruct-GGUF example displays Q2_K at 5.77GB, Q3_K_M at 7.34GB, Q4_K_M at 8.99GB, Q5_K_M at 10.5GB, Q6_K at 12.1GB and Q8_0 at 15.7GB. Those are artifact sizes before cache and runtime overhead.

• 8GB: a constrained low-bit target; even the 7.34GB Q3_K_M example leaves little nominal margin.
• 12GB: can be a practical Q4_K_M target, while 10.5GB Q5_K_M leaves less working room.
• 16GB: opens more 5-/6-bit paths, but the 15.7GB Q8_0 artifact does not leave safe runtime headroom.

See the sourced 14B VRAM requirements guide before choosing hardware around a 14B or 15B-class workload.

Reference NVIDIA products at each tier

Capacity labels appear across different generations and product variants. They identify memory size—not a local-AI speed ranking.

Reference product	Official memory configuration	Identity check
GeForce RTX 3060	12GB or 8GB GDDR6	Confirm the exact 12GB versus 8GB variant; the official page lists both
GeForce RTX 4060 Ti	16GB or 8GB GDDR6	The same product name spans two materially different capacity tiers
GeForce RTX 5060 Ti	16GB or 8GB GDDR7	Verify capacity in the listing and on the physical product; do not infer it from “5060 Ti” alone

Why 16GB is not automatically better than 12GB

If the workload uses more than 12GB, the 16GB device can succeed where the 12GB device cannot. If both fit comfortably, measured speed depends on architecture, memory bandwidth, compute, kernels, clocks, power limits and the software stack. Capacity is a threshold property; unused VRAM is not a performance multiplier.

This is especially important when comparing an older 12GB card with a newer 16GB card. The additional capacity and newer architecture may both matter, but they answer different questions. Keep fit, speed, power, purchase condition and total system cost as separate columns.

Context and concurrency use the margin

KV cache grows with token slots, model geometry, cache precision and concurrent sequences. Other GPU users—desktop displays, browsers, image tools or a second loaded model—also reduce available memory. A model that loads at short context can still fail on a long prompt or concurrent request.

Do not plan to the label’s last few hundred megabytes. Check observed memory use with a representative long request, and repeat the test after enabling the exact context, batch and concurrency settings you intend to keep.

When offload changes the choice

CPU offload can place some model layers in system RAM and allow a workload to run with less VRAM, but transfers and CPU work can change latency and throughput. System RAM does not become VRAM. If offload is part of the plan, benchmark the intended layer split rather than treating successful loading as proof of usability. Read the CPU-offload guide.

RTX 3060 12GB vs 4060 Ti 16GB vs 5060 Ti 16GB
Compare mainstream NVIDIA paths by exact capacity, architecture, CUDA generation, power and verified software fit—not gaming charts or peak metrics.

A buying checklist

1. Choose the exact model repository and quantized artifact.
2. Record its displayed size, context, cache precision and concurrency.
3. Add runtime buffers, other GPU users and safety headroom.
4. Verify the exact 8GB, 12GB or 16GB board variant—not only the product family.
5. Confirm OS, driver, CUDA/ROCm, framework, runtime and compiled-extension support.
6. Compare measured prompt processing, generation, power and noise using the same workload.
7. For used hardware, inspect identity, condition, seller evidence and return terms.

RTX 4060 Ti 16GB vs RTX 5060 Ti 16GB
Hold capacity constant while comparing Ada and Blackwell software support, power, partner-card design and measured workload behavior.

RTX 5070 Ti 16GB for local AI
Evaluate the higher-power 16GB Blackwell path through exact model fit, capability 12.0 software support, partner-card integration and measured workload evidence.

RTX 5060 Ti 16GB vs RTX 5070 Ti 16GB
Hold capacity and Blackwell software generation constant while comparing memory subsystem, measured workload behavior, 180W-versus-300W reference power and complete offer value.

RTX 5080 16GB for local AI
Evaluate the high-throughput 16GB Blackwell path through exact model fit, capability 12.0 software support, 360W reference power, physical integration and larger-memory alternatives.

RTX 5070 Ti 16GB vs RTX 5080 16GB
Hold capacity and Blackwell software generation constant while comparing measured workload behavior, 896-versus-960GB/s published bandwidth, 300W-versus-360W reference power and complete offer value.

Best 16GB GPU for local AI? Four RTX paths compared
Choose among RTX 4060 Ti, 5060 Ti, 5070 Ti and 5080 with one capacity-first framework covering software support, measured workload behavior, power, partner-card requirements and total cost.

Current vision family: the Qwen3-VL 2B/4B/8B/32B guide shows complete official Q4/Q8 projector paths from compact through workstation tiers.

Compact vision example: the Qwen3-VL 4B guide maps complete official Q4/Q8 projector pairs to constrained 4GB, practical 6GB/8GB and BF16 12GB/16GB tiers.

Sources

• NVIDIA GeForce RTX 3060 family specifications
• NVIDIA GeForce RTX 4060 Ti and 4060 specifications
• NVIDIA GeForce RTX 5060 family specifications
• Official Qwen2.5-7B-Instruct-GGUF repository
• Official Qwen2.5-14B-Instruct-GGUF repository
• Official Qwen3-8B-GGUF repository
• HighVRAM methodology

Reviewed October 25, 2026. Specifications and repository display sizes are linked to primary sources. HighVRAM has not independently benchmarked every listed card, model or tier. Capacity does not establish speed, output quality or value.

Targeting the classic 13B class? Compare 13B and 14B VRAM requirements before assuming the smaller parameter label has a smaller cache.

RTX 5060 Ti 16GB for local AI
Check 7B, 14B and constrained 32B model fit, Blackwell/CUDA capability 12.0 software support, 16GB-versus-8GB identity and complete-system requirements.

RTX 4060 Ti 16GB for local AI
Evaluate 7B, 14B and constrained 32B model fit, Ada/CUDA capability 8.9 support, exact 16GB-versus-8GB identity and partner-card requirements.

RTX 3060 12GB for local AI
Check 7B, constrained 14B and offload-only 32B paths, exact 12GB-versus-8GB identity, Ampere support and used-card evidence.

\n\n

Best GPU for a 14B LLM?
Turn the 12GB-versus-16GB capacity decision into a complete buying workflow, including the point where 24GB becomes useful.

\n\n

Choosing a GPU specifically for a 7B or 8B LLM?
Turn the 8GB/12GB/16GB capacity comparison into a reproducible buying workflow for the exact artifact and context.

Gemma example: the Gemma 3 family VRAM guide shows why the official 12B QAT file starts at 12GB while 16GB provides more working margin.

Concrete 12GB case: the Gemma 3 12B VRAM guide explains why Google’s 8.07GB QAT model does not fit wholly on 8GB and why 12GB is the first common starting tier.

Compact multimodal example: the Gemma 3 4B VRAM guide explains why 4GB is constrained and 6GB/8GB provide more practical starting points.

Concrete 14B case: the Phi-4 VRAM guide shows which Microsoft GGUF artifacts can target 12GB or 16GB and why Q8_0 points toward 24GB.

Vision-model example: see the Qwen2.5-VL 7B VRAM guide for why a 4.683GB Q4 language file becomes a 5.54–6.04GB complete file path before cache and runtime memory.

Qwen vision family: the Qwen2.5-VL 3B-to-72B guide connects mainstream capacity tiers to complete multimodal checkpoints and cache geometry.

Compact vision example: the Qwen2.5-VL 3B guide maps complete Q4/Q8 projector pairs to constrained 4GB, practical 6GB/8GB and BF16 12GB.

Current vision example: the Qwen3-VL 8B guide maps complete official Q4/Q8 projector pairs to constrained 8GB, practical 12GB/16GB and BF16 24GB/32GB.

Current hybrid vision example: Qwen3.5-9B makes 8GB Q4 and 12GB Q8 boundary cases, with 12GB and 16GB as the more practical starting tiers.

Compact hybrid vision example: Qwen3.5-4B treats 4GB Q4 and 6GB Q8 as boundary cases, with 6GB and 8GB as the more practical starting tiers.

Small hybrid vision example: Qwen3.5-2B treats 2GB Q4 as a boundary experiment, 4GB as the practical quantized tier and 8GB as the practical BF16 tier.

\n

Buying specifically at 12GB? Use the RTX 3060 vs RTX 5070 local-AI buyer guide for equal-capacity model gates, power, software and used-card safeguards.

Starting at the 8GB tier? Compare equal-capacity Ada and Blackwell paths in Best 8GB GPU for local AI: RTX 4060 vs RTX 5060, including exact Qwen3.5-4B file margins and the 12GB/16GB exit criteria.

Starting in the 8GB tier? The RTX 4060 8GB local-AI guide turns the capacity boundary into a board, software and complete-system checklist.

Current-generation 8GB path: the RTX 5060 8GB local-AI guide turns the capacity boundary into a Blackwell software, board and complete-system checklist.

Current-generation 12GB path: the RTX 5070 12GB local-AI guide turns this capacity boundary into a Blackwell software, board and complete-system checklist.

Direct 12GB-versus-16GB hardware decision: compare RTX 5070 12GB with RTX 5060 Ti 16GB without confusing GPU tier with memory capacity.

Same GPU family, different capacity ceiling: compare RTX 5060 Ti 8GB and 16GB using exact 7B/14B artifacts, runtime headroom and complete-system checks.

Same Ada family, different capacity ceiling: compare RTX 4060 Ti 8GB and 16GB using exact model files, runtime headroom and complete-system checks.

Same Ampere name, two capacity tiers: compare RTX 3060 8GB and 12GB using exact model files, 128-bit/192-bit identity and runtime headroom.

See the 8GB-versus-12GB purchase fork in hardware: RTX 3060 12GB vs RTX 4060 8GB for local AI maps the same capacity boundary onto exact NVIDIA platforms and Qwen files.

See the 8GB-versus-12GB fork across three generations: after the Ada comparison, use RTX 3060 12GB vs RTX 5060 8GB for the Blackwell path.