Short answer: use 4GB only for a constrained Q4 vision experiment, 6GB as the practical Q4 tier, 8GB for Q8 with bounded inputs, and 12GB for the official 7.509GB BF16 checkpoint. Every GGUF path must include its separate multimodal projector, then leave room for cache, image/video processing and runtime buffers.
The 1.930GB Q4 file is not a complete vision model. The ggml-org conversion also needs an 845MB Q8_0 or 1.338GB F16 projector. A runtime that loads the language file but ignores images has not completed the Qwen2.5-VL deployment.
Qwen2.5-VL 3B files and practical tiers
| Deployment path | Displayed files | Combined file size | Planning tier |
|---|---|---|---|
| ggml-org Q4_K_M + Q8 projector | 1.930GB + 0.845GB | 2.775GB | 4GB constrained; 6GB practical |
| ggml-org Q4_K_M + F16 projector | 1.930GB + 1.338GB | 3.268GB | 4GB tight; 6GB practical |
| ggml-org Q8_0 + Q8 projector | 3.285GB + 0.845GB | 4.130GB | 6GB bounded; 8GB practical |
| ggml-org Q8_0 + F16 projector | 3.285GB + 1.338GB | 4.624GB | 6GB tight; 8GB practical |
| Official Qwen BF16 safetensors | 2 shards totaling 7.509GB | 7.509GB | 12GB practical |
Why 4GB is only an experiment
The smallest complete displayed pair occupies about 2.775GB before KV cache, image processing, compute buffers, allocator behavior and display use. That leaves little space on a 4GB card. A short prompt and small image may work in a compatible runtime, but 4GB is not a reliable target for arbitrary images, context lengths or backends.
Six gigabytes gives Q4 useful working room and can attempt Q8 with bounded inputs. Eight gigabytes is the clearer Q8 tier. The official integrated BF16 checkpoint is already 7.509GB on disk, making 8GB a file-size trap and 12GB the first common full-precision planning tier.
KV cache: 36KiB per allocated text token
Qwen’s official configuration lists 36 language layers, two KV heads and a 128-element head dimension. With FP16/BF16 keys and values, one sequence uses 2 × 36 × 2 × 128 × 2 = 36,864 bytes, or 36KiB per allocated text token.
| Allocated text tokens | FP16/BF16 KV payload |
|---|---|
| 8K | 0.281GiB |
| 32K | 1.125GiB |
| 64K | 2.25GiB |
| 128K architectural maximum | 4.5GiB |
Small model does not mean fixed vision memory
- Dynamic resolution: image dimensions change the visual-token workload.
- Image count: multiple images increase preprocessing and prompt state.
- Video: resolution, sampled frames, duration and frame rate affect memory and work.
- Projector precision: the community Q8_0 and F16 projector options differ by about 494MB.
- Concurrency: parallel requests multiply cache and intermediate allocations.
- Unified memory: the OS and applications share the advertised pool.
Reproducible Qwen2.5-VL 3B fit test
- Record repository, revision, every language/projector file, size and checksum.
- Record runtime/version and confirm image input works for this exact package.
- Record model, projector and cache precision plus accelerator placement.
- Fix image dimensions, image count, video settings, text allocation, output target and concurrency.
- Measure accelerator and system memory before load, after load, after vision encoding and at peak generation.
- Ask image-grounded questions; text-only output does not prove the vision path.
- Capture preprocessing time, time to first token, throughput, fallback and out-of-memory errors.
Start with the Qwen2.5-VL family guide, compare the 7B exact guide, the multimodal hub, and the generic 3B/4B sizing guide.
Use the 8GB–16GB capacity guide, VRAM calculator, unified-memory guide, workload test and benchmark submission.
Continue planning
Browse all guides and tools. Capacity fit does not establish speed, image accuracy or output quality.
Newer adjacent model: compare the Qwen3-VL 4B VRAM guide for DeepStack vision, official projector files and native 256K architecture.
Newer compact sibling: compare the Qwen3-VL 2B VRAM guide for official projector files, DeepStack vision and native 256K architecture.
Sources and limits
- Qwen official 3B repository
- Qwen official configuration
- Qwen2.5-VL release article
- ggml-org community GGUF conversion
- HighVRAM methodology
Reviewed October 25, 2026. Artifact sizes and architecture fields were read from the cited repositories. HighVRAM has not independently benchmarked every runtime, image resolution, video configuration or quantization. File size is not peak resident memory.