How to Run Muse Glimmer 30B Locally: llama.cpp, Ollama and LM Studio

The most direct local path for Muse Glimmer 30B is Meta’s published GGUF with llama.cpp b10353 or newer. Start with the 17GB Q4_K_M checkpoint on a validated 24GB system. Use the roughly 20GB dynamic Q4_K_XL build when you have a 32GB tier. This guide follows Meta’s current commands; it does not claim a speed result for hardware we have not benchmarked.

1. Pick the right official artifact

Artifact	Role	Published size	Planning tier
Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf	Recommended text checkpoint	About 17GB	24GB
Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf	Dynamic K-quant text checkpoint	About 20GB	32GB
mmproj-Muse-Glimmer-30B-Q4_K_M.gguf	Vision projector	About 1.4GB	Additional memory
dflash-Muse-Glimmer-30B-Q4_K_M.gguf	Optional speculative-decoding draft	About 1.6GB	Additional memory

Full-precision weights are not published as GGUF. Meta’s BF16 path uses the safetensors checkpoint with vLLM and has a materially larger serving footprint. See the VRAM requirements breakdown before choosing hardware.

2. Install a compatible llama.cpp build

Muse Glimmer requires llama.cpp build b10353 or newer. An older build can fail with an unknown-architecture error, so check the runtime before diagnosing the GPU or model file.

curl -LsSf https://llama.app/install.sh | sh

That is Meta’s documented Linux installer route. macOS and Windows users can use the llama.cpp team’s Llama app; building from source is also supported upstream.

3. Run a first terminal smoke test

llama cli -hf meta-models/Muse-Glimmer-30B-GGUF \
  --hf-file Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
  -ngl 99 -c 32768 -st

This downloads the named checkpoint, offloads all layers with -ngl 99, allocates a 32,768-token context and exits after one answer with -st. Without single-turn mode, the CLI remains interactive and waiting for input can look like a hang.

Do not begin a hardware comparison at the 131,072-token ceiling. First prove that the exact artifact loads and answers at a controlled context. Then raise context deliberately while recording peak device memory and system memory.

4. Start the OpenAI-compatible server

llama serve -hf meta-models/Muse-Glimmer-30B-GGUF \
  --hf-file Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
  -a muse-glimmer \
  -ngl 99 -c 131072 -np 1 \
  --host 127.0.0.1 --port 8080 --api-key <your-key> \
  --chat-template-kwargs '{"reasoning_strength":"low"}'

This is Meta’s published full-context server form. The model supports 131,072 tokens, but allocation still consumes memory. One slot matters: llama.cpp divides the total context among concurrent slots, so -np 4 with -c 131072 provides 32,768 tokens per slot. Read n_ctx_slot in the startup log to confirm the actual per-request limit.

Verify text output

curl -s --noproxy '*' http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <your-key>" \
  -d '{"model":"muse-glimmer","messages":[{"role":"user","content":"What is 17 * 23?"}]}'

The alias from -a muse-glimmer must match the request’s model name. Reasoning is returned separately from final content by the server. The published chat template supports low, medium, high and xhigh reasoning strength; reasoning cannot be fully disabled on this path.

5. Add vision only after text works

The vision projector downloads with the text model and needs no extra serve flag. Use --no-mmproj when you explicitly want text-only operation. Image input can be a remote URL, a base64 data URI or a local path, and image resolution changes prompt-token use.

Keep vision disclosed in any benchmark record. A text-only run and a multimodal run are different workloads and should not be merged into one performance claim.

6. Add DFlash separately

Download the optional DFlash GGUF from the same repository and append Meta’s documented flags:

-md dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
--spec-type draft-dflash -ngld 99 --spec-draft-n-max 4

DFlash adds another component and changes both memory use and decoding behavior. Establish a clean baseline without it, then repeat the same prompt and context with it enabled. Report both configurations separately.

7. Run Muse Glimmer with Ollama

Ollama is the shortest supported path when you want a managed local download and interactive prompt instead of selecting GGUF files and server flags yourself. Meta requires Ollama 0.32.7 or later.

ollama --version
ollama run muse-glimmer

On Apple silicon, Meta documents the MLX-backed tag:

ollama run muse-glimmer:30b-mlx

Ollama’s native API listens on http://localhost:11434. Verify a non-streaming response before connecting an agent:

curl http://localhost:11434/api/chat -d '{
  "model": "muse-glimmer",
  "stream": false,
  "messages": [
    {"role": "user", "content": "In one sentence, what is Muse Glimmer good at?"}
  ]
}'

Ollama also exposes an OpenAI-compatible chat-completions endpoint. Confirm the resolved tag, context and memory behavior before comparing it with the pinned Meta GGUF route; a library alias is not enough to prove the underlying artifact matches another test.

8. Run Muse Glimmer with LM Studio

LM Studio Bionic is the graphical path in Meta’s deployment guide. In Settings → Explore, select Muse Glimmer and download the recommended file for the available hardware. Create a project, enable Allow Coding only if the model should run commands in local folders, and then select Muse Glimmer from the model picker.

To expose LM Studio’s OpenAI-compatible local API, start the server:

lms server start

The default port is 1234. Query GET /v1/models and copy the exact served model ID rather than assuming the display name is the API name. If loading fails, unload other models and lower context. Meta’s guide reports that LM Studio measured the Q4_K_M path at about 24.1GiB for one request at 16K context, so a nominal 24GB label should not be treated as guaranteed room for this specific application stack.

Security boundary: keep local servers bound to the local machine unless remote access is intentional and authenticated. Review enabled tools before allowing an agent to execute commands or access files.

9. Use vLLM for the BF16 serving path

Meta positions vLLM for production serving of the BF16 checkpoint on NVIDIA GPUs. The weights are around 60GB and Meta states about 72GB of VRAM to serve, so this is not the 24GB or 32GB GGUF path.

docker run --rm --gpus all --ipc=host \
  -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:muse-glimmer \
  meta-models/Muse-Glimmer-30B \
  --served-model-name muse-glimmer \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser muse_glimmer \
  --reasoning-parser muse_glimmer \
  --generation-config auto

Validation checklist

• Runtime name and exact build or container tag
• Model repository revision and complete filename
• Text-only or vision; DFlash on or off
• Total context, slot count and observed n_ctx_slot
• Reasoning strength and output-token limit
• GPU, driver, backend and system-memory configuration
• Peak device memory, peak system memory, time to first token and throughput
• Prompt, image dimensions, output correctness and any errors or throttling

Stage a reproducible benchmark record only after the text smoke test passes. Staging does not submit or publish data.

Plan the hardware

• Check the Mac unified-memory decision guide
• Compare 24GB and 32GB Muse Glimmer candidates
• Inspect screened 24GB and 32GB offer observations — confirm price, condition, shipping, availability and seller details.
• Stage the private Muse Glimmer hardware preset
• Report a correction with primary-source evidence

Primary sources

• Meta Muse Glimmer deployment documentation
• Meta Muse Glimmer 30B GGUF repository
• Meta Muse Glimmer 30B BF16 repository

Checked October 25, 2026. Commands reproduce the cited publisher documentation and are not a claim that HighVRAM Hardware Watch executed every path on every listed GPU.