How to Run Muse Spark Open Locally: Official Runtime Paths

Short answer: choose the checkpoint and hardware first, then use the runtime that Meta documents for that path. The official NVFP4 checkpoint is a multi-GPU deployment: Meta’s current quick starts use 4×B300 with SGLang or vLLM, while its model card states a minimum of 700GB aggregate GPU memory. The BF16 checkpoint needs roughly 2TB of GPU memory. Smaller local paths use different GGUF or MLX artifacts and must not be presented as equivalent to the official NVFP4 recipe.

Before you download 533GB of weights

1. Confirm access and license. Request access to the gated Hugging Face repository and read the Muse Model License before downloading or deploying.
2. Choose NVFP4, BF16 or an alternate quant. The site’s measured repository inventory found 533.011GB of listed safetensors for the official NVFP4 revision checked on October 25, 2026. That is not the same as live-serving memory.
3. Name the exact hardware and topology. Record device model, per-device exposed memory, NVLink/NVSwitch or other interconnect, host memory, storage and network.
4. Start below maximum context. The model supports up to 1,048,576 tokens, but KV-cache memory grows with context. A maximum is not a recommended starting allocation.
5. Pin the runtime and container or commit. These recipes are new and change quickly. Record the image tag, framework revision, driver and CUDA or ROCm versions.

Official NVFP4 path with SGLang

Meta positions SGLang for low-latency, high-throughput concurrent serving. Its current NVFP4 quick start uses tensor parallelism 4 and expert parallelism 4 on four B300 GPUs, the modelopt_fp4 quantization setting and the named Muse Spark Open development image. Do not copy only the model ID and omit the parallelism, kernel and parser settings.

docker pull lmsysorg/sglang:dev-muse-spark-open

sglang serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --tp-size 4 \
  --ep-size 4 \
  --quantization modelopt_fp4 \
  --moe-runner-backend flashinfer_trtllm \
  --mem-fraction-static 0.8 \
  --host 0.0.0.0 \
  --port 30000

This is an abbreviated orientation command, not a replacement for Meta’s full recipe, which also specifies launch-time performance, speculative-decoding, multimodal transport, cache and parser flags. Use the complete current command from the official deployment documentation.

Official NVFP4 path with vLLM

Meta’s current vLLM recipe also uses four B300 GPUs. It enables tensor parallelism 4, expert parallelism, the Muse reasoning and tool parsers, and DSpark speculative decoding. The published --max-model-len 1048576 value exposes the full context ceiling; lower it when your workload does not need that allocation.

export VLLM_USE_RUST_FRONTEND=1

vllm serve meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --max-model-len 1048576 \
  --reasoning-parser muse \
  --tool-call-parser muse \
  --enable-auto-tool-choice

Meta lists dedicated NVIDIA and AMD vLLM images, but that does not make every GPU count or checkpoint format interchangeable. Follow the vendor-specific recipe and validate each device’s peak allocation.

Portable local path with llama.cpp

Meta’s deployment guide documents llama.cpp for ggml-org GGUF artifacts across Metal, CUDA, ROCm, Vulkan and CPU. These are alternate quantized artifacts—not the official NVFP4 checkpoint analyzed in the 700GB guide.

llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K \
  --chat-template-kwargs '{"reasoning_effort":"low"}'

The official page also lists Q4_K_S and Q8_0. A smaller quant can reduce the weight footprint, but cache, runtime state and model quality still need to be measured. For long-context agents, Meta notes that llama.cpp’s concurrent slots divide the available context; start with one slot when the context requirement matters.

Unsloth and Apple MLX paths

Meta’s Unsloth example starts a local server with a Dynamic 2-bit GGUF and a 32,768-token context, then exposes OpenAI- and Anthropic-compatible endpoints. Its MLX reference is different: it distributes the official NVFP4 checkpoint across two or four Mac Studio computers. A published single-512GB-Mac result exists only for separate Q2_K and oQ3e artifacts; it does not prove that one Mac can run the official NVFP4 recipe.

Smoke-test the server before a real workload

1. Wait for the runtime’s documented ready signal.
2. Query /v1/models and record the exact returned model ID.
3. Send one short text request at batch 1 and concurrency 1.
4. Record peak memory on every device, not only aggregate memory.
5. Verify reasoning content, final content and stop behavior. Meta warns not to stop on <|eom|>.
6. Add image, audio or video inputs separately and check correctness before increasing context or concurrency.
7. Only then measure latency, throughput, failures, power and throttling.

Which path should you choose?

• 4×B300 production node: start from Meta’s complete SGLang or vLLM NVFP4 recipe.
• Blackwell optimization work: use the exact TensorRT-LLM branch and validated four-GPU configuration named in Meta’s guide.
• Very large Apple-silicon deployment: use Meta’s distributed MLX recipe across two or four Mac Studios.
• Single 512GB Mac experiment: use the separately documented GGUF/MLX quant paths and preserve their artifact identity.
• No suitable hardware: choose a hosted provider from Meta’s current list rather than pretending a local configuration fits.

Primary sources

• Meta: Run Muse Spark Open inference
• Official NVFP4 model card and repository
• Official BF16 model card and repository
• Hugging Face Transformers documentation for Muse Spark

Evidence boundary: this page summarizes the current official runtime choices and a few orientation commands checked October 25, 2026. Framework support, images, branches and flags can change; use the linked primary documentation as the operational source of truth. No command on this page was executed against the named hardware by HighVRAM Hardware Watch, so this is a deployment map—not an independent performance benchmark.