Day 0 Enablement and Optimization for Muse Spark Open on AMD Instinct GPUs

Oct 15, 2026

Introduction

AMD is excited to announce Day 0 support for Meta’s latest Muse Spark open-weight release on AMD Instinct™ MI355X and MI350X GPUs with AMD ROCm™ Software and vLLM. 

Muse Spark Open is the first open-weight model in the Muse Spark family, and is the most capable US open-weights model to date. It is built for coding agents handling multi-file refactoring, extended debugging, and other long-running tasks. The natively multimodal Mixture-of-Experts model has 816 billion parameters, with 42 billion active per token, a 1M-token context window, configurable reasoning, and native tool calling. It is available on Hugging Face and through Muse Code, Meta Model API, and OpenRouter. On AMD Instinct GPUs, vLLM supports BF16 and AMD-quantized MXFP4 and FP8 variants.

Deploying Muse Spark Open on AMD Instinct GPUs with vLLM

Day 0 on vLLM

Muse Spark Open serves on a single node of 8x AMD Instinct MI350X/MI355X (288 GB HBM3E per GPU, 8 TB/s) with tensor parallelism across the 8 GPUs. The stack is ROCm 7.2 with vLLM, using the AMD AITER kernel library for attention, MoE and communication, and CUDA-graph capture of the decode step. The following checkpoints are supported:

Variant

Model Name

Notes

BF16 (the open release)

meta-models/Muse-Spark-Open-816B-A42B

Full precision; 32 concurrent 32k-token requests per node

MXFP4 (AMD Quark export)

 amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4

This checkpoint quantizes the routed MoE experts to OCP MXFP4 (except those in layers 0 and 61) and keeps everything else in BF16.

FP8 (AMD Quark export, PTPC)

 amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8

This checkpoint quantizes the routed MoE experts to OCP MXFP4 (except those in layers 0 and 61) and keeps everything else in BF16.

The BF16 checkpoint can also be served with an opt-in mode that rounds the routed experts to MXFP4 at load time, which gives the MXFP4 memory and throughput profile without a separate checkpoint.

Key Optimizations:

  • Muse Spark Open (816B total / 42B active MoE, 1M context) serves on one 8x MI350X node with vLLM in BF16, MXFP4 and FP8, plus a BF16 mode that rounds the experts to MXFP4 at load time.
  • Decode-step work (fused attention prologue and MoE finalize, sliding-window layers on AITER paged attention, ~180 fewer kernel launches per step, exact bf16x3 router GEMM) took the MXFP4 checkpoint from 206 to 350 output tok/s at 4 concurrent requests and from 719 to 1,180 at 32.
  • Expert parallelism (--enable-expert-parallel, 32 of 256 experts per GPU) adds 7-20% throughput at 32-128 concurrent requests on MXFP4 and cuts time to first token by ~24%: 2,197 tok/s at 128.
  • A tuned AITER fused-MoE table routes small decode batches to a one-stage MXFP4 assembly kernel that skips the sort. Single-request decode goes from 98 to 114 tok/s (+16%) on TP8, with no code change: the table ships in the image.
  • DSpark speculative decoding, with AITER's short-query 3D attention schedule for the verify step and the draft's KV-cache insert kept out of its compiled graph, reaches 199 tok/s single-stream: +72% over TP8 with the tuned table on the same image, +47% at 2, +31% at 4 and still +6% at 32 concurrent requests.
  • GSM8K stays at 94.4 - 95.5% across all MXFP4 configurations (TP, EP, tuned table, DSpark).
  • Pick by load: TP8 + EP8 + DSpark up to about 64 concurrent requests (plain TP8 + DSpark for a single user), TP8 + EP8  for beyond that.

Day 0 vLLM Docker

		vllm/vllm-openai-rocm:muse-spark-open

Day 0 Recipe

The Day 0 recipe is available in vllm-recipe.

How to run

Inside the ROCm vLLM image on an 8x MI350X node. The tuned AITER MoE table and the short-query attention schedule ship in the image; the V2 model runner, breakable CUDA graphs and AITER linear/MoE kernels are auto-selected for this model on ROCm.

Recommended: MXFP4 (AMD Quark export), TP8 + EP8 + DSpark with AITER attention on both target and draft, capped at 64 sequences. This is the fastest configuration from four to 64 concurrent requests and matches TP8 + DSpark below that; for the lowest single-request latency, drop `--enable-expert-parallel`:

		VLLM_ROCM_USE_AITER=1 \ 
VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16 \ 
VLLM_USE_BREAKABLE_CUDAGRAPH=1 \
AITER_UNIFIED_ATTN_SHORT_QUERY_3D=1 \ 
VLLM_KV_CACHE_LAYOUT=LBHNC \ 
vllm serve amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \ 
  --served-model-name musespark \ 
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \ 
  --language-model-only --dtype bfloat16 \ 
  --tensor-parallel-size 8 --enable-expert-parallel \
  --attention-backend ROCM_AITER_FA \ 
  --max-model-len 32768 --max-num-seqs 64 --max-num-batched-tokens 8192 \ 
  --gpu-memory-utilization 0.90 \ 
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' \ 
  --no-enable-prefix-caching --enable-chunked-prefill \ 
  --reasoning-parser musespark1x --tool-call-parser musespark1x --enable-auto-tool-choice \ 
  --speculative-config '{"method":"dspark","model":"Inferact/Muse-Spark-open-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false,"attention_backend":"ROCM_AITER_FA"}'

For throughput beyond 64 concurrent requests, MXFP4 with TP8 + expert parallelism (32 of 256 experts per GPU) and no DSpark, 128 sequences. 

		VLLM_ROCM_USE_AITER=1 \ 
VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16 \ 
VLLM_USE_BREAKABLE_CUDAGRAPH=1 \
vllm serve amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \ 
  --served-model-name musespark \ 
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \ 
  --language-model-only --dtype bfloat16 \ 
  --tensor-parallel-size 8 --enable-expert-parallel --distributed-executor-backend mp \ 
  --attention-backend ROCM_AITER_FA \ 
  --max-model-len 32768 --max-num-seqs 128 --max-num-batched-tokens 8192 \ 
  --gpu-memory-utilization 0.90 \ 
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' \ 
  --no-enable-prefix-caching --enable-chunked-prefill \ 
  --reasoning-parser musespark1x --tool-call-parser musespark1x --enable-auto-tool-choice

BF16 (the open release) with online MXFP4 experts, if you want to serve the original checkpoint without a second download. The routed experts of layers 1-60 are rounded to MXFP4 at load and run on the AITER MXFP4 MoE kernels, while the checkpoint stays BF16; this gives the MXFP4 memory profile, so the server takes 128 sequences. It takes --enable-expert-parallel the same way:

		VLLM_ROCM_USE_AITER=1 \ 
VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16 \ 
VLLM_USE_BREAKABLE_CUDAGRAPH=1 \
VLLM_ROCM_MUSE_MOE_ONLINE_QUANT=mxfp4 \ 
vllm serve meta-models/Muse-Spark-Open-816B-A42B 
 \ 
  --served-model-name musespark \ 
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \ 
  --language-model-only --dtype bfloat16 \ 
  --tensor-parallel-size 8 --distributed-executor-backend mp \ 
  --attention-backend ROCM_AITER_FA \ 
  --max-model-len 32768 --max-num-seqs 128 --max-num-batched-tokens 8192 \ 
  --gpu-memory-utilization 0.90 \ 
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' \ 
  --no-enable-prefix-caching --enable-chunked-prefill \ 
  --reasoning-parser musespark1x --tool-call-parser musespark1x --enable-auto-tool-choice

Requests use the standard OpenAI-compatible API; pass reasoning_effort (minimal .. xhigh) to control reasoning, and tools in the usual tools field.

Conclusion

With the release of Muse Spark Open, Meta continues to push the boundaries of open-weight models. By providing Day 0 support on AMD Instinct™ GPUs with vLLM, we ensure that developers have the compute power and optimized software stack needed to run these massive, high-context models at production scale.

Acknowledgments

 We sincerely thank our partners, Inferact and Meta, as well as the AMD Quark team, for their invaluable collaboration in making this successful launch and Day 0 support possible.

 

Share:

Related Blogs

View All Blogs