MiniMax-M3.1 on AMD Instinct GPUs: Day-0 support and optimization

TL;DR

• MiniMax-M3.1, MiniMax’s natively multimodal MoE model for coding and agent workloads (1M-token context, NVFP4 experts, a DSpark speculative-decoding draft), serves on AMD Instinct™ MI350 GPUs with both vLLM and ATOM on day 0.
• An MXFP4 checkpoint produced with AMD Quark, and an equivalent load-time conversion of the NVFP4 checkpoint, run the routed experts on AITER’s MXFP4 fused-MoE kernels; accuracy on both matches within the evaluation error on GSM8K, AIME and MMMU-Pro.
• vLLM: mono decode, one fused kernel launch per MoE layer for the whole decode step, cuts the per-token decode latency 3.2x at one request, 2.1x at four and 1.5x at sixteen compared with the same server with mono decode off, and gives 2.2-2.4x the interactivity on real coding-agent traces at 1-8 concurrent users.
• vLLM: DSpark speculative decoding composes with mono decode and adds about a third more interactivity at 1-4 users and 13% at 8; on its own it is the fallback where mono decode cannot run.
• ATOM: per-channel FP8 attention and a fused shared expert are applied at start-up, and the DSpark draft pass is CUDA-graph captured, accepting about half of its 7 drafts on real text.
• Serving commands for both engines and both checkpoints, for latency-bound (few users) and throughput-bound (many users) deployments, and for agentic workloads, are given in each engine’s section.

Introduction

We are excited to announce Day-0 support for MiniMax-M3.1 on AMD Instinct™ MI350 Series GPUs across the AMD inference software stack: the ROCm™ build of vLLM and ATOM, the AMD inference engine, both running on ROCm with AITER kernels, with an MXFP4 checkpoint produced by AMD Quark. MiniMax-M3.1 is MiniMax’s latest natively multimodal Mixture-of-Experts model for coding and agent workloads, with a 1M-token context, built-in reasoning with selectable effort, native tool calling and a DSpark speculative-decoding draft.

This post covers what that enablement involved and what we optimized afterwards: how the checkpoint’s NVFP4 experts map onto the MI350 GPUs native MXFP4 and MXFP8 datapaths, the fused “mono decode” kernels and DSpark speculative decoding on vLLM, per-channel FP8 attention, a fused shared expert and CUDA-graph-captured DSpark on ATOM, the accuracy of each configuration, and serving recipes for both engines and both checkpoints.

MiniMax-M3.1 at a glance

MiniMax-M3.1 is the successor of MiniMax-M3: a vision-language Mixture-of-Experts model with a 1M-token context, built-in reasoning with selectable effort, and native tool calling. For inference, four properties matter:

• Sparse attention on every layer. A lightweight indexer scores 128-token KV blocks and attention runs over the top-k blocks only, so decode attention cost does not grow with context.
• MoE with a shared expert. 128 routed experts with 4 active per token, plus a shared expert every token passes through, from layer 3 onward; the first three layers are dense.
• Mixed-precision checkpoint. The routed experts are quantization-aware trained NVFP4; attention projections, the indexer, the dense layers and the shared expert are MXFP8; routers and the LM head are high precision.
• Speculative decoding by DSpark. A separate 5-layer draft reads the target’s residual stream and proposes up to 7 tokens per step.

Enabling the MXFP4 chckpoint with the AMD Quark

AMD Quark provides broad quantization support across dense and MoE models, text, and multimodal architectures, and deployment formats including MXFP4, INT4, FP8, and others. We used Quark to enable the MXFP4 checkpoint for MiniMax-M3.1.

Starting from the baseline NVFP4 checkpoint, Quark selectively re-quantizes the MoE routed expert layers in the main model to MXFP4, while keeping the remaining layers in their original precision. Quark’s file-to-file workflow processes the model checkpoint shard by shard, enabling scalable conversion from the pre-quantized NVFP4 checkpoint to MXFP4 without loading the full model into memory. Quark’s MXFP4 rounding method further helps minimize quantization error, enabling high-fidelity conversion to the lower-precision format.

Quark also supports flexible mixed-precision configurations, creating room for further performance optimization as new kernels and inference capabilities become available on AMD GPUs.

Day 0 support for AMD Instinct GPUs

MiniMax-M3.1 is supported on AMD Instinct MI350 Series GPUs in both inference engines from day 0: vLLM and ATOM. Both run the routed experts on AITER’s MXFP4 fused MoE with fp8 KV and index caches, AITER’s INT4 quick-reduce for the all-reduces, and tensor parallelism over 4 GPUs per model replica, so two replicas fit on an 8-GPU node. vLLM serves both checkpoints below (the MiniMax-M3.1-Flash release with its experts converted at load, and the Quark MXFP4 export); ATOM serves the Quark MXFP4 export. Where the two engines differ is in how they treat the MXFP8 attention linears and the shared expert at start-up, described in each engine’s section.

Supported checkpoints:

Both checkpoints carry the DSpark draft in their dspark/ subdirectory, which the speculative config below points at. The serving stack below accepts either checkpoint: the Quark MXFP4 export as-is, or the original NVFP4 checkpoint with the same NVFP4-to-MXFP4 conversion of the routed experts applied at load time.

Serving with vLLM

What we Optimized

The decode path. A decode step on the day-0 path was dominated by kernel-launch overhead rather than arithmetic: about 1,200 small kernels per step. Before fusing anything, each per-layer module was moved to the fastest available kernel: fp8 KV and index caches, AITER’s sparse paged attention with the shuffled KV layout, the fused QK-norm/RoPE kernel, a stacked router GEMM, and the AITER MXFP4 fused MoE with the shared expert optionally folded into the expert table. Each of these is accuracy-neutral.

Mono decode: one kernel launch per layer. The decode step then went from about 1,200 launches to one per MoE layer. Each sparse MoE layer runs as a single FlyDSL kernel that fuses the QKV projection, QK normalization and RoPE, the KV and index-cache insert, the indexer scoring and top-k block selection, the sparse attention itself, the output projection, the tensor-parallel all-reduces (done in-kernel over peer memory), the RMSNorm and router, and the MXFP4 MoE with the shared expert. The kernels are adapted from ROCm ATOM’s MiniMax-M3 mono decode; the vLLM host side dispatches a step to them whenever it is a pure decode of at most 16 tokens under a full-decode CUDA graph, and leaves everything else (prefill, wider steps) on the standard path. Throughout this post “mono decode off” means that standard path: the same server and kernels with the per-layer modules run one after another instead of as the fused kernel. Inside the kernels the attention projections run in per-channel fp8 (re-quantized once at startup from the checkpoint’s MXFP8); the shared expert runs in MXFP8. Every accuracy number below for a mono configuration was measured with this in effect.

Speculative decoding with DSpark. The shipped DSpark draft runs on ROCm with its attention on the Triton backend. The draft commits about 3 tokens per step at K=3 on real text (two accepted draft tokens plus the target’s own) and about 4.5 at its full 7-token block. It composes with mono decode: the verify step’s K+1 rows per request go through the mono kernels as long as the total stays within the 16-row step, which covers up to 4 concurrent requests at K=3.

Agentic serving configuration. Agent workloads (long shared prefixes, multi-turn tool calls, 100k+ token contexts) use the same engine with a GPU-resident prefix cache, large prefill chunks, admission at twice the expected concurrency, a dense CUDA-graph capture list, AITER unified attention, INT4 quick-reduce for the all-reduces, streaming every 20 tokens and the reasoning parser on; this matches the InferenceX AgentX server configuration.

Additional enablement and tuning. Load-time re-quantization through vLLM’s standard online-quantization option; tuned MXFP8 tiles for the small-batch dense and shared-expert GEMMs; tuned AITER fused-MoE rows for the Quark export’s expert shape; the vision encoder on AITER flash attention with data-parallel encoding; AITER’s INT4 quick-reduce for the decode-sized all-reduces. Native NVFP4 support (serving the NVFP4 experts and an NVFP4 KV cache without conversion) was added last and is available as an option; the recipes here use the MXFP4 expert path, which is the tuned one. The same vLLM build also serves MiniMax-M3.1 on AMD Instinct MI300 Series (gfx942) GPUs on the standard path; mono decode and the native MXFP8 GEMMs are MI350 Series (gfx950) features.

Performance, mono decode on vs off (same GPUs, same checkpoint, same server settings)

Decode latency and throughput on random prompts (8k-in / 1k-out unless noted):

Above 16 decode tokens per step the mono kernels do not apply and the dispatcher leaves the step on the original path, so the two converge at high concurrency. With mono decode off, folding the shared expert into the expert table is worth +19% at 32 and +15% at 64 concurrent requests; INT4 quick-reduce adds +12% at 32 and +19% at 64 on that path and cuts time to first token by 10-29% on every path.

On the InferenceX agentic-coding traces (real coding-agent sessions with 100k+ token contexts, one hour per point), interactivity (tokens per second per user) and throughput per GPU, relative to mono decode off at the same number of agents:

DSpark at K=7 without mono reaches 80-90% of mono’s interactivity at 8-16 agents at equal throughput, so it is the recipe for deployments that cannot run the mono kernels rather than a step up from them. The Quark MXFP4 export measures the same curve as the converted NVFP4 checkpoint at every point.

How to run

The commands below run inside the ROCm vLLM container on an MI350 Series node, 4 GPUs per replica. The upstream vLLM recipe for MiniMax-M3.1 on the AMD Instinct GPUs is at vLLM recipes: MiniMax-M3.1 on AMD; the container that ships the MiniMax-M3.1 work is vllm/vllm-openai:minimax-m31-flash.

Recipe 1: release checkpoint, few concurrent users – mono decode + DSpark. Up to 4 concurrent requests. Remove --language-model-only and add --mm-encoder-tp-mode data --mm-encoder-attn-backend ROCM_AITER_FA --mm-processor-cache-type shm  for image and video inputs.

The reasoning is returned in reasoning_content, and the answer 391 in content.

Accuracy

Evaluated with lm-eval against the running server, per engine and checkpoint. GSM8K on the full 1,319-question test set (5-shot chat with the few-shot turns as prior chat turns, flexible / strict match); AIME 2025 and AIME 2026 as mean@16 (16 seeded passes, temperature 1.0, top_p 0.95); MMMU-Pro standard and vision splits (vision tower on). The vLLM rows were served with recipe 1 and recipe 2 above (mono decode + DSpark draft, maximum reasoning effort); the ATOM rows with the ATOM commands above (the accuracy-first row is PTPC attention with the shared expert kept as a separate MXFP8 MLP, with DSpark).

What’s next

• Wider mono steps. The 16-token step limit is why mono + DSpark stops at 4 concurrent requests and mono alone converges with mono-off at 32; wider steps are the next kernel item and the unlock for speculative decoding at 8-16 users.
• ATOM: mono decode for M3.1. Bring ATOM’s fused per-layer decode kernels, which serve MiniMax-M3’s small decode steps, to M3.1, including its DSpark verify steps; vLLM and SGLang with the ATOM plugin.

Acknowledgements

We thank the MiniMax model team, Inferact, and the AMD vLLM, ATOM, ROCm, AITER and Quark contributors for the model, serving integration, kernel and quantization work that underpins this enablement effort.