Hi everyone — first post here. I recently got a DGX Spark Founders Edition and have been working through the local inference stack. Genuinely impressed with the hardware, and the community around it has been incredible.
I’m posting because I’ve hit a wall trying to get native NVFP4 compute working, and I’d love some guidance from NVIDIA engineering on whether there’s a timeline for software support. The community has done amazing work proving the hardware is capable — I’m hoping to understand what’s coming on the official software side.
What the community has proven
Community researcher baristankut demonstrated 356 TFLOPS on SM121 dense NVFP4 GEMM using CUTLASS 4.4.0. The approach was a one-line Python DSL patch adding sm_121a to BlockScaledMmaOp.admissible_archs. This confirms that native FP4 tensor core compute works on GB10 silicon — which is exciting.
The architectural context I’ve pieced together
From what I can tell, datacenter Blackwell (SM100) uses tcgen05 (fifth-generation tensor core instructions), while desktop/consumer Blackwell (SM12x, including GB10) uses a different instruction path. Both are marketed as “Blackwell,” but SM100 FP4 kernels can’t run directly on SM12.1.
baristankut’s work shows that SM121-specific FP4 kernels are achievable through a different kernel path. If I’m understanding this correctly, the hardware capability is there — it’s just a matter of the production software stack targeting it.
Please correct me if I’m getting any of this wrong.
Current CUTLASS status
CUTLASS issues #2800, #2802, and #2947 document that tcgen05 / FP4 MMA ops in the CuTe Python DSL are currently restricted to sm_100a and sm_103a, and sm_121a is explicitly excluded. These appear to still be open as of March 2026.
What I’ve observed across the available software paths
Based on community findings and documentation, here’s the current state across available paths:
Vanilla vLLM: ~1 tok/s with NVFP4 models. Appears to be a catastrophic fallback, likely PTX illegal instruction.
NVIDIA official vLLM image (26.01): NVFP4 runs via FlashInfer-CUTLASS, but performance matches AWQ exactly. No measurable advantage over generic INT4.
Avarok community image: Best result so far — NVFP4 via Marlin backend runs ~20% faster than AWQ.
Thanks to the Avarok team for this image, and to community members eugr and RobTand for their ongoing contributions to GB10 inference work.
A distinction I want to make sure I understand correctly
The Marlin backend (which is the best-performing path right now) appears to use FP4 as a weight storage/compression format, with dequantization to BF16 at runtime. The actual compute runs on BF16 tensor cores. This gives a memory bandwidth benefit but isn’t using the FP4 tensor cores directly.
Native NVFP4 — what baristankut demonstrated — feeds FP4 inputs directly to the FP4 tensor cores with no dequantization. These seem like fundamentally different execution paths.
Am I understanding this correctly? And if so, is the native path something the production stack is planning to support on SM121?
Questions I’m hoping NVIDIA engineering can help with
1. Is there a planned timeline for SM121-targeted NVFP4 MoE grouped GEMM kernels in CUTLASS, FlashInfer, or an official vLLM image that uses native FP4 compute rather than Marlin dequant?
2. Are CUTLASS issues #2800 and #2802 being tracked for sm_121a FP4 MMA support in the CuTe DSL? Any visibility into the roadmap would be really helpful.
3. NVFP4 was highlighted as a key DGX Spark feature at launch and again at GTC 2026 with the Nemotron 3 Super announcement. Given the community has confirmed native FP4 compute works on GB10 hardware, is there an official plan to bring this to the production software stack?
Thanks in advance for any guidance. The DGX Spark is a fantastic piece of hardware, and I’m looking forward to seeing the software stack catch up to what the silicon can do.