I am writing my own self-supervised models from scratch. I am trying to apply MXFP8 and MXFP4 training according to the Transformer Engine guide. Unfortunately, I am getting the message that “AssertionError: MXFP8 (for all gemm layouts) is not supported on 12.0+ architectures yet.” MXFP4 just causes…

I’m bumping this because it’s actually important if using the Spark to develop models that scale on GB200’s etc. To maximize my Sparks, and I bought 4 of them, I want to use MXFP8 and MXFP4, along with structured sparsity. As far as I can tell, none of this works yet with sm_120 (50xx series and RTX…

Unfortunately, transformerengine does not yet support sm121. I will try to update you and others when it does [image] How to enable nvfp4 DGX Spark / GB10 Hello, I’m trying to use nvfp4 on DGX Spark. From the errors I get and the issues and PRs in TransformerE…

Thanks for getting back to me. I am training Hopper-style with FP8 and delayed scaling so the Transformer Engine is working in FP8 right now. sm_121 will be supported completely for MXFP8 and MXFP4 at some point though?

Transformer Engine and GB10 - MXFP8 and MXFP4 training not yet supported?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

aniculescu November 19, 2025, 5:44pm 3

Unfortunately, transformerengine does not yet support sm121. I will try to update you and others when it does

Topic		Replies	Views
How to enable nvfp4 DGX Spark / GB10	5	1015	November 6, 2025
4090 doesn't have fp8 compute? CUDA Programming and Performance	20	19536	August 6, 2024
FP4 on DGX Spark — Why It Doesn't Scale Like You'd Expect DGX Spark / GB10	213	6593	March 13, 2026
Dearest CUTLASS TEAM, When the hell are you going to properly fix tcgen05 FP4 support for DGX Spark / GB10 (SM121)? DGX Spark / GB10	37	2392	April 25, 2026
Request escalation to Product Manager: NVFP4 training support on DGX Spark / SM120 (Hadamard/RHT) DGX Spark / GB10	1	267	February 4, 2026
nvFP4 training - Playbook request DGX Spark / GB10	12	540	March 16, 2026
Will FP8 releated schema in TransformerEngine upstream to PyTorch? GPU-Accelerated Libraries	1	422	January 18, 2024
I am EXTREMely disappointed with the current state of DGX Spark DGX Spark / GB10	90	16798	June 17, 2026
Running GLM-4.7-FP8 (355B MoE) on 4x DGX Spark with SGLang + EAGLE Speculative Decoding DGX Spark / GB10 Projects	39	2186	April 20, 2026
GB10 (SM12.1) vLLM FP8 inference — any progress on native SM12.1 kernels? DGX Spark / GB10 cublas , nemotron	3	791	March 27, 2026

Transformer Engine and GB10 - MXFP8 and MXFP4 training not yet supported?

Related topics