1PFLOP - how?

[raphael.amorim]

Official answer from NVIDIA on NVFP4:

NVFP4 Performance Update

Hi everyone, we’ve seen a lot of questions about optimizing vLLM’s inference performance using NVFP4, and GB10’s underlying NVFP4 capability more generally. We’ll answer some of those questions here. First, we’re working to improve NVFP4 performance in vLLM on several fronts. CUTLASS back end improvements. We are regularly contributing fixes and changes to FlashInfer and vLLM upstream. Over the past couple of weeks, FlashInfer and CUTLASS have significantly improved the CUTLASS backend for NV…

Have you tried CUTLASS profiler yet?

CUTLASS Profiler — NVIDIA CUTLASS Documentation

Some of the forum members tried similar things before:

Question on Reproducing DGX Spark (GB10) FP4 1 PFLOPS Performance Using CUTLASS Profiler

Dear NVIDIA Team, I am currently evaluating the FP4 AI performance claim stated on the NVIDIA DGX Spark (GB10) product page, which indicates that the system can deliver up to one petaFLOP of FP4 AI performance. To better understand and validate this claim, I would like to confirm whether the following CUTLASS-based methodology is an appropriate and NVIDIA-aligned way to reproduce or approximate the advertised FP4 peak performance, or whether NVIDIA recommends a different or more official appro…

For thor: Verifying claimed TOPS performance on Jetson Thor – CUTLASS kernel for SM110 does not run, SM80 gives very low performance (~6.9 TFLOP/s)