Only getting half the advertised performance and capping at 100W?

This is originally from a post by John Carmack

DGX Spark appears to be maxing out at only 100 watts power draw, less than half of the rated 240 watts, and it only seems to be delivering about half the quoted performance (assuming 1 PF sparse FP4 = 125 TF dense BF16) . It gets quite hot even at this level, and I saw a report of spontaneous rebooting on a long run, so was it de-rated before launch?

Link to original tweet

Does anyone from NVidia want to address this? I have been running nvidia-smi in a loop during high demand tasks and I’ve never seen it go above 100W usage and even at 85W it gets quite hot at 83C which is nearing the 85C throttling threshold.

What are the plans to address this?

Hi, this is a misunderstanding. Spark GPU can draw a max of 120W and the rest of the system including CPU, CX7 and peripherals can draw 100W. The power supply is rated for 240 Watt of peak draw.

We’re updating our product documentation with these additional details and will follow up here once published.

I’ve never see it draw more than 100W on mine and by that time the temperature is at 85C. I’m using it for finetuning, abliteration, and training.

How do you measure power consumption? At the plug?

nvidia-smi

It’s only pulling 65W and GPU-Util is already at 96% and temperature is at 81°C

Seems like there’s a temperature control issue here.

It shows only GPU power draw and I’m not sure what temperature sensor they are using, it could be for the entire SOC.

Have you reviewed the numbers after the last update?

@tymat This was marked as solved. I’m pretty much in the same boat as you using Nvidia-SMI under load I never exceed 86W

Try to load both the CPUs and GPUs.
Have you done just inference or training as well?

So new firmware update fixes this?

If you are finetuning LLMs, you are probably mostly engaging the Tensor Cores. Those have been relatively the same stats I see when engaging mostly TC, 65W @ 80C range.

Which is good news, right? :D
Low cost training

To follow up on this, I started to hit 90-103W while inferencing WAN2.2 in ComfyUI.

The update helps with the early flushCache situation when trying to get an accurate memory report.

Also I’ve noticed since the update has lower OS memory usage I was at a steady 14GB on startup now I’m at 3GB which is nice.

My problem is solved for now.