Cooler GB10 Temps - (almost) no performance lost

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

So a few times Iowered the clocks of my GB10 during long tasks to reduce the temps and I honestly never felt a lot (if any) performance lost, so today, I decided to actually measure:

image
image2468×1276 380 KB

I ran the same benchmark twice (codeneedle) in this case with Qwen3.6-35b-a3b-NVFP4. First run, normal clocks (hovered between 2400 and 2470mhz) and the 2nd run by limiting the clocks to 2000Mhz.

Run 1 - Stock speed:

  • GPU Power = 43w
  • Speed (avg) = ~113tk/s

you can see from the graph the orange line oscillating between 100 and 130 t/k.

Run 3 - 2000 mhz:

  • GPU Power = 25w
  • Speed (avg) = ~113t/s

Again, you can clearly see that the Generation throughput moves between the same range even at ~18w less.

There are many other benchmarks I can run and document (pre-fill speed, lama-benchy at different pressures, etc,etc) and also other models (I eyeballed a lot 122b-a10b as well) but the result is always the same. Less power draw, lower temps, about the same speed. I was able to bench lower speeds once in a repetitive way, but it was ~2% less at 30% less power, so still a win.

Try it out for yourselves! grab your results and your use case and report back, I’m curious if it’s the same for everyone or if there’s a use case that its really hurt by the reduced speeds.

sudo nvidia-smi -lgc 0,2000 - Reduced speeds
sudo nvidia-smi -lgc 0,3000 - Stock settings.
2

Interesting

3

I have made a similar discovery when reducing the GPU speeds, comparing image generation runs with full and reduced speeds.

In my experience, the reduced speeds caused a roughly ~3% reduction in speed, as posted here:

4

Maybe the overclocked CPU is actually resource starved so the clock Mhz is more for show than usable performance. Such an massive difference in power consumption for such a marginal gain.

5

My mind is ~ the same line of thought.

I found that when at full speed in some context-prefill or heavier models, the power will go up to ~80W, at 2000Mhz it will be ~60W or less. Also, I managed to get longer tasks at full speed that will make the GPU throttle down to ~2150Mhz and I could see how the speed was basically unaffected.

I run almost every GPU in my house down-clocked or undervolted since lately efficiency gains normally show better results that just plain overclocking in all devices (my driving simulator with a 7900XTX runs at a -10% power limit with a -0.150mv underclock). Lower temps, less throttling, more consistent FPS.

I will be running my DGX locked at 2000mhz moving forward :)

6

Will it survive reboot?

For now I see no real drawback from locking to 2000 Mhz.

7

Very nice!

I use to undervolt and powercap my GPU since ages. My RTX5090 e.g. looses roughly 6% performance in case I reduce the power consumption by 25%

Or have a look at the RTX6000 Pro Blackwell GPU. The server version is capped at 300 W, while the workstation version has a power limit of 600W. For AI workloads the server version delivers 88% performance at half the power consumption

8

No with that command, but you can add “–persistent-mode 1” to the end of the command for that :)

9

Awesome test! can you compare the performance differences between prefill only and decode only? If that is even possible.

10

I can consistently capture power consumption during prefill, but instant speed is trickier since the /monitor API from vLLM is not responding me with real-time decoding speed. But I can definitely try to record more data (and I will) with more prefill heavy scenarios.

The scenario I showed in my graph is very VERY similar to my real-world use, so very relevant to me.

If you want to try it in real time, the easiest way is to use Sparkview — GPU monitor tool with GB10-aware unified memory handling as it will show you clocks, power draw, temperature and everything. You can try both clocks configuration and watch the real time results.

11

Update:
Different results but on the same line. I used Introducing Tool Eval Bench CLI with --perf-only which uses Llama-benchy and stresses the GB10 in a different way.

(changed colors this time. RED is GPU temp, Orange here is GPU Power)

image
image1600×877 191 KB

Full Bench run (stock first 2000mhz afterwards)

image
image1098×1328 88.7 KB

You can see a slight drop in most of the runs.

For THIS benchmark, there’s still a drop in power consumption and temperatures but definitely shows a ~5% (or less) drop in performance. Every scenario is different, hence my invite for everyone to test their workflows :)

12

It seems you didn’t notice any difference during the decode phase because the speed there is strictly bound by memory bandwidth. By downclocking the chip, you didn’t really affect that bottleneck. To get the full picture, we should properly look at how the actual decode speed changed.

But to be fair, the Spark has an excellent prefill capability, so a ~10% performance loss there might not even impact real-world usability.

I’ll definitely have to test this out myself.

13

Qwen122b-a10b-hybrid (Albond’s recipe) results.

2000Mhz FIRST and stock last. Red is temperature, orange is Wattage.

image
image1600×647 162 KB

Looks like (consistently) when using lower clocks, the power consumption varies over a wider range but it’s overall lower and the temperatures are definitely lower with non-existent throttling (for me).

With this model, is basically the same performance on middle-size workloads. And better performance when heat-soaked in long-runs.

image
image1199×1066 70.9 KB

@whpthomas I would love to know these results on your workload as you do parallelize things heavily :)

14

It may seem strange, but by and large I am not surprised or seeing the results unexpected. If you avoid throttling in total according to your workload then your results will be better, throttling has performance issues of its own. It depends on your workload.

15

I think this setting makes the most difference to tempts with no meaningful (< 1%) performance loss

I have been running quantisation at 2000Mhz 96% utilisation for 12 hours straight. Peak tempts have been 68°C GPU and 78°C CPU I switched it over to ~3000Mhz for an hour and there is virtually no performance difference, just more heat and more power consumption – for nothing. I believe the overclocking at 3000 Mhz is purely performative and worthless.

Clock Peak GPU °C Peak CPU °C Power Time Performance Loss
1982 Mhz 68°C 78°C 43 Watts 799 s/it
2456 Mhz 82°C 93°C 74 Watts 804 s/it 0.6%
16

I’ve been seen the same repeatedly. Glad it’s working for your workflow too!

17

Are you sure this is correct? These docs say:

Set the persistence mode for the target GPUs. See the (GPU ATTRIBUTES) section for a description of persistence mode. Requires root. Will impact all GPUs unless a single GPU is specified using the -i argument. The effect of this operation is immediate. However, it does not persist across reboots. After each reboot persistence mode will default to “Disabled”. Available on Linux only.

18

Use systemctl to make it persistent across reboots (this is adapted from my GPU config that limits power instead).

create this file (sudo) : /etc/systemd/system/nvidia-power-limit.service

With this content

[Unit]
Description=Set NVIDIA GPU Power Limits
After=nvidia-persistenced.service
Wants=nvidia-persistenced.service

[Service]
Type=oneshot
# Enable persistence mode first, wait for GPU to be ready, then set power limits
ExecStart=/usr/bin/bash -c 'sleep 5 && nvidia-smi -pm 1 && nvidia-smi -lgc 0,2000 && echo "GPU : Persistence enabled, Clock limit set to 2000MHz"'
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Post edit/create

sudo systemctl daemon-reload # reload systemd services
sudo systemctl enable nvidia-power-limit.service # enable service
sudo systemctl start nvidia-power-limit.service # Start service

Credit to : Andrew Zhu

19

I tested this on ZimageTurbo image generation on my Asus GX10. With default settings, it takes an average of 7.25s to generate a 1024x1024 image. Peak temperature is around 84°C by the 4th or 5th image being generated in a batch of 6. Temps can do to high 80s with longer batches.

When I limited the clocks to 2000MHz, it took an average of 8.17s per image, and peak temperature was around 62°C. So a drop-off in speed of around 12.5% or so.

Meanwhile, using the sparkrun recipe for Qwen3.6-35b-a3b with mtp, I barely see any slowdown, maybe 2% on decode speed. And temps never go much above the mid-50s. At least on the dozen or so astronomy queries I gave it.

20

uh, how do you get 113 tk/sec?