Dual Spark Ducted Cooling Cage

I extracted these CPU limiting commands from a script that I have been running this on my machines with noticeably cooler temps. I have had no trouble with it, but use at your own discretion.

I know this works because I thought people were saying to underclock the CPU the whole time, but it turns out they were talking about the GPU. This doesn’t bring temps down as much as GPU underclocks, but with both CPU p and e cores and GPU underclocked to about 1.8GHz, I rarely reach over 70C. Of course, you will experience slower TG and PP.

echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/policy*/auto_select >/dev/null          # required first, or cap is ignored
echo 2000000 | sudo tee /sys/devices/system/cpu/cpufreq/policy*/scaling_max_freq >/dev/null   # kHz, i.e. MHz * 1000

Also, the p vs e core script which allows you to maintain higher e-core clocks under the assumption they generate less heat:

grep -H . /sys/devices/system/cpu/cpufreq/policy*/cpuinfo_max_freq       # low number = E cluster, high = P
echo 2000000 | sudo tee /sys/devices/system/cpu/cpufreq/policy5/scaling_max_freq >/dev/null    # P
echo 2200000 | sudo tee /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq >/dev/null    # E

Thanks @urtho for the140mm Spice Harvester design! Combined mine with an NF-A14 industrialPPC-3000 PWM controlled by a esp32-c3 which polls GitHub - ateska/dgx-spark-prometheus: A Prometheus metrics exporter for NVIDIA DGX Spark clusters. · GitHub for CPU/GPU temps and can adjust speeds according to a user-defined fan curve.

I don’t suppose anyone has tried flipping the fan exhaust direction on this design and having it act as a forced air intake rather than an exhaust? On my AI TOP ATOM variant I think much of the intake is being done by a cutout on the bottom which is quite nicely covered by this existing design without any modification. This might have the added benefit of allowing you to use a standard fan mesh dust filter without any additional printing or bending.

Hi, I’m designing with Claude’s a single enclosure for two GX10s linked over ConnectX-7. Before I turn the concept into a printable model (Snapmaker U1, PETG), I’d like to hear what’s wrong or naive about it.

Facts the design is built on: the GX10 draws air through the bottom and exhausts at the rear; the ribbed front is decorative. Instead of a stack, both units stand vertically on their sides, lids facing each other (36 mm channel with a stiffening beam), bottoms facing outward.

  • Intake: each bottom sits in a sealed pocket with a 140 mm fan (Noctua industrialPPC-2000) pushing outside air in, 12 mm from the bottom; a G4 filter fleece in front of the fan under a magnet-held panel.

  • Exhaust: a sealed rear plenum covering both exhaust grilles and the channel between the units, two 80 mm fans (NF-A8) stacked, with a 20 mm tunnel between them for the DAC cable running from the lower QSFP of one unit to the upper QSFP of the other (the units are mirrored).

  • Ports stay outside the plenum with 22 mm of clearance for plugs on each side.

  • Control: Aquacomputer QUADRO hub in the base, speeds set from the GX10 via liquidctl based on both machines’ temperatures.

Just found out this thread exists. Here is my setup:

I’m using the IKEA UPPATVIND with a 3D printed duct to help force air into the sparks. Works well and keeps the stack cool and clean.

You can download the model here:

https://makerworld.com/en/models/3288796-ikea-uppatvind-airflow-duct-for-dgx-sparks#profileId-3731241

I love it! please share the files and specs when you feel ready, I’d love to print this one for my DGX Spark

Any good solution to rack mount (in a regular 19in rack) 2 Spark (Dell GB10 and ASUS GX10). I would like those to fit in 1U or 2U (2U probably best with maybe added colling behid the “face” under them (?).

Racknex has a solution a lot use. It looks nice but no cooling. I did this.

Here is my contribution to the art.

Obviously no cage. It’s just to move the air around my 2 units. As well, the usb provides 5 volts but the fan is rated for12 volts. It runs slower, but still moves the air.

I have 2 prototype designs, they both help gx10 drop 18 Celsius when fully loaded.
The the key idea is centrifugal fan. Axial fan produces relatively low static pressure, it works badly in this case.

based on the drawings, if I put FE sparks in this and stood it up so the exhaust was on top, the sparks would be upside down? Can someone confirm if I’m reading that right?
nvm, i think the plans offer both sides if i understand that correcly

Finally finished designing and printing my dual spark rack. It houses 2x MSI Sparks, a 6 port switch, a nuc and a fan controller. All the power bricks are inside the enclosure too. The back has an IKEA UPPÅTVIND air filter with 2x 120mm fans that forces air into the sparks and the overall enclosure cooling the nuc, switch, and power bricks. Exhaust is on top of the rack.

Okey, I must admit that looks cool, well done.

This thread is brilliant since my 2 Spark cooling set up just ran it’s course and now need a 4 unit setup all of the testing done here is brilliant. Well done indeed.

This is how my 3d printed 4-node setup ended up.

I made a way to cool one spark and then just repeat it as many times as needed. The main thing is that I insert air both through the main grill and through the bottom opening. In order for the fan to have somewhere to take air, I set the Spark to stand vertically. Now it does not go out even when the outside temperature is very high.

what switch are you running for your 4 node setup?

It is two CRS504 that I took out of their cases and powered using PoE++.

As a result the AC PSUs were redundant and I could reduce the footprint by over 50%. I left the 40mm screamers in their old cases and use a single 80mm Noctua PWM fan for each switch.

The switch setup is documented here:

That looks very sleek and intentional the fitted CRS504 dual-connect is a very nice touch coupled with the 8-fan hidden set-up. Very well executed

How’s the dBA noise-level? I was worried I might need to resurrect my Skrillex playlist to keep my 4-Sparks cool enough and my mind distracted from all the buzzing! :)

Thanks,

These OEM blower fans have some funny behaviour, the main one is that at a 0% PWM they are idling at 900RPM as I missed the minor point that they were “anti-stall”, well now I know what that means. I have them go to 100% PWM during bootup for a few seconds. But during normal operation they are limited to 20% PWM at most as that was the sweetspot between noise and cooling ability. At 100% speed Skrillex would be a good idea but as of now not an issue at all to sit next to all day long.

The Noctua NF-A8 PWM fans (one in each switch) are very quiet so all in all with the fans going I don’t notice them.

Now the original 40mm MikroTik fans, those are worse than the blower fans at 100%.

That’s a mindfully engineered infrastructure setup.

I’ll wait for my MikroTik CRS804 & Bundle’s arrival by mid-month to see everything in one spot and then factor in the full mini-lab layout with the OMADA & Firewalla Gold Pro also in play.

3D printing was a very wise move and your fan selection, and intentional switch fan swap out is an elegant optimization. I appreciate the share!