Yeah, that was the main selling point for me as well. For training is even better. I was running pre-training for nanochat and it increased tok/sec processing to 33k from 14k. On an 8XH100 node at $24/hr, the pre-training gives a total run time of about 4 hours, so cost is -$100.
On stacked sparks, it takes 5 days and it costs $2.1 * 2 * 5 (depreciation) + $8 * 2 (energy) = $37 for pre-training an LLM (which is not even the focus of the product)
Pretty neat
I know Iām late on this thread but I thought Iād share that we got Talos Linux supported on the Spark as of the 1.12 release. It comes with the LTS 6.18 kernel.
I havenāt done benchmarks yet.
Thanks for the tip on the realtek kernel module. Iāll make sure thatās built for the next release of Talos.
Nice! The other out-of-tree chunk of code is the FFA support. I do want to investigate it more, I would like to know which CPU(s) are used to manage the thermals of the system.
Anyone tried ubuntu 26 on dgx spark lately? Itching to get some firmware updates I need for one of my apps.
Iām using GX10 with ubuntu 26.04 (cuda 13.3, 610 drivers, ZFS), works fine for me (single mode, not sure how itāll do in multi spark config). https://forums.developer.nvidia.com/t/ubuntu-26-04-drivers-610-cuda-toolkit-13-3-zfs-on-gx10
Just thought it was worth mentioning that Iām currently running the 7.0 kernel, CUDA 13.4 and the 615 driver on my Spark using NixOS: Release v1.3.0 Ā· graham33/nixos-dgx-spark Ā· GitHub. It seems to be working well. I also built a 64k page kernel but havenāt tested it yet. Obviously none of this is an NVIDIA-supported configuration.
One thing I ran into was the change in CDDM mode in the 615 driver for coherent memory systems. One effect of that seemed to be that vLLM was very slow to load checkpoints, but --safetensors-load-strategy eager seemed to fix it. See the release notes link above for details.
Just sharing in case itās useful for others.
So other than running latest and greatest. What is the benefit over DGX 7.6.0?
Just the usual reasons to use NixOS vs traditional distros (declarative configuration etc.)
Oh and there was some small performance benefit to the new driver/CUDA/kernel stack. I did some brief testing with vLLM and ComfyUI, I think it was something like 1-8% improvement depending on the use-case.
There are proper tools for Linux hardware bring-up that do this job betterāor, we could say, properly: Yocto, Armbian, Buildroot.
The structural perfection philosophy behind NixOS might actually backfire when dealing with hardware support. When bringing up hardware, the userspace or distribution is largely irrelevant. Trying one distro after another is mostly a waste of time.
Most upstream distributions ship upstream kernels with little or no functional changes relevant to hardware support. Often, the only meaningful difference is the kernel version or the date it was built.
TL;DR: If we want this hardware to work properly with Linuxāany Linux distributionāactual engineering work needs to be done. Itās not just a matter of telling AI to āfix it.ā Someone needs to understand the hardware, identify and fix the underlying problems, and eventually get those fixes merged upstream into the Linux kernel.
Thatās how hardware support becomes available to everyone, regardless of distribution.