(I realise this would be totally unsupported and likely painful). Just wondering if anyone has considered this?
See this thread:
Lack of kernel/driver support will make other distros not that much fun. Some donât even have ARM support.
Thanks for the reply. I was able to get a NixOS (my usual distro) USB image to boot fairly easily, so Iâm planning on installing that going forward. Iâll share details soon in case itâs any help to anyone else.
3 Likes
Hi @graham36 I have moved this topic to the DGX Spark / GB10 User Forum > DGX Spark / GB10. The DGX Spark / GB10 User Forum > DGX Spark / GB10 Projects category is intended for discussion related to projects built on DGX Spark.
1 Like
With Fedora, Iâm experiencing weird issues with LAN, where Ethernet adapter gets disabled on reboot and wonât function until I turn it off completely and then cold boot. I disabled fast boot in BIOS - same thing. I think itâs somehow related to ACPI, but not sure. Also, GPU performance is worse there (not much, but still) for some reason.
Definitely not ready for prod use. Iâll keep experimenting as time permits, but will continue using DGX OS for real work.
For the networking part, you need the changes to add the r8127 driver, at least. There are probably quite a few other changes youâd need to cherry-pick to get up to par with the DGX OS kernel; havenât gone through them and figured out which ones are essential vs. just nice to have.
1 Like
This is useful, thanks! Any plans to upgrade the kernel in DGX OS to a more modern one? 6.11 is quite oldâŠ
EDIT: Actually, rtl8127 driver was a part of Linux kernel since 6.16, I believe. It definitely works with my 6.17.5 kernel. The only issue is that it stops working during shutdown cycle, and is not visible after reboot (even in BIOS and DGX OS), unless I completely shut it down and power on the system again. Then it works again.
That doesnât happen when I reboot DGX OS though.
The GPU performance issue is a head scratcher though. Is NVIDIA driver different for Spark than a regular 580.95.5 one?
So far things have been working pretty straightforwardly with NixOS. I put my config here in case anyone is interested: GitHub - graham33/nixos-dgx-spark: Basic NixOS config for the DGX Spark . You can generate a USB image to boot from with that repo.
1 Like
Does your NIC experience the same behavior as I described, IOW ethernet adapter is not available after reboot? What kernel are you running?
@aplattner is there anything else important for functionality or performance in the Nvidia kernel branch? If so I may try to build it for my NixOS install. Thanks!
@eugr I havenât actually tried the ethernet adaptor yet, Iâll let you know. I think Iâm using 6.17.5 as well.
Youâll probably experience the same issue. Wifi adapter doesnât do this, only Ethernet.
Iâm tempted to build 6.17 kernel from Nvidia-kernel repo that was linked here before, but probably not until next week - spent way too much time tinkering with the system :)
1 Like
If you want to do computer-aided computing, there are plenty of alternative solutions you can consider. I suggest, for example, compiling ROCm on Strix Halo, or compiling vllm from scratch on any platform, on a machine with 16GB of RAM, just for the fun of it. Compile ROCm on Strix Halo with PyTorch, multiply two matrices to see what happens, and then repeat the process at the segfault. What a treat!
The support of native Nvidia distributions, with, for example, native software support for the ConnectX7 QSFP56 interface, allows you to have CUDA in a plug-and-play manner.
I also recommend compiling CUDA with native support for your Nvidia card. Iâve tested it for you, and honestly, even though Iâm an old Gentoo user, having an up-to-date and proven version saves a considerable amount of time. Time that can be invested elsewhere.
Why on Earth did you post this?
But since we are here - DGX Spark is far from plug-n-play for many things other than supplied examples.
I suggest you try to compile vLLM from scratch on DGX Spark and try to run it using just the official guidance, with no patches or workarounds. Have fun!
And, no, NVIDIA provided Docker image wonât cut it, because support for certain models (Qwen3-VL, Qwen-Next) were added in 0.11.x branch, and NVIDIA is still using 0.10.x even in their October 25 version of the container.
2 Likes
I am also a NixOS user/contributor so this is particularly interesting to me. I did some digging and figured I would leave some notes here; first off, the default repositories have multiple kernels available, including newer ones like 6.14 and 6.17, in -lowlatency (RT_PREEMPT?) and -64k page size variants too. Run apt search âlinux-image.*nvidiaâ to see some results.
I suspect these kernels should work just dandy given that a random distro with an upstream kernel (NixOS) can UEFI+ACPI boot cleanly today. (If anything, Iâm only disappointed there are not -16k page size kernels available, as I find that a wonderful sweet spot, but perhaps the datacenter Grace CPUs donât support that configuration.) So, just installing them as expected in Ubuntu should work if all you need is a newer kernel and donât care about the specifics of your Linux distro â or you simply prefer Ubuntu already. I have not yet done this myself, but I planned to soon before seeing this thread.
The current latest nvidia kernel series available in the apt repositories is 6.17, whose list of outstanding patches vs upstream 6.17 can be seen here: Comparing linux...24.04_linux-nvidia-6.17-next · NVIDIA/NV-Kernels · GitHub
This is about 800 patches, but note that the vast majority of those patches are actually generic Ubuntu kernel patches that only are e.g. touching .deb packaging configuration, or patches for other supported devices (like x86 DGX systems, etc.) The vast majority of the actual nvidia+ARM-related patches seem to be near the top of the stack and quite recent, and there seems to not be that many of them.
(If there is a major performance delta between the upstream 6.17 kernel + CUDA, and the nvidia kernel + CUDA, I suspect the answer is in there, somewhere. For example, there may be bugs affecting aggregate link bandwidth, perhaps.)
Note that it appears these kernels are actually a joint collaboration between Canonical and Nvidia, and Canonical engineers are heavily involved it seems. I had considered politely asking them over on Launchpad if they were planning on helping upstream any âgenerally good stuffâ and if they had any insights on upstream plans.
In general, it does look like solid upstream support is not too far away, and the fact that the existing system can already successfully use wifi reliably and the existing display support bodes very well, I think. It might be worth putting some extra effort into getting the upstream kernel in tip-top shapeâŠ
1 Like
I donât see 6.17 there - looks like 6.14 are the latest available (I wonder why donât they run at least 6.14 by default?):
eugr@spark:~$ apt search 'linux-image-*nvidia' | grep -oP '\d+\.\d+\.\d+(?=-)' | sort -u
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
6.11.0
6.14.0
6.8.0
Zen. Keep calm, nothing personal.
Just having a working Pytorch+CUDA implementation is great.
I promise to compile vLLM to figure out. It canât be worse than my last experiences.
For me, the major performance difference was 3-4x faster model loading into (V)RAM. But weirdly enough, inference (and prompt processing) were slower with upstream kernel, at least on llama.cpp. Not sure whatâs going on there, but model loading is definitely slower than it should be.
I have a Strix Halo system with the same memory bandwidth (actually, a bit slower), and model loading speed is the same between Spark and Strix Halo if using 6.17.5 kernel on Fedora, but 3-4x slower on DGX OS.
Youâll probably experience the same issue. Wifi adapter doesnât do this, only Ethernet.
Youâre right, after reboot the wired ethernet doesnât work (WiFi still does). I will try to build the Nvidia kernel and see if that helps.
And yes confirmed Iâm on 6.17.5:
Linux nixos 6.17.5 #1-NixOS SMP Thu Oct 23 14:24:41 UTC 2025 aarch64 GNU/Linux
The Ethernet controller is using the Realtek r8127 driver:
0007:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. Device 8127 (rev 05)
Subsystem: Realtek Semiconductor Co., Ltd. Device 0123
Kernel driver in use: r8127
Kernel modules: r8127
What is the output of modinfo r8127 on Fedora or NixOS?