DGX Spark repeatedly rebooting after most recent update

jk.chik · October 30, 2026, 6:16am

Hi, Everyone!

After just updating my DGX Spark (Gigabyte) an hour ago, it repeatedly reboots after a minute or so. However, this seems to stop when I unplug my second monitor (connected via USB-C). When connected to the one monitor (HDMI), it seems fine.

John

jk.chik · October 30, 2026, 6:22am

Quick update: It seems to reboot on the monitor, but after a much longer time (~5 minutes). J

leotim.browser · October 30, 2026, 9:45am

We are experiencing the same issue

dengfa.cheng · October 30, 2026, 12:38pm

Hi everyone,

Adding a confirmed root cause + workaround for the reboot issue several of
us hit after the most recent update.

Hardware: DGX Spark Founders Edition, DGX OS 7.6.0, driver 580.178.04 (open)

Timeline (2026-10-30, KST)
15:19 DGX Dashboard ran role-upgrade-system:
linux-nvidia-hwe-24.04 7.0.0-1019.19 → 7.0.0-1024.25
+ linux-modules-7.0.0-1024-nvidia
+ linux-modules-nvidia-580-open-7.0.0-1024-nvidia
+ linux-modules-nvidia-fs-7.0.0-1024-nvidia
15:44 First failure, ~25 min after the upgrade.

Symptoms

Entering the password at gdm drops the session instantly back to the
login screen (no desktop).
Even without logging in, the machine reboots every ~1 minute.
Ctrl+Alt+F3 also aborts back to gdm, so no text console was reachable.

Evidence

No Xid, no kernel panic, no Oops in any of the crash boots.
NVRM 580.178.04 loads normally.
At shutdown: “watchdog0: watchdog did not stop!” and systemd using the
SBSA Generic Watchdog. The session crash is not the reboot itself - the
shutdown path hangs afterwards and the hardware watchdog resets the host.
The driver branch is unchanged (580-open), so the evidence points at a
kernel regression rather than a driver issue.

Workaround (verified on this unit)

Boot 7.0.0-1019-nvidia instead of 1024:
GRUB > Advanced options for DGX OS > … with Linux 7.0.0-1019-nvidia
Pin it:
sudo apt-mark hold linux-nvidia-hwe-24.04 \
linux-image-nvidia-hwe-24.04 linux-headers-nvidia-hwe-24.04 \
linux-tools-nvidia-hwe-24.04 \
linux-modules-nvidia-580-open-nvidia-hwe-24.04 \
linux-modules-nvidia-fs-nvidia-hwe-24.04
Make 1019 the GRUB default.

Result: with 1019 the desktop runs normally for hours on the same display
setup; with 1024 the crash reproduces reliably within a minute.

To check whether you have the affected kernel:
dpkg -l | grep 7.0.0-1024

Side note for anyone who cannot get the GRUB menu to appear:
/etc/default/grub is overridden by /etc/default/grub.d/*.cfg, which are
sourced in alphabetical order. no-grubmenu.cfg (GRUB_TIMEOUT=0,
GRUB_TIMEOUT_STYLE=hidden) loads after menu.cfg, so editing
/etc/default/grub has no effect. Adding
/etc/default/grub.d/zz-timeout.cfg with GRUB_TIMEOUT=10 and
GRUB_TIMEOUT_STYLE=menu fixes it.

Questions for NVIDIA:

Is 7.0.0-1024-nvidia a known regression? Any bug ID we can track?
When a fixed kernel ships, what is the recommended way to unpin and
re-test safely?
Could the DGX Dashboard offer a “roll back the last system update”
action? Right now the only supported path is Dashboard-driven updates,
but when an update lands in this state there is no supported way back.

Thanks

dengfa.cheng · October 30, 2026, 12:46pm

Correction to my post above: the reboots started immediately after the
update, not ~25 minutes later.

The upgrade transaction finished at 15:20 and the reboot that followed was
the first boot into 7.0.0-1024-nvidia - the machine was already in the
reboot loop from that very boot. 15:44 was simply when I started reporting
it, and I mistakenly presented that as the time of first failure.

So the causal chain is tighter than I first described: the first boot into
1024 was already broken, with no window of normal operation in between.

kuwii · October 30, 2026, 4:16pm

I’m using ASUS GX10. My case however is the exact opposite. It hard-resets whenever I plug in HDMI monitor. Using DP via USB-C works fine to me. I did some investigation using AI: Any use of the HDMI (TMDS) output causes a hard reset on kernel 7.0.0-1024-nvidia

aniculescu · October 30, 2026, 4:35pm

We will look into this kernel issue. We need a bit more information.

Do the reboots still occur when no monitors are connected?
During the ~5 minutes that unit is on, can you collect an nvidia-bug-report and send to me?

mfulghum · October 30, 2026, 5:06pm

As long as I don’t have a monitor connected things seem to stay up, but then connecting via either HDMI or DP reboots within a minute or so.

user158576 · October 30, 2026, 5:29pm

This could mean the correct Nvidia GPU driver is not loaded.
Can you run these?

uname -a
nvidia-smi

kirk16 · October 30, 2026, 5:57pm

I can confirm the issue on another DGX Spark Founders Edition. With an HDMI monitor, the Spark rebooted approx every 30 seconds. Everything is fine if I run it headless.

Linux spark1 7.0.0-1024-nvidia #25~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Oct 22 21:37:03 UTC 2 aarch64 aarch64 aarch64 GNU/Linux

±----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 2497 G /usr/lib/xorg/Xorg 19MiB |
| 0 N/A N/A 2690 G /usr/bin/gnome-shell 6MiB |
±----------------------------------------------------------------------------------------+

user158576 · October 30, 2026, 6:06pm

How did you install the 595 driver? 580 should be the default.
This command?
sudo ubuntu-drivers install nvidia:595-open

kirk16 · October 30, 2026, 6:12pm

Ubuntu installed the drivers, as I tried to debug the issue:

"sudo ubuntu-drivers install"

user158576 · October 30, 2026, 6:21pm

these commands should help you find the root cause of your HDMI reboot
sudo journalctl --list-boots
#Identify the HDMI-failing boot, then use its boot ID:
sudo journalctl -b --no-pager > hdmi-failing-boot.txt

haidij · October 30, 2026, 6:23pm

I also had this issue. I had two monitors connected and then after the update experienced an endless cycle of immediate reboots after entering my password. I have a founders edition just replaced on RMA a few weeks ago.

Looking at this forum I disconnected one monitor (connected via USB) and left the one connected via HDMI and it has stayed running (at least 10 mins so far).

Update: I spoke too soon! It restarted again several times. Luckily it stayed up long enough for me to have Claude help me reset to the previous kernel.

vegax87 · October 30, 2026, 6:59pm

I’ve sent you in DM my nvidia-bug-report and journalctl log, hope it helps

Kirk_SparkX4 · October 30, 2026, 7:08pm

Same problem here, on a Dell Pro Max with GB10 FCM1253 (BIOS 5.36_4.1.0, DGX OS 7.6.0). Answering @aniculescu’s questions with what I captured.

Versions

Kernel 7.0.0-1024-nvidia #25~24.04.1 (linux-image-7.0.0-1024-nvidia 7.0.0-1024.25~24.04.1), installed today with apt upgrade at ~14:57 UTC. Previous kernel: 7.0.0-1019-nvidia.
Driver nvidia-driver-580-open 580.178.04-0ubuntu0.24.04.1, the same before and after. Only the kernel module package was rebuilt for 1024 (linux-modules-nvidia-580-open-7.0.0-1024-nvidia 7.0.0-1024.25~24.04.1).
VBIOS 9A.0B.2D.00.00, GSP firmware 580.178.04, CUDA 13.0. dgx-dashboard went 0.39.3-4 → 0.39.3-5 in the same upgrade.

Monitors: one monitor, [HDMI / USB-C: FILL IN], attached the whole time on both kernels; GNOME on Xorg. I have not tested 1024 with no monitor.

Before, on 1019: it ran 3.6 days straight (Oct 26 to Oct 30) with two llama.cpp CUDA servers loaded. No Xid, no Oops.

After, on 1024 (all times UTC, Oct 30). Three crashes and one side effect in about 3.5 hours of uptime:

16:23:49, first boot on 1024, 66 min of uptime: kernel Oops, then a panic (panic_on_oops=1). The journal lost it; pstore kept it. It’s SLUB corruption hit during a clone(), so the process named is probably just the victim:

[ 3981.414331] Unable to handle kernel paging request at virtual address 000f5fd7d47bf529
[ 3981.414384] Internal error: Oops: 0000000096000004 [#1]  SMP
[ 3981.445981] CPU: 9 UID: 1003 PID: 2086258 Comm: codex-main Tainted: G           O        7.0.0-1024-nvidia #25~24.04.1-Ubuntu PREEMPT(lazy) 
[ 3981.448001] Hardware name: Dell  Inc. Dell Pro Max with GB10 FCM1253/038RY7, BIOS 5.36_4.1.0 09/23/2025
[ 3981.450054] pc : __refill_objects_node+0x2f8/0x788
[ 3981.450663] lr : __refill_objects_node+0x224/0x788
[ 3981.460656] Call trace:
[ 3981.460966]  __refill_objects_node+0x2f8/0x788 (P)
[ 3981.461569]  refill_objects+0x88/0x168
[ 3981.462041]  refill_sheaf+0x78/0xa0
[ 3981.462481]  __pcs_replace_empty_main+0x130/0x2e0
[ 3981.463074]  __kmalloc_noprof+0x534/0x608
[ 3981.463579]  lsm_blob_alloc+0x60/0xa0
[ 3981.464045]  security_task_alloc+0x40/0x288
[ 3981.464572]  copy_process+0x5ec/0x1080
[ 3981.465047]  kernel_clone+0xa4/0x500
[ 3981.465498]  __do_sys_clone+0x78/0xe0
[ 3981.465960]  __arm64_sys_clone+0x2c/0x58
[ 3981.466455]  invoke_syscall+0xa8/0x138
[ 3981.466929]  el0_svc_common.constprop.0+0x4c/0x140
[ 3981.467532]  do_el0_svc+0x28/0x58
[ 3981.467951]  el0_svc+0x40/0x1d8
[ 3981.468349]  el0t_64_sync_handler+0xc0/0x108
[ 3981.468887]  el0t_64_sync+0x1b8/0x1c0
[ 3981.469350] Code: aa0203e6 b9402b80 f9405b84 8b204041 (f8604843) 
[ 3981.779123] Kernel panic - not syncing: Oops: Fatal exception

16:34, 10 min after that auto-reboot: hard freeze. No panic, no pstore dump, and the hardware watchdog didn’t fire. The journal stops at 16:34:28, but the box kept sending network requests for about 3 more minutes, so storage writes seem to have stopped first. It needed a power-button hold. The NVMe SMART and error logs are clean.
After that forced power-off, the GPU stayed at about 700 MHz under load (about 10 W, no clocks-event reason). A clean poweroff didn’t fix it; unplugging AC for 60 s did (back to 2340 MHz). This may just be the forced power-off.
18:20:25, 19 min after the AC-drain boot: GPU firmware fault. Two llama.cpp servers were loaded, and an embedding batch had finished about 8 s before. Then FECS ucode fatal, Reset Required, gnome-shell channel timeouts, a frozen desktop, and every new allocation failing with NV_ERR_RESET_REQUIRED until a reboot:

NVRM: Xid (PCI:000f:01:00): 13, Graphics FECS Exception: UCODE Fatal Error
NVRM: Xid (PCI:000f:01:00): 13, pid=4197, name=llama-server, channel 0x00000002   (x16)
NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57   (x16)
NVRM: Xid (PCI:000f:01:00): 154, GPU recovery action changed from 0x0 (None) to 0x1 (GPU Reset Required)
NVRM: Xid (PCI:000f:01:00): 38, pid=16630, name=gnome-shell, channel 0x00000017 0000ce97 00000000 00000000
NVRM: Xid (PCI:000f:01:00): 109, pid=16630, name=gnome-shell, channel 0x00000017, errorString CTX SWITCH TIMEOUT, Info 0x94007   (x7)
NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7   (x143)
NVRM: Xid (PCI:000f:01:00): 8, pid=16630, name=gnome-shell, channel 0x00000017   (x2)

For completeness: on Oct 26, 1019 had one Oops (a NULL deref in mstflint_access, 12 min after a ConnectX-7 AER fatal error on both root ports). It looks like a different bug.

Next: I’m booting 7.0.0-1019 now and will report whether it holds with the monitor attached. I have an nvidia-bug-report taken on 1024, the pstore dump, and the kernel log of each failing boot. I can send them privately, since they include serial numbers and network details.

Kirk_SparkX4 · October 30, 2026, 7:56pm

Reverting back to 1019 has resolved the issue.

elsaco · October 30, 2026, 8:01pm

Here’s a longer output when the desktop goes dark. SSH keeps working:

[ 1030.013029] nvidia 000f:01:00.0: Using 40-bit DMA addresses
[ 1862.372222] NVRM: GPU at PCI:000f:01:00: GPU-a19f88f3-1499-32bb-8191-c7cc72957d33
[ 1862.372259] NVRM: Xid (PCI:000f:01:00): 13, Graphics FECS Exception: UCODE Fatal Error
[ 1862.373443] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x00000006
[ 1862.376479] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.378234] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x00000007
[ 1862.381268] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.382788] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x00000008
[ 1862.385816] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.387350] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x00000009
[ 1862.390381] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.391764] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x0000000a
[ 1862.394827] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.396275] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x0000000b
[ 1862.399327] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.400775] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x0000000c
[ 1862.403847] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.405264] NVRM: Xid (PCI:000f:01:00): 13, pid=2731, name=gnome-remote-de, channel 0x0000000d
[ 1862.408348] NVRM: _kgspProcessRpcEvent: Failed to process received event 0x1004 (RC_TRIGGERED) from GPU0: status=0x57
[ 1862.409953] NVRM: Xid (PCI:000f:01:00): 154, GPU recovery action changed from 0x0 (None) to 0x1 (GPU Reset Required)
[ 1862.974060] NVRM: Xid (PCI:000f:01:00): 38, pid=2802, name=gnome-shell, channel 0x00000004 0000ce97 00000000 00000000
[ 1866.668761] NVRM: Xid (PCI:000f:01:00): 109, pid=2802, name=gnome-shell, channel 0x00000004, errorString CTX SWITCH TIMEOUT, Info 0x34001

[ 1871.970161] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1880.424933] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1889.378160] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1897.570162] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1905.762159] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1909.281726] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00022; hParent=0xcef80000; hObject=0xbeef0100; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1909.281739] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1909.281748] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1909.292554] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00022; hParent=0xcef80000; hObject=0xbeef0100; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1909.292562] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1909.292568] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1909.323887] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00062; hParent=0xcef80000; hObject=0xbeef0100; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1909.323903] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1909.323913] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1909.335977] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00062; hParent=0xcef80000; hObject=0xbeef0100; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1909.335991] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1909.335999] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1913.955159] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1922.383670] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1931.362158] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1939.554158] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1947.746158] NVRM: krcWatchdog_IMPL: RC watchdog: GPU is probably locked!  Notify Timeout Seconds: 7
[ 1960.171632] rfkill: input handler enabled
[ 1965.342434] NVRM: nvAssertOkFailedNoLog: Assertion failed: Requested object not found [NV_ERR_OBJECT_NOT_FOUND] (0x00000057) returned from pRmApi->Control( pRmApi, hClient, hDevice, NV0080_CTRL_CMD_INTERNAL_MEMSYS_SET_ZBC_REFERENCED, &params, sizeof(params)) @ mem_mgr_gm107.c:283
[ 1965.357274] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.359517] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.361627] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.363769] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.365978] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.368168] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.370321] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1965.372532] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from pRmApi->Control(pRmApi, RES_GET_CLIENT_HANDLE(pKernelChannel), RES_GET_HANDLE(pKernelChannel), NVA06F_CTRL_CMD_STOP_CHANNEL, &stopChannelParams, sizeof(stopChannelParams)) @ nv_gpu_ops.c:10961
[ 1975.097166] Unable to handle kernel paging request at virtual address 00027ffd80016bb0
[ 1975.097176] Mem abort info:
[ 1975.097180]   ESR = 0x0000000096000004
[ 1975.097182]   EC = 0x25: DABT (current EL), IL = 32 bits
[ 1975.097183]   SET = 0, FnV = 0
[ 1975.097185]   EA = 0, S1PTW = 0
[ 1975.097186]   FSC = 0x04: level 0 translation fault
[ 1975.097187] Data abort info:
[ 1975.097188]   ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
[ 1975.097189]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[ 1975.097190]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[ 1975.097192] [00027ffd80016bb0] address between user and kernel address ranges
[ 1975.097194] Internal error: Oops: 0000000096000004 [#1]  SMP
[ 1975.105024] Modules linked in: aes_ce_ccm xt_conntrack xt_MASQUERADE xfrm_user xfrm_algo xt_set ip_set nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 snd_seq_dummy snd_hrtimer xt_addrtype nft_compat nf_tables overlay nvidia_uvm(O) nvidia_drm(O) nvidia_modeset(O) qrtr binfmt_misc nls_iso8859_1 mt7925e mt7925_common rpcrdma mt792x_lib mt76_connac_lib snd_hda_codec_tegrahdmi sunrpc mt76 snd_hda_codec_hdmi airoha_eth airoha_npu rdma_ucm mac80211 nvidia(O) ib_iser libiscsi snd_seq_midi sbsa_gwdt snd_hda_acpi snd_seq_midi_event scsi_transport_iscsi cfg80211 arm_smmuv3_pmu arm_spe_pmu snd_hda_codec snd_rawmidi ib_umad dsa_core snd_hwdep rdma_cm hsr snd_hda_core ib_ipoib iw_cm snd_seq bridge snd_pcm video stp snd_seq_device drm_ttm_helper llc cppc_cpufreq libarc4 ttm ib_cm snd_timer snd snd_sof_mt8901 soundcore mtk_pcie_hotplug acpi_tad mlx5_fwctl fwctl input_leds joydev sch_fq_codel mstflint_access(O) dm_multipath ipmi_devintf ipmi_msghandler nvme_fabrics efi_pstore nfnetlink dmi_sysfs ip_tables x_tables
[ 1975.105080]  autofs4 btrfs libblake2b raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor xor_neon raid6_pq raid1 raid0 linear dm_mirror dm_region_hash dm_log mlx5_ib ib_uverbs macsec ib_core hid_generic usbhid hid ghash_ce sm4_ce_gcm sm4_ce_ccm mlx5_core sm4_ce sm4_ce_cipher nvme sm4 arm_smccc_trng sm3_ce nvme_core mlxfw psample nvme_keyring r8127 nvme_auth tls hkdf pci_hyperv_intf xhci_plat_hcd spi_mt65xx aes_neon_bs aes_neon_blk aes_ce_blk
[ 1975.121511] CPU: 5 UID: 1000 PID: 2876 Comm: cuda00008800023 Tainted: G           O        7.0.0-1024-nvidia #25~24.04.1-Ubuntu PREEMPT(lazy) 
[ 1975.123113] Tainted: [O]=OOT_MODULE
[ 1975.123552] Hardware name: NVIDIA NVIDIA_DGX_Spark/P4242, BIOS 5.36_0ACUM028 09/04/2026
[ 1975.124556] pstate: 81400009 (Nzcv daif +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
[ 1975.125431] pc : free_next_available_root_chunk+0xc4/0x1b8 [nvidia_uvm]
[ 1975.126282] lr : free_next_available_root_chunk+0x48/0x1b8 [nvidia_uvm]
[ 1975.127123] sp : ffff80008a1e39b0
[ 1975.127542] x29: ffff80008a1e39b0 x28: ffffaa27497e9668 x27: ffff80008a1e50d0
[ 1975.128438] x26: 0000000000001a08 x25: bfff0000916e1a08 x24: 0000000000000000
[ 1975.129335] x23: 0000000000000000 x22: bfff0000916e1af0 x21: ffff0000916e1b48
[ 1975.130231] x20: bfff0000916e1a08 x19: ffff0000916e1b38 x18: ffff8000bafc3090
[ 1975.131127] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 1975.132025] x14: 0000000000000000 x13: ffff55f7d4109000 x12: 0000000000000000
[ 1975.132922] x11: 00000000000000c0 x10: 0bce5b1259b087cc x9 : ffffaa2749764300
[ 1975.133818] x8 : ffff000093f56588 x7 : 0000000000000000 x6 : 0000000000000000
[ 1975.134714] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000000
[ 1975.135610] x2 : 0000000000000010 x1 : 00027ffd80016b80 x0 : 000000000000000d
[ 1975.136507] Call trace:
[ 1975.136818]  free_next_available_root_chunk+0xc4/0x1b8 [nvidia_uvm] (P)
[ 1975.137658]  uvm_pmm_gpu_deinit+0x7c/0x148 [nvidia_uvm]
[ 1975.138324]  remove_gpu+0x2d8/0x5f0 [nvidia_uvm]
[ 1975.138917]  uvm_gpu_release_locked+0x9c/0x160 [nvidia_uvm]
[ 1975.139627]  uvm_va_space_destroy+0x550/0x798 [nvidia_uvm]
[ 1975.140325]  uvm_release.isra.0+0x78/0x1b0 [nvidia_uvm]
[ 1975.140992]  uvm_release_entry.part.0.isra.0+0xd8/0x108 [nvidia_uvm]
[ 1975.141797]  uvm_release_entry+0x54/0x80 [nvidia_uvm]
[ 1975.142441]  __fput+0xe4/0x330
[ 1975.142843]  ____fput+0x20/0x48
[ 1975.143241]  task_work_run+0x80/0x100
[ 1975.143706]  do_exit+0x1e0/0x440
[ 1975.144122]  do_group_exit+0x40/0xa8
[ 1975.144572]  get_signal+0xa60/0xab0
[ 1975.145013]  arch_do_signal_or_restart+0xa4/0x350
[ 1975.145606]  exit_to_user_mode_loop+0x184/0x268
[ 1975.146182]  el0_svc+0x1a8/0x1d8
[ 1975.146596]  el0t_64_sync_handler+0xc0/0x108
[ 1975.147134]  el0t_64_sync+0x1b8/0x1c0
[ 1975.147598] Code: 8b011041 53093400 7100541f 54000660 (f9401822) 
[ 1975.148367] ---[ end trace 0000000000000000 ]---
[ 1975.400687] note: cuda00008800023[2876] exited with preempt_count 1
[ 1975.400689] Fixing recursive fault but reboot is needed!
[ 1975.860348] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00064; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1975.860359] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1975.860367] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1977.007685] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00067; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1977.007698] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1977.007706] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1978.016763] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d0006a; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1978.016776] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1978.016784] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1979.051754] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d0006d; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1979.051766] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1979.051774] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1980.063697] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00070; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1980.063712] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1980.063720] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1981.069707] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00073; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1981.069721] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1981.069729] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1993.237137] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00076; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1993.237151] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1993.237160] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1994.379890] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00079; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1994.379902] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1994.379910] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1995.394658] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d0007c; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1995.394671] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1995.394679] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1996.411837] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d0007f; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1996.411851] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1996.411858] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970
[ 1997.435089] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00082; hParent=0x01000001; hObject=0x01000013; hClass=0x0000ca6f; paramsSize=0x00000170; paramsStatus=0x00000062; status=0x00000062
[ 1997.435103] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from status @ kernel_channel.c:2871
[ 1997.435110] NVRM: nvAssertOkFailedNoLog: Assertion failed: Reset required [NV_ERR_RESET_REQUIRED] (0x00000062) returned from _kchannelSendChannelAllocRpc(pKernelChannel, pChannelGpfifoParams, pKernelChannelGroup, bFullSriov) @ kernel_channel.c:970

ChatGTP summary:

This trace shows a GPU firmware-level failure followed by an unsuccessful GPU recovery, and eventually a kernel crash inside NVIDIA's Unified Virtual Memory (UVM) driver.

dengfa.cheng · October 30, 2026, 11:32pm

Thanks - below are the two outputs, the failure-side data, and one more
display data point.

Caveat on the two commands: they were collected on the known-good kernel
7.0.0-1019-nvidia, because the unit is pinned back to 1019 to stay usable.
nvidia-smi cannot be collected on 1024 in this state.

$ uname -a
Linux spark-7626 7.0.0-1019-nvidia #19~24.04.2-Ubuntu SMP PREEMPT_DYNAMIC
Sun Sep 6 18:35:11 UTC 2 aarch64 aarch64 aarch64 GNU/Linux

Failing boot (7.0.0-1024-nvidia)
I exported the full journal of the last 1024 boot - attached, gzipped
(dgx-1024-boot-20261030.log.gz). Precise summary:

boot window 18:21:00 → 18:21:47 local
no panic, no Oops, no BUG, no Xid anywhere in that boot
“[drm] Initialized nvidia-drm 0.0.0 … minor 1” - driver initialises fine
gdm starts and session 2 of user dfc is created
the session then dies: logging in, or switching VT with Ctrl+Alt+F3,
drops straight back to the gdm login screen
that boot ends with “Power key pressed short” / “Powering off” in
systemd-logind - I powered it off manually because the desktop was
unusable. So this log does not show a crash-triggered reset; the resets in
earlier boots come with “watchdog0: watchdog did not stop!” and the SBSA
watchdog.

Display data point - this lines up with @kuwii
Single Sony TV over HDMI at 3840x2160. No second display, nothing on USB-C.
So the interface type matches what @kuwii reports as the trigger (HDMI/TMDS),
though here it presents as a dead gdm session plus watchdog resets rather
than an immediate hard reset. @jk.chik’s unit appears to be the USB-C/DP
case instead.

HDMI direct to the Spark’s HDMI port → Sony TV, 3840x2160
single display only
keyboard and mouse behind a USB hub

Current state
Pinned to 7.0.0-1019-nvidia (10 s GRUB menu), with these held:
linux-nvidia-hwe-24.04, linux-image-nvidia-hwe-24.04,
linux-headers-nvidia-hwe-24.04, linux-tools-nvidia-hwe-24.04,
linux-modules-nvidia-580-open-nvidia-hwe-24.04,
linux-modules-nvidia-fs-nvidia-hwe-24.04

If it helps, I can run these on 1024:

headless (no display attached) - do the reboots still happen?
HDMI only - does unplugging HDMI stop the resets?
nvidia-bug-report.sh during the first minutes of a 1024 boot.
Each run is short, and the unit falls back to 7.0.0-1019-nvidia on the next
reboot, so it is safe to try. Just say go.

dengfa.cheng · October 30, 2026, 11:40pm

dgx-1024-boot-20261030.log.gz (78.4 KB) I exported the full journal of the last 1024 boot - attached, gzipped:
dgx-1024-boot-20261030.log.gz

Topic		Replies	Views
`sudo reboot now` and Spark doesn't come back.. started after installing updates today DGX Spark / GB10	11	726	October 31, 2026
Random reboots and 00 screen DGX Spark / GB10 reboot	46	2422	April 20, 2026
DGX Spark constantly shut down after first set up DGX Spark / GB10	8	783	January 11, 2026
DGX spark keeps rebooting every 20-30 minutes DGX Spark / GB10 boot	6	1858	March 19, 2026
DGX Spark reboots every ~20 minutes DGX Spark / GB10	11	850	September 3, 2026
After the latest update of DGX SPark it's roasting like a hell in stale DGX Spark / GB10 dgx-spark-issue	7	1135	August 4, 2026
Spark abruptly shuts down DGX Spark / GB10 dgx-spark-issue	47	4187	October 9, 2026
[Root Cause Analysis] DGX Spark driver failure — kernel 6.17.0-1008-nvidia aarch64 panics cause DOE mailbox failure (pstore evidence) DGX Spark / GB10 pcie , boot , kernel , ota , driver	3	930	April 17, 2026
System degraded, GPU inoperable DGX Spark / GB10 cuda , dgx-spark-issue	6	1170	August 26, 2026
BOOT LOOP ISSUE RESOLVED DGX Spark / GB10	71	5505	November 17, 2025

DGX Spark repeatedly rebooting after most recent update

Related topics