Xid 31 MMU fault in gnome-shell and ~96 GiB vmalloc request from nvidia.ko when an application sets a custom cursor

Graphics / Linux Linux
1

nvidia-bug-report.log.gz (1.6 MB)

kernel-boot-incident1.log (728.0 KB)

kernel-boot-incident2.log (189.6 KB)

session-incident3-hang.log (12.2 KB)

Two desktop freezes with the same trigger, both traceable to a single allocation request from the driver that the kernel cannot satisfy. Forcing mutter to draw the cursor in software stops it.

Summary

Selecting QGIS’s Identify tool and moving the pointer onto the map canvas froze the whole desktop, twice out of two attempts. The kernel logged Xid 31 (MMU fault) against gnome-shell, then a vmalloc request of about 96 GiB coming from os_alloc_mem inside nvidia.ko, and gnome-shell died with SIGSEGV. Every other GPU client lost its channel at the same moment. A third freeze on the same trigger produced no Xid at all: gnome-shell simply stopped responding.

The requested allocation size, 103079135288 bytes, is 96 GiB on a machine with 30 GiB of RAM and an 11 GiB card, so it looks like a corrupted or uninitialised size rather than a genuine request. The two incidents differ by 2256 bytes (103079135288 and 103079133032), both a little above 0x1800000000.

Setting MUTTER_DEBUG_DISABLE_HW_CURSORS=1 has stopped it: the same steps ran cleanly afterwards, with 33 hours of normal use and no Xid of any kind. That points at the hardware cursor path.

System

  • GPU: GeForce GTX 1080 Ti (Pascal), single GPU, no integrated graphics in use
  • Driver: 580.178.04 (Ubuntu package nvidia-driver-580)
  • Kernel: 7.0.0-31-generic (Ubuntu 26.04.1 LTS)
  • Compositor: GNOME Shell 50.1 / mutter 50.1, Wayland session
  • Motherboard: ASUS ROG STRIX B550-A GAMING, BIOS 1205
  • Application: QGIS 4.2.2 (Qt 6.10.2), running as a native Wayland client
  • Cursor: Yaru theme, cursor-size 48
  • mutter experimental features: scale-monitor-framebuffer, xwayland-native-scaling
  • Monitors: four, mixed scale factors
    • DP-2, AOC AG352UCG 3440x1440, scale 1.0, primary
    • DP-1, LG 4K 3840x2160, scale 1.5
    • DP-3, LG 4K 3840x2160, scale 1.5
    • HDMI-1, LG 4K 3840x2160, scale 1.0

Reproduction

  1. Log into a GNOME Wayland session with the monitor layout above.
  2. Open QGIS with any project that has a vector or raster layer.
  3. Select the Identify (info) tool from the toolbar.
  4. Move the pointer onto the map canvas, which switches to QGIS’s own cursor.

The desktop froze at this step on 14 September at 16:37 and again at 16:46, and a third time on 16 September at 11:14. No other application has triggered it so far.

Incident 1, 14 September 16:37:07

NVRM: Xid (PCI:0000:08:00): 31, pid=2319, name=gnome-shell, channel 0x00000008,
      intr 10000000. MMU Fault: ENGINE GRAPHICS GPC0 GPCCLIENT_T1_3
      faulted @ 0x0_80000000. Fault is of type FAULT_PDE ACCESS_TYPE_READ
NVRM: Xid (PCI:0000:08:00): 32, pid=2899, name=Renderer, channel 0x00000040 intr 00040000
NVRM: Xid (PCI:0000:08:00): 32, pid=167158, name=papers, channel 0x05000051 intr1 00000008
gnome-shell: vmalloc error: size 103079135288, exceeds total pages,
      mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0
CPU: 7 UID: 1000 PID: 2319 Comm: gnome-shell Tainted: P OE 7.0.0-31-generic

Call trace, trimmed to the relevant frames:

warn_alloc+0x172/0x1a0
__vmalloc_node_range_noprof+0x206/0x210
__vmalloc_node_noprof+0x4b/0x70
__vmalloc_noprof+0x3c/0x90
os_alloc_mem+0xab/0x140 [nvidia]
_nv015310rm+0x34/0x50 [nvidia]
...
rm_ioctl+0x7f/0x3d0 [nvidia]
nvidia_unlocked_ioctl+0x441/0x940 [nvidia]
__x64_sys_ioctl+0xa3/0x100

gnome-shell then logged “GNOME Shell crashed with signal 11”. Memory was not short: 18.7 GB of swap free, gnome-shell resident at 1.5 GB.

Incident 2, 14 September 16:46:34

Same trigger, nine minutes later, on a fresh session:

NVRM: Xid (PCI:0000:08:00): 32, pid=2440, name=gnome-shell, channel 0x00000008 intr 00040000
NVRM: Xid (PCI:0000:08:00): 32, pid=2998, name=Renderer, channel 0x00000030 intr1 00000008
gnome-shell: vmalloc error: size 103079133032, exceeds total pages, ...
NVRM: Xid (PCI:0000:08:00): 31, pid=3170, name=Xwayland, channel 0x00000010,
      intr 10000000. MMU Fault: ENGINE HOST0 HUBCLIENT_HOST faulted @ 0x0_04000000.
      Fault is of type FAULT_PTE ACCESS_TYPE_READ
NVRM: Xid (PCI:0000:08:00): 31, pid=1521, name=modprobe, channel 0x05000003,
      intr 10000000. MMU Fault: ENGINE CE2 HUBCLIENT_CE1 faulted @ 0x1_01000000.
      Fault is of type FAULT_PDE ACCESS_TYPE_WRITE
NVRM: Xid (PCI:0000:08:00): 154, GPU recovery action changed from 0x0 (None)
      to 0x2 (Node Reboot Required)

The modprobe faults come from the driver being reloaded during recovery. The GPU ended in a state the driver itself declared unrecoverable without a reboot.

Incident 3, 16 September 11:14

Same trigger, but a different failure. No Xid, no vmalloc error, no crash. The pointer became laggy (libinput logged “event processing lagging behind by 87ms”), and 23 seconds later gnome-shell stopped responding to the accessibility bus. The session had to be reset. I mention it because a hang and a fault may share a cause here, and because it means the absence of an Xid does not rule this out.

Workaround

MUTTER_DEBUG_DISABLE_HW_CURSORS=1 in ~/.config/environment.d/, applied at the next login. With it set, the same steps run cleanly and the pointer behaves normally. I sampled gnome-shell’s state every two seconds through the test and saw no stall, and no Xid has appeared in 33 hours of use since.

Related but not the same

Attached

nvidia-bug-report.log.gz, plus the full kernel log for both crash boots.