Can't unload nvidia-drm module: Freeze with "GspRmAlloc failed" error in dmesg

Graphics / Linux Linux
1

When I’m trying to unload nvidia modules with either rmmod or modprobe -r commands, nvidia-drm unload freezes with the following errors showing up in dmesg:

[  109.204277] NVRM: x86emu_do_int: x86emu: int $0 (eax = 00015d64)
[  109.204281] NVRM: unixCallVideoBIOS: int10h(4f02, 0003) vesa call failed! (15d64, 2008)
[  113.222351] NVRM: rpcRmApiAlloc_GSP: GspRmAlloc failed: hClient=0xc1d00001; hParent=0x00010011; hObject=0x00010016; hClass=0x0000ca7d; paramsSize=0x00000028; paramsStatus=0x00000065; status=0x00000065
[  113.222369] nvidia-modeset: ERROR: GPU:0: Display engine push buffer channel allocation failed: 0x65 (Call timed out [NV_ERR_TIMEOUT])
[  113.222442] nvidia-modeset: ERROR: GPU:0: Failed to allocate display engine core DMA push buffer

RTX 5080, running Gentoo with kernel version 6.15.0 and open driver version 575.57.08, had the same issue with 570.153.02.
What might be the cause here, and how can I debug this further?

Thanks for your time.

2

have you verified with nvidia-smi that no processes were using the GPU before attempting to remove the module? if so, you can try attaching a bug report file for Nvidia engineers to hopefully have a look.

3

Yes, I checked that nothing is using GPU & nvidia modules using nvidia-smi and lsmod (I’m connected from another machine via SSH when trying to unload the module).

Running nvidia-bug-report.sh after the rmmod freeze causes the script to freeze as well, leaving nothing except log header in nvidia-bug-report.log.gz
I’m attaching the file created by running the script after the boot, in case it helps.
nvidia-bug-report.log.gz (444.9 KB)