For me this is the first driver in years for which resume is completely broken for me:
kernel: NVRM: gpuWaitForGfwBootComplete_TU102: failed to wait for GFW_BOOT: (progress 0x3)
kernel: NVRM: kgspWaitForGfwBootOk_TU102: failed to wait for GFW boot complete: 0x55 VBIOS version 95.04.69.40.36
kernel: NVRM: kgspWaitForGfwBootOk_TU102: (the GPU may be in a bad state and may need to be reset)
kernel: NVRM: _kgspLogXid119: ********************************* GSP Timeout **********************************
kernel: NVRM: _kgspLogXid119: Note: Please also check logs above.
kernel: NVRM: GPU at PCI:0000:09:00: GPU-c3ad889c-6ef1-2ea2-2635-079a8d083975
kernel: NVRM: Xid (PCI:0000:09:00): 119, pid=55768, name=nvidia-sleep.sh, Timeout after 6s of waiting for RPC response from GPU0 GSP! Expected function 76 (GSP_RM_CONTROL) (0x20801117 0x1).
kernel: NVRM: GPU0 GSP RPC buffer contains function 76 (GSP_RM_CONTROL) and data 0x0000000020801117 0x0000000000000001.
kernel: NVRM: GPU0 RPC history (CPU -> GSP):
kernel: NVRM: entry function data0 data1 ts_start ts_end duration actively_polling
kernel: NVRM: 0 76 GSP_RM_CONTROL 0x0000000020801117 0x0000000000000001 0x00062f0b794056b3 0x0000000000000000 y
kernel: NVRM: -1 47 UNLOADING_GUEST_DRIVE 0x0000000000000000 0x0000000000000000 0x00062f09365e4005 0x00062f09365f299e 59801us
kernel: NVRM: -2 10 FREE 0x00000000c1e00b62 0x0000000000000000 0x00062f09365e3ea6 0x00062f09365e3fb6 272us
kernel: NVRM: -3 10 FREE 0x000000000000000a 0x0000000000000000 0x00062f09365e3d3e 0x00062f09365e3ea4 358us
kernel: NVRM: -4 10 FREE 0x000000000000000b 0x0000000000000000 0x00062f09365e3c21 0x00062f09365e3cc8 167us
kernel: NVRM: -5 10 FREE 0x0000000000000006 0x0000000000000000 0x00062f09365e39ec 0x00062f09365e3c1a 558us
kernel: NVRM: -6 10 FREE 0x0000000000000002 0x0000000000000000 0x00062f09365e345e 0x00062f09365e39a1 1347us
kernel: NVRM: -7 10 FREE 0x0000000000000005 0x0000000000000000 0x00062f09365e30b9 0x00062f09365e3459 928us
kernel: NVRM: GPU0 RPC event history (CPU <- GSP):
kernel: NVRM: entry function data0 data1 ts_start ts_end duration during_incomplete_rpc
kernel: NVRM: 0 4108 UCODE_LIBOS_PRINT 0x0000000000000000 0x0000000000000000 0x00062f09365eb072 0x00062f09365eb072
kernel: NVRM: -1 4128 GSP_POST_NOCAT_RECORD 0x0000000000000002 0x0000000000000028 0x00062f09365e5d70 0x00062f09365e5d72 2us
kernel: NVRM: -2 4111 PERF_BRIDGELESS_INFO_ 0x0000000000000000 0x0000000000000000 0x00062f09365c4bd6 0x00062f09365c4bd7 1us
kernel: NVRM: -3 4128 GSP_POST_NOCAT_RECORD 0x0000000000000005 0x00000285017d990c 0x00062f09365c1d34 0x00062f09365c1d35 1us
kernel: NVRM: -4 4099 POST_EVENT 0x0000000000000000 0x0000000000000000 0x00062f09365c08c2 0x00062f09365c08c5 3us
kernel: NVRM: -5 4128 GSP_POST_NOCAT_RECORD 0x0000000000000005 0x00000285017d990c 0x00062f09365bcd2b 0x00062f09365bcd2c 1us
kernel: NVRM: -6 4099 POST_EVENT 0x0000000000000000 0x0000000000000000 0x00062f09365bb964 0x00062f09365bb966 2us
kernel: NVRM: -7 4099 POST_EVENT 0x0000000000000000 0x0000000000000000 0x00062f09365b9e4a 0x00062f09365b9e4c 2us
kernel: CPU: 13 UID: 0 PID: 55768 Comm: nvidia-sleep.sh Tainted: G O 6.13.4-zen3 #2
kernel: Tainted: [O]=OOT_MODULE
kernel: Hardware name: System manufacturer System Product Name/TUF GAMING X570-PLUS (WI-FI), BIOS 5013 03/22/2024
kernel: Call Trace:
kernel: <TASK>
kernel: dump_stack_lvl+0x4c/0x70
kernel: dump_stack+0x10/0x16
kernel: os_dump_stack+0x9/0x10 [nvidia]
kernel: _kgspRpcRecvPoll+0x54c/0x620 [nvidia]
kernel: _issueRpcAndWait+0x71/0x360 [nvidia]
kernel: rpcRmApiControl_GSP+0x67e/0xa10 [nvidia]
kernel: ? os_release_spinlock+0x15/0x20 [nvidia]
kernel: ? _tlsThreadEntryGet+0x82/0x90 [nvidia]
kernel: ? osGetCurrentThread+0x26/0x60 [nvidia]
kernel: rmresControl_Prologue_IMPL+0xd8/0x230 [nvidia]
kernel: resControl_IMPL+0xd9/0x1e0 [nvidia]
kernel: ? _tlsEntryAcquire+0x93/0xe0 [nvidia]
kernel: serverControl+0x48f/0x5c0 [nvidia]
kernel: _rmapiRmControl+0x4f2/0x840 [nvidia]
kernel: rmapiControlWithSecInfo+0x79/0x140 [nvidia]
kernel: rmapiControl+0x24/0x40 [nvidia]
kernel: rm_restart_user_channels+0x6b/0xc0 [nvidia]
kernel: nv_restore_user_channels+0x40/0x1d0 [nvidia]
kernel: nv_set_system_power_state+0xe6/0x490 [nvidia]
kernel: nv_procfs_write_suspend+0xe2/0x150 [nvidia]
kernel: proc_reg_write+0x59/0x90
kernel: vfs_write+0xd4/0x3d0
kernel: ? __x64_sys_fcntl+0x92/0xf0
kernel: ksys_write+0x56/0xd0
kernel: __x64_sys_write+0x14/0x20
kernel: x64_sys_call+0x28a/0x1d70
kernel: do_syscall_64+0x79/0x150
kernel: ? syscall_exit_to_user_mode+0x33/0x170
kernel: ? do_syscall_64+0x85/0x150
kernel: entry_SYSCALL_64_after_hwframe+0x6c/0x74
kernel: RIP: 0033:0x7fdddf751504
kernel: Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d c5 8b 10 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89
kernel: RSP: 002b:00007ffd8f06b418 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fdddf751504
kernel: RDX: 0000000000000007 RSI: 000056118bba3740 RDI: 0000000000000001
kernel: RBP: 00007ffd8f06b440 R08: 0000000000000410 R09: 0000000000000001
kernel: R10: 0000000000000004 R11: 0000000000000202 R12: 0000000000000007
kernel: R13: 000056118bba3740 R14: 00007fdddf8535c0 R15: 00007fdddf850e80
kernel: </TASK>
kernel: NVRM: _kgspLogXid119: ********************************************************************************
kernel: NVRM: _issueRpcAndWait: rpcRecvPoll timedout for fn 76!
kernel: ------------[ cut here ]------------
kernel: WARNING: CPU: 13 PID: 55768 at nvidia/nv.c:4291 nv_restore_user_channels+0x4e/0x1d0 [nvidia]
kernel: Modules linked in: uinput tun rfcomm snd_hrtimer nvidia_uvm(O) cmac algif_hash algif_skcipher af_alg msr nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nf_log_syslog vboxnetadp(O) vboxnetflt(O) nft_limit nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nfnetlink_log nft_log vboxdrv(O) bnep nf_tables libcrc32c nct6775 nct6775_core hwmon_vid input_leds hid_generic usbhid hid btusb btintel btbcm bluetooth ntfs3 nvidia_drm(O) nvidia_modeset(O) snd_hda_codec_realtek snd_hda_codec_generic snd_hda_scodec_component iwlmvm kvm_amd snd_hda_codec_hdmi ptp pps_core kvm nvidia(O) ee1004 mac80211 crct10dif_pclmul libarc4 crc32_pclmul led_class wmi_bmof snd_hda_intel snd_intel_dspcfg crc32c_intel polyval_clmulni polyval_generic sha512_ssse3 snd_hda_codec sha512_generic snd_hwdep snd_hda_core sha256_ssse3 snd_seq sha1_ssse3 snd_seq_device sr_mod backlight aesni_intel snd_pcm cdrom gf128mul crypto_simd ccp drm_client_lib cryptd drm_ttm_helper iwlwifi sha1_generic ttm snd_timer efi_pstore pcspkr snd
kernel: drm_kms_helper k10temp r8169 cfg80211 realtek i2c_piix4 mdio_devres xhci_pci rfkill libphy xhci_hcd 8250 8250_base wmi serial_base tpm_crb tpm_tis tpm_tis_core evdev fuse dm_mod nfnetlink efivarfs tpm libaescfb ecdh_generic ecc rng_core ipv6
kernel: CPU: 13 UID: 0 PID: 55768 Comm: nvidia-sleep.sh Tainted: G O 6.13.4-zen3 #2
kernel: Tainted: [O]=OOT_MODULE
kernel: Hardware name: System manufacturer System Product Name/TUF GAMING X570-PLUS (WI-FI), BIOS 5013 03/22/2024
kernel: RIP: 0010:nv_restore_user_channels+0x4e/0x1d0 [nvidia]
kernel: Code: 30 06 00 00 4c 89 f7 e8 60 94 09 ec f6 43 10 01 74 76 48 89 de 31 ff e8 d0 66 11 00 41 89 c7 85 c0 0f 84 3f 01 00 00 45 31 e4 <0f> 0b 49 81 c5 68 07 00 00 4c 89 ef e8 31 94 09 ec be 01 00 00 00
kernel: RSP: 0018:ffffb92345c27cf8 EFLAGS: 00010246
kernel: RAX: 0000000000000065 RBX: ffff996e02d05000 RCX: ffffb92345c27c78
kernel: RDX: 0000000000000000 RSI: 0000000000000282 RDI: ffffb92345c27c38
kernel: RBP: ffffb92345c27d20 R08: 0000000000000000 R09: ffffb92345c27ae4
kernel: R10: ffffffffc1339d80 R11: 0000000000000002 R12: 0000000000000000
kernel: R13: ffff996e02d05000 R14: ffff996e02d05630 R15: 0000000000000065
kernel: FS: 00007fdddf666740(0000) GS:ffff997cef140000(0000) knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: 00007f9e8a72d938 CR3: 000000013dcee000 CR4: 0000000000b50ef0
kernel: Call Trace:
kernel: <TASK>
kernel: ? show_regs.part.0+0x1d/0x30
kernel: ? show_regs.cold+0x8/0xd
kernel: ? __warn.cold+0x95/0xa3
kernel: ? nv_restore_user_channels+0x4e/0x1d0 [nvidia]
kernel: ? report_bug+0x101/0x150
kernel: ? handle_bug+0x5b/0x90
kernel: ? exc_invalid_op+0x18/0x70
kernel: ? asm_exc_invalid_op+0x1b/0x20
kernel: ? nv_restore_user_channels+0x4e/0x1d0 [nvidia]
kernel: ? nv_restore_user_channels+0x40/0x1d0 [nvidia]
kernel: nv_set_system_power_state+0xe6/0x490 [nvidia]
kernel: nv_procfs_write_suspend+0xe2/0x150 [nvidia]
kernel: proc_reg_write+0x59/0x90
kernel: vfs_write+0xd4/0x3d0
kernel: ? __x64_sys_fcntl+0x92/0xf0
kernel: ksys_write+0x56/0xd0
kernel: __x64_sys_write+0x14/0x20
kernel: x64_sys_call+0x28a/0x1d70
kernel: do_syscall_64+0x79/0x150
kernel: ? syscall_exit_to_user_mode+0x33/0x170
kernel: ? do_syscall_64+0x85/0x150
kernel: entry_SYSCALL_64_after_hwframe+0x6c/0x74
kernel: RIP: 0033:0x7fdddf751504
kernel: Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d c5 8b 10 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89
kernel: RSP: 002b:00007ffd8f06b418 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fdddf751504
kernel: RDX: 0000000000000007 RSI: 000056118bba3740 RDI: 0000000000000001
kernel: RBP: 00007ffd8f06b440 R08: 0000000000000410 R09: 0000000000000001
kernel: R10: 0000000000000004 R11: 0000000000000202 R12: 0000000000000007
kernel: R13: 000056118bba3740 R14: 00007fdddf8535c0 R15: 00007fdddf850e80
kernel: </TASK>
kernel: ---[ end trace 0000000000000000 ]---
kernel: ------------[ cut here ]------------
kernel: WARNING: CPU: 13 PID: 55768 at nvidia/nv.c:4513 nv_set_system_power_state+0x2e7/0x490 [nvidia]
kernel: Modules linked in: uinput tun rfcomm snd_hrtimer nvidia_uvm(O) cmac algif_hash algif_skcipher af_alg msr nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nf_log_syslog vboxnetadp(O) vboxnetflt(O) nft_limit nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nfnetlink_log nft_log vboxdrv(O) bnep nf_tables libcrc32c nct6775 nct6775_core hwmon_vid input_leds hid_generic usbhid hid btusb btintel btbcm bluetooth ntfs3 nvidia_drm(O) nvidia_modeset(O) snd_hda_codec_realtek snd_hda_codec_generic
snd_hda_scodec_component iwlmvm kvm_amd snd_hda_codec_hdmi ptp pps_core kvm nvidia(O) ee1004 mac80211 crct10dif_pclmul libarc4 crc32_pclmul led_class wmi_bmof snd_hda_intel snd_intel_dspcfg crc32c_intel polyval_clmulni polyval_generic sha512_ssse3 snd_hda_codec sha512_generic snd_hwdep snd_hda_core sha256_ssse3 snd_seq sha1_ssse3 snd_seq_device sr_mod backlight aesni_intel snd_pcm cdrom gf128mul crypto_simd ccp drm_client_lib cryptd drm_ttm_helper iwlwifi sha1_generic ttm snd_timer efi_pstore pcspkr snd
kernel: drm_kms_helper k10temp r8169 cfg80211 realtek i2c_piix4 mdio_devres xhci_pci rfkill libphy xhci_hcd 8250 8250_base wmi serial_base tpm_crb tpm_tis tpm_tis_core evdev fuse dm_mod nfnetlink efivarfs tpm libaescfb ecdh_generic ecc rng_core ipv6
kernel: CPU: 13 UID: 0 PID: 55768 Comm: nvidia-sleep.sh Tainted: G W O 6.13.4-zen3 #2
kernel: Tainted: [W]=WARN, [O]=OOT_MODULE
kernel: Hardware name: System manufacturer System Product Name/TUF GAMING X570-PLUS (WI-FI), BIOS 5013 03/22/2024
kernel: RIP: 0010:nv_set_system_power_state+0x2e7/0x490 [nvidia]
kernel: Code: 00 00 00 48 8b 70 78 48 8b 78 60 e8 43 cf ff ff 85 c0 0f 85 ea 00 00 00 4d 8b a4 24 58 06 00 00 4d 85 e4 75 d1 e9 c6 fd ff ff <0f> 0b e9 00 fe ff ff 48 8b 3d 83 70 3e 00 4c 89 fe e8 a3 a9 b0 eb
kernel: RSP: 0018:ffffb92345c27d30 EFLAGS: 00010206
kernel: RAX: 0000000000000065 RBX: 0000000000000002 RCX: ffffb92345c27c78
kernel: RDX: ffff996e02d05638 RSI: 0000000000000296 RDI: ffff996e02d05630
kernel: RBP: ffffb92345c27d68 R08: 0000000000000000 R09: ffffb92345c27ae4
kernel: R10: ffffffffc1339d80 R11: 0000000000000002 R12: ffff996e02d05000
kernel: R13: 0000000000000000 R14: ffffb92345c27e60 R15: ffff996e00dd6900
kernel: FS: 00007fdddf666740(0000) GS:ffff997cef140000(0000) knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: 00007f9e8a72d938 CR3: 000000013dcee000 CR4: 0000000000b50ef0
kernel: Call Trace:
kernel: <TASK>
kernel: ? show_regs.part.0+0x1d/0x30
kernel: ? show_regs.cold+0x8/0xd
kernel: ? __warn.cold+0x95/0xa3
kernel: ? nv_set_system_power_state+0x2e7/0x490 [nvidia]
kernel: ? report_bug+0x101/0x150
kernel: ? handle_bug+0x5b/0x90
kernel: ? exc_invalid_op+0x18/0x70
kernel: ? asm_exc_invalid_op+0x1b/0x20
kernel: ? nv_set_system_power_state+0x2e7/0x490 [nvidia]
kernel: nv_procfs_write_suspend+0xe2/0x150 [nvidia]
kernel: proc_reg_write+0x59/0x90
kernel: vfs_write+0xd4/0x3d0
kernel: ? __x64_sys_fcntl+0x92/0xf0
kernel: ksys_write+0x56/0xd0
kernel: __x64_sys_write+0x14/0x20
kernel: x64_sys_call+0x28a/0x1d70
kernel: do_syscall_64+0x79/0x150
kernel: ? syscall_exit_to_user_mode+0x33/0x170
kernel: ? do_syscall_64+0x85/0x150
kernel: entry_SYSCALL_64_after_hwframe+0x6c/0x74
kernel: RIP: 0033:0x7fdddf751504
kernel: Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d c5 8b 10 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89
kernel: RSP: 002b:00007ffd8f06b418 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
kernel: RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fdddf751504
kernel: RDX: 0000000000000007 RSI: 000056118bba3740 RDI: 0000000000000001
kernel: RBP: 00007ffd8f06b440 R08: 0000000000000410 R09: 0000000000000001
kernel: R10: 0000000000000004 R11: 0000000000000202 R12: 0000000000000007
kernel: R13: 000056118bba3740 R14: 00007fdddf8535c0 R15: 00007fdddf850e80
kernel: </TASK>
kernel: ---[ end trace 0000000000000000 ]---
OS: Linux 6.13.4 vanilla
GPU: 4070S
CPU: Ryzen 7 5800X
MB: ASUS TUF GAMING X570-PLUS (WI-FI)
I’m using an open source kernel module.