Black screen because of nvidia gsp firmware

Fedora 40 under 6.10.6 kernel and nvidia driver 555.58.02 stuck with black screen on boot.
NVIDIA RTX A5000 Laptop GPU

Driver installed from rpmfusion repository, secure boot is ON

Aug 22 15:36:28 fedora systemd[1]: Queuing reload/restart jobs for marked units…
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="xorg-x11-drv-nvidia-cuda-libs-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="nvidia-modprobe-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="xorg-x11-drv-nvidia-kmodsrc-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="nvidia-persistenced-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="egl-wayland-1.1.15-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="egl-gbm-2:1.1.2-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="opencl-filesystem-1.0-20.fc40.noarch" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="libglvnd-opengl-1:1.7.0-4.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="xorg-x11-drv-nvidia-libs-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="akmod-nvidia-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="nvidia-settings-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="xorg-x11-drv-nvidia-power-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="xorg-x11-drv-nvidia-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:28 fedora audit[4439]: SOFTWARE_UPDATE pid=4439 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="xorg-x11-drv-nvidia-cuda-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=1 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=fedora addr=? terminal=pts/0 res=success'
Aug 22 15:36:29 fedora systemd[1]: Starting packagekit.service - PackageKit Daemon...
Aug 22 15:36:29 fedora PackageKit[6124]: daemon start
Aug 22 15:36:29 fedora systemd[1]: Started packagekit.service - PackageKit Daemon.
Aug 22 15:36:29 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=packagekit comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 22 15:36:29 fedora audit[4436]: USER_END pid=4436 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:36:29 fedora audit[4436]: CRED_DISP pid=4436 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:setcred grantors=pam_env,pam_fprintd acct="root" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:36:29 fedora sudo[4436]: pam_unix(sudo:session): session closed for user root
Aug 22 15:36:32 fedora systemd[1]: man-db-cache-update.service: Deactivated successfully.
Aug 22 15:36:32 fedora systemd[1]: Finished man-db-cache-update.service.
Aug 22 15:36:32 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=man-db-cache-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 22 15:36:32 fedora audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=man-db-cache-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 22 15:36:32 fedora systemd[1]: man-db-cache-update.service: Consumed 1.913s CPU time, 10.4M memory peak, 0B memory swap peak.
Aug 22 15:36:32 fedora systemd[1]: run-r746a3efe73f14b5296c5992026858ce4.service: Deactivated successfully.
Aug 22 15:36:32 fedora audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=run-r746a3efe73f14b5296c5992026858ce4 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 22 15:37:03 fedora chronyd[3876]: Selected source 212.7.1.131 (2.fedora.pool.ntp.org)
Aug 22 15:37:09 fedora systemd[4286]: Starting grub-boot-success.service - Mark boot as successful...
Aug 22 15:37:09 fedora systemd[4286]: Finished grub-boot-success.service - Mark boot as successful.
Aug 22 15:37:17 fedora audit[17737]: USER_ACCT pid=17737 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="mainuser" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:37:17 fedora sudo[17737]: mainuser : TTY=tty3 ; PWD=/home/mainuser ; USER=root ; COMMAND=/usr/bin/journalctl --vacuum-time=1h
Aug 22 15:37:17 fedora audit[17737]: USER_CMD pid=17737 uid=1000 auid=1000 ses=2 subj=kernel msg='cwd="/home/mainuser" cmd=6A6F75726E616C63746C202D2D76616375756D2D74696D653D3168 exe="/usr/bin/sudo" terminal=tty3 res=success'
Aug 22 15:37:17 fedora audit[17737]: CRED_REFR pid=17737 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:setcred grantors=pam_env,pam_fprintd acct="root" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:37:17 fedora sudo[17737]: pam_unix(sudo:session): session opened for user root(uid=0) by mainuser(uid=1000)
Aug 22 15:37:17 fedora audit[17737]: USER_START pid=17737 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:37:17 fedora rsyslogd[3716]: imjournal: journal files changed, reloading...  [v8.2312.0-1.fc40 try https://www.rsyslog.com/e/0 ]
Aug 22 15:37:17 fedora sudo[17737]: pam_unix(sudo:session): session closed for user root
Aug 22 15:37:17 fedora audit[17737]: USER_END pid=17737 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:37:17 fedora audit[17737]: CRED_DISP pid=17737 uid=1000 auid=1000 ses=2 subj=kernel msg='op=PAM:setcred grantors=pam_env,pam_fprintd acct="root" exe="/usr/bin/sudo" hostname=fedora addr=? terminal=/dev/tty3 res=success'
Aug 22 15:37:34 fedora runuser[4784]: pam_unix(runuser:session): session closed for user akmods
Aug 22 15:37:34 fedora audit[4784]: USER_END pid=4784 uid=0 auid=1000 ses=2 subj=kernel msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_unix acct="akmods" exe="/usr/sbin/runuser" hostname=? addr=? terminal=? res=success'
Aug 22 15:37:34 fedora audit[4784]: CRED_DISP pid=4784 uid=0 auid=1000 ses=2 subj=kernel msg='op=PAM:setcred grantors=pam_rootok acct="akmods" exe="/usr/sbin/runuser" hostname=? addr=? terminal=? res=success'
Aug 22 15:37:43 fedora audit[17798]: SOFTWARE_UPDATE pid=17798 uid=0 auid=1000 ses=2 subj=kernel msg='op=install sw="kmod-nvidia-6.10.5-200.fc40.x86_64-3:555.58.02-1.fc40.x86_64" sw_type=rpm key_enforce=0 gpg_res=0 root_dir="/" comm="dnf" exe="/usr/bin/python3.12" hostname=? addr=? terminal=? res=success'
Aug 22 15:37:44 fedora kernel: nvidia: loading out-of-tree module taints kernel.
Aug 22 15:37:44 fedora kernel: nvidia: module license 'NVIDIA' taints kernel.
Aug 22 15:37:44 fedora kernel: Disabling lock debugging due to kernel taint
Aug 22 15:37:44 fedora kernel: nvidia: module license taints kernel.
Aug 22 15:37:45 fedora kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 509
Aug 22 15:37:45 fedora kernel: 
Aug 22 15:37:45 fedora kernel: nvidia 0000:01:00.0: enabling device (0006 -> 0007)
Aug 22 15:37:45 fedora kernel: nvidia 0000:01:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none
Aug 22 15:37:45 fedora kernel: NVRM: loading NVIDIA UNIX x86_64 Kernel Module  555.58.02  Tue Jun 25 01:39:15 UTC 2024
Aug 22 15:37:45 fedora systemd[1]: nvidia-fallback.service - Fallback to nouveau as nvidia did not load was skipped because of an unmet condition check (ConditionPathExists=!/sys/module/nvidia).
Aug 22 15:37:45 fedora kernel: nvidia_uvm: module uses symbols nvUvmInterfaceDisableAccessCntr from proprietary module nvidia, inheriting taint.
Aug 22 15:37:45 fedora kernel: nvidia-uvm: Loaded the UVM driver, major device number 507.
Aug 22 15:37:45 fedora kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  555.58.02  Tue Jun 25 01:10:21 UTC 2024
Aug 22 15:37:45 fedora kernel: [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
Aug 22 15:37:45 fedora kernel: ACPI Warning: \_SB.NPCF._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20240322/nsarguments-61)
Aug 22 15:37:45 fedora kernel: ACPI Warning: \_SB.PC00.PEG1.PEGP._DSM: Argument #4 type mismatch - Found [Buffer], ACPI requires [Package] (20240322/nsarguments-61)
Aug 22 15:37:45 fedora kernel: NVRM: GPU at PCI:0000:01:00: GPU-6985371f-62d8-eac3-b0db-d13445532ca8
Aug 22 15:37:45 fedora kernel: NVRM: Xid (PCI:0000:01:00): 120, pid='<unknown>', name=<unknown>, GSP task exception: load access page fault (cause:0xd) @ pc:0x19981c0, partition:2#4, task:3
Aug 22 15:37:45 fedora kernel: NVRM:     Reported by libos partition:2#4 kernel v3.1 [0] @ ts:1724330265
Aug 22 15:37:45 fedora kernel: NVRM:     RISC-V CSR State:
Aug 22 15:37:45 fedora kernel: NVRM:         sstatus:0x0000000200000020  sscratch:0xffffffffa300f3c0     sie:0x0000000000000220  sip:0x0000000000000000
Aug 22 15:37:45 fedora kernel: NVRM:            sepc:0x00000000019981c0     stval:0x000000080200d000  scause:0x000000000000000d
Aug 22 15:37:45 fedora kernel: NVRM:     RISC-V GPR State:
Aug 22 15:37:45 fedora kernel: NVRM:         ra:0x000000000106c0f0   sp:0x000000080200cb50   gp:0x0000000000000000   tp:0x0000000000000000
Aug 22 15:37:45 fedora kernel: NVRM:         a0:0x0000000001aa3d86   a1:0x0000000000000020   a2:0x000000000000003c   a3:0x0000000001aa3d86
Aug 22 15:37:45 fedora kernel: NVRM:         a4:0x000000080200d03c   a5:0x00000007f4010998   a6:0x0000000000000020   a7:0x000000080200d000
Aug 22 15:37:45 fedora kernel: NVRM:         s0:0x000000080200cb90   s1:0x000000080200d000   s2:0x00000007f40105d4   s3:0x000000080200cc38
Aug 22 15:37:45 fedora kernel: NVRM:         s4:0x00000007f419b330   s5:0x000000080200cc38   s6:0x00000007f407b6f0   s7:0x0000000003148864
Aug 22 15:37:45 fedora kernel: NVRM:         s8:0x0000000003149000   s9:0x0000000000000001  s10:0x0000000003149000  s11:0x0000000000000001
Aug 22 15:37:45 fedora kernel: NVRM:         t0:0x0000000000000004   t1:0x000000080200d031   t2:0x0000000000000000   t3:0x0000000000000020
Aug 22 15:37:45 fedora kernel: NVRM:         t4:0x0000000000000001   t5:0x0000000000000063   t6:0x00000007f4361000
Aug 22 15:37:45 fedora kernel: NVRM:     Stack Trace:
Aug 22 15:37:45 fedora kernel: NVRM:         0x00000000019981c0
Aug 22 15:37:45 fedora kernel: NVRM:         0x000000000106c0f0
Aug 22 15:37:45 fedora kernel: NVRM:         0x00000000010858e4
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001085a64
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001085ecc
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001085f70
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001072bcc
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001002b72
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001aa3d86
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001003194
Aug 22 15:37:45 fedora kernel: NVRM:     PC Trace:
Aug 22 15:37:45 fedora kernel: NVRM:         0x00000000019981c0  0x000000000106c0ec  0x00000000019980fc  0x000000000106c0d4  0x00000000019980a0
Aug 22 15:37:45 fedora kernel: NVRM:         0x000000000106c0bc  0x00000000010679f8  0x0000000001998500  0x000000000106798c  0x00000000010949a8
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001067920  0x000000000106c04c  0x00000000010858e0  0x0000000001085a60  0x00000000019980a0
Aug 22 15:37:45 fedora kernel: NVRM:         0x0000000001085a50
Aug 22 15:37:45 fedora kernel: NVRM:     Local I/O Register State:
Aug 22 15:37:45 fedora kernel: NVRM:         0x01281500:0x00000000   0x01281504:0x00000000   0x01281508:0x00000000   0x01281510:0x00000000
Aug 22 15:37:45 fedora kernel: NVRM:         0x01281528:0x01040002   0x0128152c:0x01000000   0x01281530:0x00000000   0x01280008:0x00000000
Aug 22 15:37:45 fedora kernel: NVRM: ------------[ end crash report ]------------
Aug 22 15:37:45 fedora kernel: NVRM: GPU0 GSP RPC buffer contains function 73 (SET_REGISTRY) and data 0x0000000000000000 0x0000000000000000.
Aug 22 15:37:45 fedora kernel: NVRM: GPU0 RPC history (CPU -> GSP):
Aug 22 15:37:45 fedora kernel: NVRM:     entry function                   data0              data1              ts_start           ts_end             duration actively_polling
Aug 22 15:37:45 fedora kernel: NVRM:      0    73   SET_REGISTRY          0x0000000000000000 0x0000000000000000 0x00062044ecf216f3 0x0000000000000000          y
Aug 22 15:37:45 fedora kernel: NVRM:     -1    72   GSP_SET_SYSTEM_INFO   0x0000000000000000 0x0000000000000000 0x00062044ecf216f0 0x0000000000000000           
Aug 22 15:37:45 fedora kernel: NVRM: GPU0 RPC event history (CPU <- GSP):
Aug 22 15:37:45 fedora kernel: NVRM:     entry function                   data0              data1              ts_start           ts_end             duration during_incomplete_rpc
Aug 22 15:37:45 fedora kernel: NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x62:2477)
Aug 22 15:37:45 fedora kernel: NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
Aug 22 15:37:45 fedora kernel: [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000100] Failed to allocate NvKmsKapiDevice
Aug 22 15:37:45 fedora kernel: [drm:nv_drm_register_drm_device [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000100] Failed to register device
Aug 22 15:37:46 fedora systemd[1]: systemd-modules-load.service: Deactivated successfully.
Aug 22 15:37:46 fedora audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-modules-load comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 22 15:37:46 fedora systemd[1]: Stopped systemd-modules-load.service - Load Kernel Modules.
Aug 22 15:37:46 fedora systemd[1]: Stopping systemd-modules-load.service - Load Kernel Modules...
Aug 22 15:37:46 fedora systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules...
Aug 22 15:37:46 fedora systemd-modules-load[17828]: Module 'msr' is built in
Aug 22 15:37:46 fedora systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules.
Aug 22 15:37:46 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-modules-load comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

workaround to get system running

sudo grubby --update-kernel=ALL --args='nvidia.NVreg_EnableGpuFirmware=0'

nvidia-bug-report.log.gz (737.6 KB)