NVidia Quadro T500 Mobile driver issue with Lenovo P14S

I cannot get the NVidia driver to work on my Lenovo P14S / Quadro T500 Mobile running Linux Mint 22.2, kernel 6.8.0-86-generic. Secure boot is disabled and I have set “options nvidia_drm modeset=1”.

nvidia-smi

No devices were found

dpkg -l |grep nvidia

ii  libnvidia-cfg1-570:amd64                      570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA binary OpenGL/GLX configuration library
ii  libnvidia-common-570                          570.195.03-0ubuntu0.24.04.1                all          Shared files used by the NVIDIA libraries
ii  libnvidia-compute-570:amd64                   570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA libcompute package
ii  libnvidia-compute-570:i386                    570.195.03-0ubuntu0.24.04.1                i386         NVIDIA libcompute package
rc  libnvidia-compute-580:amd64                   580.95.05-0ubuntu0.24.04.2                 amd64        NVIDIA libcompute package
ii  libnvidia-decode-570:amd64                    570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-decode-570:i386                     570.195.03-0ubuntu0.24.04.1                i386         NVIDIA Video Decoding runtime libraries
ii  libnvidia-egl-wayland1:amd64                  1:1.1.13-1build1                           amd64        Wayland EGL External Platform library -- shared library
ii  libnvidia-egl-wayland1:i386                   1:1.1.13-1build1                           i386         Wayland EGL External Platform library -- shared library
ii  libnvidia-encode-570:amd64                    570.195.03-0ubuntu0.24.04.1                amd64        NVENC Video Encoding runtime library
ii  libnvidia-encode-570:i386                     570.195.03-0ubuntu0.24.04.1                i386         NVENC Video Encoding runtime library
ii  libnvidia-extra-570:amd64                     570.195.03-0ubuntu0.24.04.1                amd64        Extra libraries for the NVIDIA driver
ii  libnvidia-fbc1-570:amd64                      570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-fbc1-570:i386                       570.195.03-0ubuntu0.24.04.1                i386         NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-gl-570:amd64                        570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  libnvidia-gl-570:i386                         570.195.03-0ubuntu0.24.04.1                i386         NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  nvidia-compute-utils-570                      570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA compute utilities
ii  nvidia-dkms-570-open                          570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA DKMS package (open kernel module)
ii  nvidia-driver-570-open                        570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA driver (open kernel) metapackage
ii  nvidia-firmware-570-570.195.03                570.195.03-0ubuntu0.24.04.1                amd64        Firmware files used by the kernel module
ii  nvidia-kernel-common-570                      570.195.03-0ubuntu0.24.04.1                amd64        Shared files used with the kernel module
ii  nvidia-kernel-source-570-open                 570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA kernel source package
ii  nvidia-prime                                  0.8.17.2                                   all          Tools to enable NVIDIA's Prime
ii  nvidia-settings                               510.47.03-0ubuntu4                         amd64        Tool for configuring the NVIDIA graphics driver
ii  nvidia-utils-570                              570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA driver support binaries
ii  screen-resolution-extra                       0.18.3ubuntu0.24.04.1                      all          Extension for the nvidia-settings control panel
ii  xserver-xorg-video-nvidia-570                 570.195.03-0ubuntu0.24.04.1                amd64        NVIDIA binary Xorg driver

inxi -Fxxxrz

Graphics:
  Device-1: Intel TigerLake-LP GT2 [Iris Xe Graphics] vendor: Lenovo
    driver: i915 v: kernel arch: Gen-12.1 ports: active: DP-5,DP-7,eDP-1
    empty: DP-1, DP-2, DP-3, DP-4, DP-6, HDMI-A-1 bus-ID: 00:02.0
    chip-ID: 8086:9a49 class-ID: 0300
  Device-2: NVIDIA TU117GLM [Quadro T500 Mobile] vendor: Lenovo
    driver: nvidia v: 570.195.03 arch: Turing pcie: speed: 8 GT/s lanes: 4
    bus-ID: 01:00.0 chip-ID: 10de:1fbb class-ID: 0302
  Device-3: IMC Networks Integrated Camera driver: uvcvideo type: USB
    rev: 2.0 speed: 480 Mb/s lanes: 1 bus-ID: 3-4:3 chip-ID: 13d3:5406
    class-ID: fe01 serial: <filter>
  Display: x11 server: X.Org v: 21.1.11 with: Xwayland v: 23.2.6 driver: X:
    loaded: modesetting unloaded: fbdev,vesa dri: iris gpu: i915 display-ID: :0
    screens: 1
  Screen-1: 0 s-res: 3840x2520 s-dpi: 96 s-size: 1016x667mm (40.00x26.26")
    s-diag: 1215mm (47.85")
  Monitor-1: DP-5 mapped: DP-2-1 pos: top-left res: 1920x1080 hz: 60
    size: N/A modes: max: 640x480 min: 848x480
  Monitor-2: DP-7 mapped: DP-2-3 pos: top-right model: Samsung U28E590
    serial: <filter> res: 1920x1080 hz: 60 dpi: 80
    size: 608x345mm (23.94x13.58") diag: 699mm (27.5") modes: max: 3840x2160
    min: 720x400
  Monitor-3: eDP-1 pos: primary,bottom-c model: BOE Display 0x0982
    res: 2560x1440 hz: 60 dpi: 210 size: 309x174mm (12.17x6.85")
    diag: 355mm (14") modes: 3840x2160
  API: EGL v: 1.5 hw: drv: intel iris platforms: device: 0 drv: iris
    device: 1 drv: swrast gbm: drv: iris surfaceless: drv: iris x11: drv: iris
    inactive: wayland
  API: OpenGL v: 4.6 compat-v: 4.5 vendor: intel mesa
    v: 25.0.7-0ubuntu0.24.04.2 glx-v: 1.4 direct-render: yes renderer: Mesa
    Intel Iris Xe Graphics (TGL GT2) device-ID: 8086:9a49

sudo dmesg | grep -E “nvidia|firmware|refused|error|fail”

[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.8.0-86-generic root=UUID=e46f2983-a6d5-49e8-9ae2-265a874dc8ad ro quiet splash nvidia-drm.modeset=1
[    0.059896] Kernel command line: BOOT_IMAGE=/vmlinuz-6.8.0-86-generic root=UUID=e46f2983-a6d5-49e8-9ae2-265a874dc8ad ro quiet splash nvidia-drm.modeset=1
[    0.809007] pci 0000:01:00.0: ROM [mem size 0x00080000 pref]: failed to assign
[    2.445384] psmouse serio2: trackpoint: Elan TrackPoint firmware: 0x12, buttons: 3/3
[    3.086670] nvidia: loading out-of-tree module taints kernel.
[    3.086683] nvidia: module verification failed: signature and/or required key missing - tainting kernel
[    3.142404] nvidia-nvlink: Nvlink Core is being initialized, major device number 236
[    3.143948] nvidia 0000:01:00.0: enabling device (0006 -> 0007)
[    3.241250] nvidia-modeset: Loading NVIDIA UNIX Open Kernel Mode Setting Driver for x86_64  570.195.03  Release Build  (dvs-builder@U22-I3-H04-03-1)  Sat Sep 20 00:39:49 UTC 2025
[    3.273421] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[    3.318175] iwlwifi 0000:00:14.3: loaded firmware version 77.b405f9d4.0 QuZ-a0-hr-b0-77.ucode op_mode iwlmvm
[    3.318479] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[    3.321925] Bluetooth: hci0: Found device firmware: intel/ibt-19-0-4.sfi
[    3.661493] NVRM: s_executeBooterUcode_TU102: Booter failed with non-zero error code: 0xb
[    3.661497] NVRM: kgspExecuteBooterLoad_TU102: failed to execute Booter Load: 0xffff
[    3.661499] NVRM: kgspBootstrap_TU102: failed to execute Booter Load (ucode for initial boot): 0xffff
[    3.662086] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[    3.662142] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[    3.663518] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[    3.664086] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[    3.664263] [drm:nv_drm_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000100] Failed to allocate NvKmsKapiDevice
[    3.665705] [drm:nv_drm_register_drm_device [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000100] Failed to register device
[    3.729715] thermal thermal_zone7: failed to read out thermal zone (-61)
[    3.753649] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/tgl_dmc_ver2_12.bin (v2.12)
[    3.934817] thermal thermal_zone7: failed to read out thermal zone (-61)
[    5.094796] Bluetooth: hci0: Waiting for firmware download to complete
[   12.887591] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   12.887602] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   12.887776] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   12.887816] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   12.888720] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   12.889014] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   13.040508] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   13.041315] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   13.041603] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   13.041750] NVRM: nvAssertFailedNoLog: Assertion failed: rmapiLockIsOwner() && rmGpuLockIsOwner() @ conf_compute_api.c:77
[   14.046220] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   14.046226] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   14.046371] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   14.046409] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   14.047216] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   14.047466] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   14.196200] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   14.196961] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   14.197269] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   14.197372] NVRM: nvAssertFailedNoLog: Assertion failed: rmapiLockIsOwner() && rmGpuLockIsOwner() @ conf_compute_api.c:77
[   41.073251] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   41.073256] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   41.073398] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   41.073431] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   41.074018] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   41.074365] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   58.323191] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   58.323199] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   58.323342] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   58.323421] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   58.324225] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   58.324466] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   58.480441] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   58.481344] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   58.481657] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   59.489375] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   59.489383] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   59.489528] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   59.489572] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   59.490324] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   59.490570] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   59.649179] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   59.651559] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   59.652006] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   59.810403] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   59.811206] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   59.811497] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   60.821063] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   60.821072] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   60.821215] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   60.821261] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   60.822058] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   60.822391] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   89.989632] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   89.989640] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   89.989783] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   89.989819] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   89.990680] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   89.990964] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   90.141637] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   90.142418] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   90.142698] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   91.155495] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[   91.155503] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[   91.155658] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[   91.155709] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   91.156762] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   91.157452] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   91.312472] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   91.313294] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   91.313585] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   91.463748] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   91.464549] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   91.464822] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[   91.623996] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[   91.624716] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[   91.625096] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  103.782290] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  103.782300] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  103.782698] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  103.782776] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  103.783754] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  103.783995] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  103.933381] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  103.934233] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  103.934495] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  104.941771] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  104.941779] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  104.941921] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  104.941959] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  104.942723] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  104.942970] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  105.091647] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  105.092369] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  105.092630] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  105.242449] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  105.243288] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  105.243531] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  105.392048] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  105.392756] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  105.393029] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  132.813761] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  132.813769] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  132.813939] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  132.813979] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  132.814655] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  132.815050] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  493.323887] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  493.323899] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  493.324030] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  493.324068] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  493.324822] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  493.325156] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  565.264834] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  565.264841] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  565.264972] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  565.265009] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  565.265809] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  565.266148] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  565.416091] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  565.416903] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  565.417240] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  565.571792] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  565.572582] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  565.572992] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  565.722796] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  565.724123] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  565.725155] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  566.881450] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  566.881458] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  566.881590] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  566.881627] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  566.882375] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  566.882822] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  588.314669] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  588.314678] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  588.314823] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  588.314866] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  588.315832] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  588.316153] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  588.469989] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  588.470794] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  588.471156] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  589.478220] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  589.478227] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  589.478369] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  589.478406] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  589.479128] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  589.479396] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  589.630438] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  589.631149] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  589.631406] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  589.785398] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  589.786254] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  589.786529] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  589.937026] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  589.937744] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  589.938050] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  692.232893] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  692.232902] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  692.233045] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  692.233088] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  692.233891] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  692.234151] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  692.403315] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  692.404508] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  692.404947] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  693.419401] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  693.419409] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  693.419612] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  693.419864] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  693.420973] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  693.421482] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  693.584562] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  693.585466] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  693.585784] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  693.753181] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  693.754269] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  693.754747] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  693.918226] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  693.919018] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  693.919433] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  871.577870] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  871.577879] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  871.578012] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  871.578051] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  871.579039] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  871.579443] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  871.730832] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  871.731616] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  871.732039] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  871.886688] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  871.887555] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  871.887940] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  872.046719] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  872.047532] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  872.048001] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[  873.207321] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[  873.207329] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[  873.207462] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[  873.207497] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[  873.208371] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[  873.208825] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0
[ 1047.456069] NVRM: kgspExecuteFwsec_TU102: failed to execute FWSEC for FRTS: FRTS error code 0x2c9
[ 1047.456079] NVRM: nvAssertOkFailedNoLog: Assertion failed: Failure: Generic Error [NV_ERR_GENERIC] (0x0000FFFF) returned from status @ kernel_gsp_tu102.c:482
[ 1047.456216] NVRM: Xid (PCI:0000:01:00): 140, An uncorrectable ECC error detected (possible firmware handling failure) DRAM:1227137661, LTC:0, MMU:0, PCIE:0
[ 1047.456259] NVRM: RmInitAdapter: Cannot initialize GSP firmware RM
[ 1047.457299] NVRM: GPU 0000:01:00.0: RmInitAdapter failed! (0x62:0x40:1861)
[ 1047.457712] NVRM: GPU 0000:01:00.0: rm_init_adapter failed, device minor number 0

I’ve tried newer kernels (6.14.0-33, 6.14.0-34), and other nvidia driver versions (580-open, 535), but nvidia-smi always returns “No devices were found”.

Any help would be greatly appreciated.