565 beta driver apparently incompatible with new 6.12 kernel in Debian Sid

[dwm]

Hello NVidia,

I expect you’re probably already aware, but just in case anyone searches for these symptoms: the Linux 565 beta driver appears to build against the new 6.12 kernel in Debian Sid just fine, but not function correctly:

On my machine, the gdm login manager fails to start on boot, and dmesg contains the following WARNING trace from the kernel:

Dec 07 22:51:37 hostname kernel: ------------[ cut here ]------------
Dec 07 22:51:37 hostname kernel: WARNING: CPU: 14 PID: 1227 at drivers/gpu/drm/drm_file.c:312 drm_open_helper+0x132/0x150 [drm]
Dec 07 22:51:37 hostname kernel: Modules linked in: qrtr hid_logitech_hidpp rfcomm cmac algif_hash algif_skcipher af_alg bnep sunrpc uvcvideo videobuf2_vmalloc uvc videobuf2_memops snd_usb_audio videobuf2_v4l2 snd_usbmidi_lib videodev snd_rawmidi videobuf2_common snd_seq_device xpad cdc_acm joydev mc hid_logitech_dj ff_memless binfmt_misc nls_ascii nls_cp437 amd_atl intel_rapl_msr vfat intel_rapl_common xfs fat edac_mce_amd kvm_amd nvidia_drm(O) ccp snd_hda_codec_realtek nvidia_modeset(O) snd_hda_codec_generic kvm snd_hda_scodec_component snd_hda_codec_hdmi crct10dif_pclmul iwlmvm snd_hda_intel btusb ghash_clmulni_intel snd_intel_dspcfg nvidia(O) sha512_ssse3 btrtl snd_intel_sdw_acpi mac80211 sha256_ssse3 btintel snd_hda_codec sha1_ssse3 btbcm libarc4 snd_hda_core aesni_intel btmtk snd_hwdep drm_ttm_helper gf128mul iwlwifi snd_pcm ttm crypto_simd snd_timer bluetooth sd_mod cryptd drm_kms_helper cfg80211 snd sp5100_tco sg crc16 gigabyte_wmi wmi_bmof rapl video soundcore pcspkr watchdog k10temp rfkill evdev msr parport_pc ppdev lp parport
Dec 07 22:51:37 hostname kernel:  drm configfs efi_pstore nfnetlink ip_tables x_tables autofs4 uas usb_storage btrfs blake2b_generic efivarfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c crc32c_generic raid1 hid_generic usbhid hid raid0 dm_mod ahci libahci xhci_pci md_mod libata xhci_hcd igb crc32_pclmul i2c_piix4 scsi_mod nvme usbcore dca mxm_wmi crc32c_intel i2c_smbus scsi_common nvme_core usb_common i2c_algo_bit wmi button
Dec 07 22:51:37 hostname kernel: CPU: 14 UID: 0 PID: 1227 Comm: systemd-logind Tainted: G           O       6.12.3-amd64 #1  Debian 6.12.3-1
Dec 07 22:51:37 hostname kernel: Tainted: [O]=OOT_MODULE
Dec 07 22:51:37 hostname kernel: Hardware name: [redacted]
Dec 07 22:51:37 hostname kernel: RIP: 0010:drm_open_helper+0x132/0x150 [drm]
Dec 07 22:51:37 hostname kernel: Code: 75 39 dc 31 c0 48 83 c4 08 5b 5d 41 5c 41 5d 41 5e 41 5f e9 30 ea 39 dc 48 89 df 89 44 24 04 e8 64 fa ff ff 8b 44 24 04 eb db <0f> 0b b8 ea ff ff ff eb d2 b8 ea ff ff ff eb cb b8 f0 ff ff ff eb
Dec 07 22:51:37 hostname kernel: RSP: 0018:ffffa93180d4fa80 EFLAGS: 00010246
Dec 07 22:51:37 hostname kernel: RAX: ffffffffc1cf12a0 RBX: ffff8fca6cfc43e8 RCX: 0000000000000000
Dec 07 22:51:37 hostname kernel: RDX: ffff8fca55261980 RSI: ffff8fca6cfc43e8 RDI: ffff8fca5c8f4000
Dec 07 22:51:37 hostname kernel: RBP: ffff8fca5c8f4000 R08: ffff8fca45195750 R09: ffff8fca40249dd0
Dec 07 22:51:37 hostname kernel: R10: 00000000000000e2 R11: 0000000000000002 R12: ffff8fca5bb4c000
Dec 07 22:51:37 hostname kernel: R13: ffffffffc1cf12a0 R14: 00000000ffffffed R15: ffff8fca45195750
Dec 07 22:51:37 hostname kernel: FS:  00007f2933a469c0(0000) GS:ffff8fe11eb00000(0000) knlGS:0000000000000000
Dec 07 22:51:37 hostname kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec 07 22:51:37 hostname kernel: CR2: 00005558ed03a1c8 CR3: 000000011b0e6000 CR4: 0000000000350ef0
Dec 07 22:51:37 hostname kernel: Call Trace:
Dec 07 22:51:37 hostname kernel:  <TASK>
Dec 07 22:51:37 hostname kernel:  ? drm_open_helper+0x132/0x150 [drm]
Dec 07 22:51:37 hostname kernel:  ? __warn.cold+0x93/0xf6
Dec 07 22:51:37 hostname kernel:  ? drm_open_helper+0x132/0x150 [drm]
Dec 07 22:51:37 hostname kernel:  ? report_bug+0xff/0x140
Dec 07 22:51:37 hostname kernel:  ? handle_bug+0x58/0x90
Dec 07 22:51:37 hostname kernel:  ? exc_invalid_op+0x17/0x70
Dec 07 22:51:37 hostname kernel:  ? asm_exc_invalid_op+0x1a/0x20
Dec 07 22:51:37 hostname kernel:  ? drm_open_helper+0x132/0x150 [drm]
Dec 07 22:51:37 hostname kernel:  drm_open+0x73/0x110 [drm]
Dec 07 22:51:37 hostname kernel:  drm_stub_open+0x9e/0xd0 [drm]
Dec 07 22:51:37 hostname kernel:  chrdev_open+0xb5/0x230
Dec 07 22:51:37 hostname kernel:  ? __pfx_chrdev_open+0x10/0x10
Dec 07 22:51:37 hostname kernel:  do_dentry_open+0x14f/0x440
Dec 07 22:51:37 hostname kernel:  vfs_open+0x2e/0xe0
Dec 07 22:51:37 hostname kernel:  path_openat+0x82e/0x12d0
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? __x64_sys_getrandom+0x84/0xe0
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  do_filp_open+0xc4/0x170
Dec 07 22:51:37 hostname kernel:  do_sys_openat2+0xae/0xe0
Dec 07 22:51:37 hostname kernel:  __x64_sys_openat+0x55/0xa0
Dec 07 22:51:37 hostname kernel:  do_syscall_64+0x82/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? syscall_exit_to_user_mode+0x4d/0x210
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? syscall_exit_to_user_mode+0x4d/0x210
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  ? do_syscall_64+0x8e/0x190
Dec 07 22:51:37 hostname kernel:  ? srso_return_thunk+0x5/0x5f
Dec 07 22:51:37 hostname kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Dec 07 22:51:37 hostname kernel: RIP: 0033:0x7f293350cb3c
Dec 07 22:51:37 hostname kernel: Code: 83 e2 40 75 51 89 f0 f7 d0 a9 00 00 41 00 74 46 80 3d 37 c5 0e 00 00 74 6a 89 da 48 89 ee bf 9c ff ff ff b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 90 00 00 00 48 8b 54 24 28 64 48 2b 14 25
Dec 07 22:51:37 hostname kernel: RSP: 002b:00007ffc9ebec710 EFLAGS: 00000202 ORIG_RAX: 0000000000000101
Dec 07 22:51:37 hostname kernel: RAX: ffffffffffffffda RBX: 0000000000080902 RCX: 00007f293350cb3c
Dec 07 22:51:37 hostname kernel: RDX: 0000000000080902 RSI: 000055e00ac62c10 RDI: 00000000ffffff9c
Dec 07 22:51:37 hostname kernel: RBP: 000055e00ac62c10 R08: 0000000000000020 R09: 0be7cd1f7881148d
Dec 07 22:51:37 hostname kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001
Dec 07 22:51:37 hostname kernel: R13: 0000000000000008 R14: 0000000000000001 R15: 000055e00ac253d0
Dec 07 22:51:37 hostname kernel:  </TASK>
Dec 07 22:51:37 hostname kernel: ---[ end trace 0000000000000000 ]---

I’ve rolled back to the previous kernel version for now.

Best wishes,
dwm

[Att1la]

Same identical situation on my Debian Testing and same your solution. I posted my experience here.
I hope devs will fix it soon.

[shelter]

Should be fixed in 565.77 which isn’t beta.

[dwm]

Aha, I hadn’t seen that that had been released! It looks like the Debian-format package repository provided by NVidia hasn’t been updated just yet; hopefully whichever folks crank that particular handle will be get some updated packages through QA soon. :)

[dwm]

I don’t suppose anyone knows when we can expect the CUDA package repositories to be updated with the new 565.77 driver release?

(I’m somewhat loathe to pull out the current working packaged drivers to manually modify my OS installation using the .run version!)

[user27558]

Same issue here

[Att1la]

Searching i found a patch already included in driver 565.77, it solves the issue on kernel 6.12.6 and driver 565.57.
This is the link for the patch:

Patch for 565.57.01 + Linux kernel 6.12

A simple patch is needed for the 565.57.01 driver release for nvidia-drm compatibility with Linux kernel 6.12. https://github.com/Binary-Eater/open-gpu-kernel-modules/commit/8ac26d3c66ea88b0f80504bdd1e907658b41609d.patch Without this patch, the symptoms described in 6.12: drm_open_helper RIP · Issue #712 · NVIDIA/open-gpu-kernel-modules · GitHub will be observed. This is needed as a follow-up to commit 641bb4394f40 (“fs: move FMODE_UNSIGNED_OFFSET to fop_flags”) in the Linux kernel. No oth…

[slinux.ve]

There’s a work-around that works:

https://mxlinux.org/blog/6-12-kernel-rollout-is-not-without-issues-with-nvidia-drivers-sigh/

sudo apt install nvidia-xconfig
sudo nvidia-xconfig

[dwm]

@dwm

I don’t suppose anyone knows when we can expect the CUDA package repositories to be updated with the new 565.77 driver release?

NVidia have now released 570.86.10 to their package repository, skipping over the 565.77 release. Everything now seems to be working very nicely, including with Debian’s 6.12.10 kernel.

Marking fixed!