Nvidia-modeset: ERROR: GPU:0: Failed to allocate memory for the display color lookup table

Hello everyone,

sometimes, after the waking up from hibernation, the system throws error:

nvidia-modeset: ERROR: GPU:0: Failed to allocate memory for the display color lookup table.

and then the system freezes in the login screen. I don’t know when or why. I can’t find any information about this error.

System: 
  Kernel: 6.2.0-33-generic x86_64 bits: 64 compiler: N/A Desktop: Cinnamon 5.8.4 tk: GTK 3.24.33
    wm: muffin dm: LightDM Distro: Linux Mint 21.2 Victoria base: Ubuntu 22.04 jammy
Machine:
  Type: Laptop System: Acer product: Predator PH315-52 v: V1.12 serial: <superuser required>
    Chassis: type: 10 serial: <superuser required>
  Mobo: CFL model: Covini_CFS v: V1.12 serial: <superuser required> UEFI: Insyde v: 1.12
    date: 07/28/2020
Battery:
  ID-1: BAT0 charge: 38.4 Wh (100.0%) condition: 38.4/38.4 Wh (100.0%) volts: 16.4 min: 15.4
    model: SMP KT00407009 AP18E7M serial: <filter> status: Full
CPU:
  Info: 6-core model: Intel Core i7-9750H bits: 64 type: MT MCP arch: Coffee Lake rev: A cache:
    L1: 384 KiB L2: 1.5 MiB L3: 12 MiB
  Speed (MHz): avg: 1324 high: 2600 min/max: 800/4500 cores: 1: 900 2: 900 3: 900 4: 900 5: 900
    6: 2600 7: 2600 8: 900 9: 900 10: 899 11: 900 12: 2600 bogomips: 62399
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Graphics:
  Device-1: Intel CoffeeLake-H GT2 [UHD Graphics 630] vendor: Acer Incorporated ALI driver: i915
    v: kernel ports: active: eDP-1 empty: DP-1, DP-2, DP-3, HDMI-A-1, HDMI-A-2 bus-ID: 00:02.0
    chip-ID: 8086:3e9b
  Device-2: NVIDIA TU116M [GeForce GTX 1660 Ti Mobile] vendor: Acer Incorporated ALI
    driver: nvidia v: 535.113.01 pcie: speed: 2.5 GT/s lanes: 16 ports: active: none
    empty: DP-4,HDMI-A-3 bus-ID: 01:00.0 chip-ID: 10de:2191
  Device-3: Chicony HD User Facing type: USB driver: uvcvideo bus-ID: 1-7:13 chip-ID: 04f2:b64f
  Display: x11 server: X.Org v: 1.21.1.4 driver: X: loaded: modesetting,nvidia
    unloaded: fbdev,nouveau,vesa gpu: i915 display-ID: :0 screens: 1
  Screen-1: 0 s-res: 1920x1080 s-dpi: 98
  Monitor-1: eDP-1 model: BOE Display res: 1920x1080 dpi: 142 diag: 398mm (15.7")
  OpenGL: renderer: Mesa Intel UHD Graphics 630 (CFL GT2) v: 4.6 Mesa 23.0.4-0ubuntu1~22.04.1
    direct render: Yes
Audio:
  Device-1: Intel Cannon Lake PCH cAVS vendor: Acer Incorporated ALI driver: snd_hda_intel
    v: kernel bus-ID: 00:1f.3 chip-ID: 8086:a348
  Device-2: NVIDIA TU116 High Definition Audio vendor: Acer Incorporated ALI
    driver: snd_hda_intel v: kernel pcie: speed: 2.5 GT/s lanes: 16 bus-ID: 01:00.1
    chip-ID: 10de:1aeb
  Sound Server-1: ALSA v: k6.2.0-33-generic running: yes
  Sound Server-2: PulseAudio v: 15.99.1 running: yes
  Sound Server-3: PipeWire v: 0.3.48 running: yes
Network:
  Device-1: Intel Cannon Lake PCH CNVi WiFi
    vendor: Rivet Networks Killer Wireless-AC 1550i Wireless driver: iwlwifi v: kernel
    bus-ID: 00:14.3 chip-ID: 8086:a370
  IF: wlp0s20f3 state: up mac: <filter>
  Device-2: Qualcomm Atheros Killer E2500 Gigabit Ethernet vendor: Acer Incorporated ALI
    driver: alx v: kernel pcie: speed: 2.5 GT/s lanes: 1 port: 3000 bus-ID: 07:00.0
    chip-ID: 1969:e0b1
  IF: enp7s0 state: down mac: <filter>
  IF-ID-1: vmnet1 state: unknown speed: N/A duplex: N/A mac: <filter>
  IF-ID-2: vmnet8 state: unknown speed: N/A duplex: N/A mac: <filter>
Bluetooth:
  Device-1: Intel Bluetooth 9460/9560 Jefferson Peak (JfP) type: USB driver: btusb v: 0.8
    bus-ID: 1-14:3 chip-ID: 8087:0aaa
  Report: hciconfig ID: hci0 rfk-id: 1 state: up address: <filter> bt-v: 3.0 lmp-v: 5.1
    sub-v: 100
Drives:
  Local Storage: total: 1.38 TiB used: 725.65 GiB (51.5%)
  ID-1: /dev/nvme0n1 vendor: Western Digital model: PC SN720 SDAPNTW-512G-1014 size: 476.94 GiB
    speed: 31.6 Gb/s lanes: 4 serial: <filter> temp: 41.9 C
  ID-2: /dev/sda vendor: Seagate model: ST1000LM049-2GH172 size: 931.51 GiB speed: 6.0 Gb/s
    serial: <filter>
Swap:
  ID-1: swap-1 type: partition size: 18.63 GiB used: 0 KiB (0.0%) priority: -2 dev: /dev/nvme0n1p3
Sensors:
  System Temperatures: cpu: 27.8 C pch: 50.0 C mobo: N/A
  Fan Speeds (RPM): N/A
Repos:
  Processes: 438 Uptime: 2h 22m Memory: 15.33 GiB used: 7.34 GiB (47.9%) Init: systemd v: 249
  runlevel: 5 Compilers: gcc: 11.4.0 alt: 11/12 Client: Cinnamon v: 5.8.4 inxi: 3.3.13

Bootconfig:

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=3
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Ubuntu`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash resume=UUID=XXXXXXXX pcie_aspm=off"
GRUB_CMDLINE_LINUX="nouveau.blacklist=1"
GRUB_CMDLINE_LINUX="nvidia-drm.modeset=1"
CONFIG_CALGARY_IOMMU=n
CONFIG_CALGARY_IOMMU_ENABLED_BY_DEFAULT=n
CONFIG_PINCTRL=r
CONFIG_X86_INTEL_LPSS=r

Driver:

$ nvidia-smi
Tue Oct  3 17:50:39 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.113.01             Driver Version: 535.113.01   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 1660 Ti     Off | 00000000:01:00.0 Off |                  N/A |
| N/A   39C    P8               5W /  80W |    316MiB /  6144MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1326      G   /usr/lib/xorg/Xorg                           84MiB |
|    0   N/A  N/A      7576    C+G   ...06762826,6119812730160646140,262144      229MiB |
+---------------------------------------------------------------------------------------+

Thanks!

nvidia-bug-report.log.gz (1.1 MB)