Summary. When an EDID override also applies to an empty port, nvidia-drm leaks drm_display_mode objects on that disconnected connector every time it is probed: here about ×4 per probe, never freed. The leak is old (the same allocation site was reported on 580/610), but 615’s new HDCP code sends a hotplug uevent for each TMDS connector at every display power-off, and GNOME re-probes every connector on each uevent. So every screen lock multiplies the leak by 16, and the 4th lock of every boot runs the machine out of memory (5 crashes in 7 days). Root cause with code references, a minimal sysfs reproducer and suggested fixes are below; nvidia-bug-report.log.gz is attached.
System
- GPU: GeForce RTX 2060 (TU104, 10de:1e89). Two Dell U3011 on DVI-D-1 and HDMI-A-1; DP-1 is empty.
- Driver: 615.78.08 open kernel modules (also hit on 615.71.09), RPM Fusion akmod,
nvidia_drm.modeset=1 fbdev=1. Last good: 610.57.04. - OS: Fedora Linux 44 Workstation, kernel 7.2.11-200.fc44 (Fedora’s stable build), GNOME/mutter 50.6 on Wayland.
- Posting here rather than on GitHub because the leaking code is nvidia-drm, which the open and proprietary packages build from the same source.
What happens. 30–60 s after the screen is locked for the 4th time in a boot, gnome-shell hangs, the OOM killer runs and the machine needs a hard reset. All the memory is in a single kmalloc-128 cache (kmalloc-part-NN-128, Used == Total). The OOM report of gnome-shell’s KMS thread names the allocator, identical in every crash where it was captured:
KMS thread invoked oom-killer: gfp_mask=0xc2cc0(GFP_KERNEL|__GFP_NOWARN|__GFP_COMP|__GFP_NOMEMALLOC), order=0
allocate_slab
__kmalloc_cache_noprof
drm_mode_duplicate+0x12/0xb0
add_alternate_cea_modes+0x1d6/0x3a0
_drm_edid_connector_add_modes.part.0+0x1e3/0x400
drm_edid_connector_add_modes+0x41/0x70
drm_edid_override_connector_update+0x9a/0xa0
__nv_drm_detect_encoder+0x8f/0x210 [nvidia_drm]
__nv_drm_connector_detect_internal+0xdc/0x190 [nvidia_drm]
drm_helper_probe_single_connector_modes+0x577/0x690
drm_mode_getconnector+0x34b/0x520
drm_ioctl
struct drm_display_mode is 120 bytes, so it lands in kmalloc-128.
Root cause (nvidia-drm code identical in 610.57.04 and 615.78.08; line numbers from 615.78.08 and kernel 7.2.11):
__nv_drm_detect_encoder()(kernel-open/nvidia-drm/nvidia-drm-connector.c:153) callsdrm_edid_override_connector_update(connector)to test whether an override EDID exists, on every.detectand once for each possible encoder of the connector. The kernel documents that helper as “Only to be used from drm_helper_probe_single_connector_modes() as a fallback” (drivers/gpu/drm/drm_edid.c:2326). As a side effect it adds every mode of the override EDID toconnector->probed_modes.drm_edid_connector_add_modes()ends withadd_alternate_cea_modes()(drm_edid.c:7042), which walks the wholeprobed_modeslist and appends a 59.94/60 Hz (or 23.976/24 Hz) twin for every CEA-matching mode, including twins added by earlier calls. Each call roughly doubles the CEA part of the list.- For a disconnected connector,
drm_helper_probe_single_connector_modes()takes its earlyconnector_status_disconnectedexit (drm_probe_helper.c:641) and never reaches__drm_helper_update_and_validate()/drm_connector_list_update()(:665,:454), soprobed_modesis never merged or freed. - Forcing the port off doesn’t avoid it: nvidia-drm’s
.force(__nv_drm_connector_force(),nvidia-drm-connector.c:359) also runs__nv_drm_connector_detect_internal(). - Result: every forced probe of a disconnected connector multiplies its
probed_modesby about 2^(possible encoders), and the memory stays until reboot. DP-1 here has two possible encoders (834 and 837), so ×4 per probe. Eventually a single GETCONNECTOR ioctl allocates tens of GiB.
Why 615 makes it fatal. 615 added HDCP content-protection support:
- On every display power-off (screen lock, then DPMS off) the driver posts
NV2080_NOTIFIERS_HDCP_STATUS_CHANGE. It reaches nvidia-drm viaReceiveCpEvent()(nvkms-rm.c:1842) →nvHandleCpEventDeferredWork()(nvkms-event.c:283) →nv_drm_connector_update_topology_property()(nvidia-drm-connector.c:980). - That emits
HOTPLUG=1 CONNECTOR=<id> PROPERTY=<content protection>for each TMDS connector (here HDMI-A-1 and DVI-D-1). - For DVI-D it also trips the
WARN_ONatdrm_sysfs.c:497, because the content-protection property is only attached to DP and HDMI connectors (nvidia-drm-connector.c:794). - mutter 50.6 answers every HOTPLUG uevent with
drmModeGetConnector(), a forced probe, on every connector (meta-kms-impl-device.c:688).
So each lock probes the empty DP-1 twice, ×16, and the 4th lock is the one that runs out of memory:
| DP-1 list size, expected | |
|---|---|
| After login | ~7k modes |
| After lock 1 | ~15 MiB |
| After lock 2 | ~240 MiB |
| After lock 3 | ~4 GiB (measured: 4.30 GiB on 23 Oct, 3.98 GiB on 25 Oct, vs 0.34 GiB after boot) |
| Lock 4 | asks for ~60 GiB |
Minimal reproducer (no GNOME, HDCP or screen lock needed; run as root on this machine with 615.78.08 and kernel 7.2.11). DP-1 is a disconnected connector, and any EDID with a CTA block works, because the growth comes from add_alternate_cea_modes():
# give the disconnected DP-1 an override EDID (keep any existing entries, comma-separated)
echo -n 'DP-1:edid/u3011-hdmi.bin' > /sys/module/drm/parameters/edid_firmware
# force a probe of DP-1; it stays disconnected, so no uevent goes out
echo detect > /sys/class/drm/card1-DP-1/status
# count kmalloc-128 objects (all kmalloc-*-128 caches), then repeat the probe
awk '$1 ~ /^kmalloc.*-128$/ {n += $2} END {print n}' /proc/slabinfo
Result here, kmalloc-128 objects added by each of six probes (45,261 before):
| Probe | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| Added | +0 | +240 | +1,380 | +5,400 | +21,540 | +86,040 |
Once above normal slab noise it is ×4 per probe (DP-1’s two possible encoders, ×2 each): ~115k objects, ~15 MB, after six probes. Going by the code, removing the override afterwards stops further growth but frees nothing until reboot.
Full-system reproduction (5 of 5 boots): nvidia_drm.modeset=1; an EDID override without a connector prefix, so every connector gets it (drm.edid_firmware=edid/fixed_edid.bin, an EDID with a CTA block); at least one disconnected connector; GNOME 50 on Wayland. Boot, then lock the screen so the displays power off and unlock, four times. SUnreclaim is ~4 GiB after the 3rd lock; on the 4th, gnome-shell stops responding within 20–40 s and the system OOMs 30–62 s after the lock.
Crashes (UTC, 4th display power-off to OOM; cache size from the first OOM report’s “Unreclaimable slab info”):
| # | 4th power-off | gnome-shell unresponsive | first OOM | cache, size at first OOM |
|---|---|---|---|---|
| 1 | 19 Oct 22:57:27 | +19 s | +52 s | kmalloc-part-15-128, 23.2 GiB |
| 2 | 21 Oct 09:45:58 | +30 s | +30 s | kmalloc-part-05-128, 22.7 GiB |
| 3 | 23 Oct 12:18:29 | +38 s | +54 s | kmalloc-128, 22.3 GiB |
| 4 | 24 Oct 21:36:17 | +38 s | +62 s | kmalloc-part-09-128, 27.1 GiB |
| 5 | 25 Oct 22:58:30 | +20 s | +53 s | kmalloc-part-10-128, 20.1 GiB |
Crash 5 was predicted beforehand from the 3.98 GiB measured after that boot’s 3rd lock. A user service logged the lock state, mutter’s power-save mode and udevadm monitor DRM uevents next to the kernel log:
22:58:30.657 LOCK screen locked
22:58:30.777 DRM ACTION=change HOTPLUG=1 CONNECTOR=830 PROPERTY=831 (830 = HDMI-A-1)
22:58:30.777 DRM ACTION=change HOTPLUG=1 CONNECTOR=828 PROPERTY=831 (828 = DVI-D-1)
22:58:30.777 kernel: WARNING: drivers/gpu/drm/drm_sysfs.c:497 at drm_sysfs_connector_property_event+0x14d/0x160, CPU#…: nvidia-modeset/…
nv_drm_connector_update_topology_property+0x5a/0x70 [nvidia_drm]
nv_drm_handle_display_cp_topology_change+0x9e/0xe0 [nvidia_drm]
nv_drm_event_callback+0x53/0xb0 [nvidia_drm]
nvKmsKapiHandleEventQueueChange+0xba/0x110 [nvidia_modeset]
22:58:30.782 POWER displays off (mode 3)
22:58:50.962 at-spi2-registryd: Disabling unresponsive app (gnome-shell)
22:59:24 OOM killer; slab_unreclaimable 5,375,748 pages (20.5 GiB)
Every lock looks like this: two property uevents and one WARN, from the DVI connector. After the first OOM kill the leak kept growing at ~325 MiB/s (20.5 → 29.7 GiB in 29 s).
610 and related reports. 610.57.04 has none of the HDCP code, so locking sends no uevents and nothing re-probes. The override leak itself is there in 610 and 580 too:
- KWin 6.7.x causes massive kmalloc-128 kernel slab leak on NVIDIA driver (Wayland) — same
drm_mode_duplicate,bytes_req=120site, triggered by a debugfsedid_override; no reply yet - nvidia-modeset: nvHandleCpEventDeferredWork displayId invalid on suspend — monitor doesn't recover after resume (HP USB-C Dock G5, Optimus laptop) · Issue #1380 · NVIDIA/open-gpu-kernel-modules · GitHub — another 615 regression in the same content-protection event path (
nvHandleCpEventDeferredWork: displayId … invalid)
Suggested fixes
- nvidia-drm: don’t call
drm_edid_override_connector_update()from.detector.force. Check for an override without adding modes, or only consult it from.get_modes/ when connected. Never leave modes onprobed_modesfrom detection. - nvidia-drm 615: only emit the content-protection property uevent for connectors that have the property attached (also fixes the
drm_sysfs.c:497WARN on DVI), and only when the HDCP topology actually changed, not on every DPMS transition. - As hardening in the DRM core, the disconnected path of
drm_helper_probe_single_connector_modes()could drop leftoverprobed_modes.
Workaround. Give the override connector prefixes so it only applies to connected outputs, e.g. drm.edid_firmware=DVI-D-1:edid/<file>.bin,HDMI-A-1:edid/<file>.bin. The parameter can be changed at runtime through /sys/module/drm/parameters/edid_firmware. Forcing empty ports off with video=DP-1:d doesn’t help, because nvidia-drm’s .force runs detection too. A monitor switched off at the wall still grows ×2 per probe until it reconnects.
Addendum: the EDID-override leak is a crash bug on its own, without 615’s HDCP uevents. Any real hotplug makes mutter re-probe every connector, so each hotplug event grows the disconnected connector’s list ×4. On 25 Oct (615.78.08, no screen lock), switching both monitors off and on at their power buttons produced 4 hotplug events and took SUnreclaim from 0.36 GiB to ~15 GiB, with gnome-shell hung for ~30 s. With the same nvidia-drm code on 610/580, about 7–8 hotplug events in one boot would run a machine like this out of memory.
nvidia-bug-report.log.gz (242.5 KB)