VT switch results in black screen + cursor on Xorg with certain compositors, 545+

Graphics / Linux Linux
1

Not sure if this is the right place to put this since it’s been mentioned several times already (here and elsewhere, though this one specifically is a bit different), but basically I’m having an issue where switching consoles and resuming from suspend/hibernate while running Xorg with some compositors (can confirm with Xfwm4 built in compositor and picom with egl, glx and xrender backends (vsync can be either on or off, same behavior), but not with fastcompmgr, with which this works fine) results in getting a black screen with only a cursor on it.

The cursor can be moved, and changes shape upon e.g. hovering over a text box, window border, etc., and it is possible to get the session back by killing the compositor from another terminal and switching back to the console with the desktop session on it (and the compositor can be relaunched after getting back in), but it is not, for some reason, possible to Ctrl+C e.g. picom if ran inside a terminal window that I’m (apparently) hovering with the mouse cursor over (the ^C’s are visible in the terminal after killing the compositor externally, so it’s not a keyboard input issue necessarily).

This appears to be exclusive to Xorg; I tried some Wayland compositors (Hyprland and sway --unsupported-gpu, in this case) and those behave as intended on console switches. It seems to happen with it running both as root, and in rootless mode, so it’s not that, necessarily. No combination of nvidia_drm.modeset={0,1} and nvidia_drm.fbdev={0,1} seems to have any effect on this.

My GPU is too old for the open kernel drivers so I’m using the proprietary modules here, so not sure if this happens with the open ones; will probably get a newer card for the sole purpose of testing Linux graphics things but not until this one moves into legacy driver territory (which should be pretty soon anyway).

Seems to be somewhat recent (depends on your definition of “recent”) as this works just fine with driver 535.230.02, but not with 550.144.03, and according to this, it started happening with 545 (not packaged in AUR anymore so didn’t try, at least not yet)?

Distro: Arch Linux (only for the purposes of testing this, though)
Kernel: 6.12.20-1-lts (also tried 6.6 LTS from here, behaves identically)
Initramfs generator: mkinitcpio, busybox-based initramfs; mostly unconfigured
DE: XFCE 4.20
DM: SDDM (also tried lightdm, same thing)
Driver: 570.133.07
Display server: Xorg 1.21.1.16 (running as root)
CPU: i7-4770S
GPU: GTX 970

nvidia-bug-report.log.gz (617.0 KB)

(attached nvidia-bug-report.log.gz is from this configuration (first option listed, not the stuff in brackets), and from right after switching to a TTY and back to Xorg with the black screen, then to a TTY again to kill the compositor and back to Xorg again to generate the report)

2

Nvm nvidia.NVreg_PreserveVideoMemoryAllocations=1 seems to fix it on 545+ (not marking as solution yet)

3

I can reproduce what appears to be a closely related suspend/resume failure on a current XFCE/Xorg/NVIDIA setup, and I was able to capture the blocked process with GDB while the screen was black.

System:

  • Manjaro Linux

  • kernel 6.18.49-1-MANJARO, S3/deep suspend

  • XFCE 4.20

  • xfwm4 4.20.0

  • Xorg 21.1.24

  • RTX 3060

  • NVIDIA 610.57.04, open kernel module

  • compositing enabled

The failure is deterministic when XFWM is explicitly started with:

xfwm4 --replace --vblank=glx

After resume, XFWM remains alive but its main thread is indefinitely blocked here:

poll()
→ xcb_wait_for_reply64()
→ _XReply()
→ libGLX_nvidia.so
→ libnvidia-glcore.so.610.57.04
→ glXBindTexImageEXT()
→ XFWM compositor code

I took two independent GDB snapshots several seconds apart and the same XFWM PID was in the identical call chain with no forward progress.

This does not appear to be a general Xorg/GPU/display failure. During the black screen:

  • Xorg remained responsive and its main thread was normally waiting for events.

  • xrandr continued to report the connected DP display and correct mode.

  • nvidia-smi and nvidia-settings remained responsive.

  • DRM/RandR state remained valid.

  • A separate fresh glxinfo -B invocation successfully created an NVIDIA OpenGL context and reported direct rendering normally.

The vblank backend is the important discriminator:

  • --vblank=glx: reproducibly produces the black screen after resume.

  • --vblank=xpresent: survives repeated suspend/resume.

  • --vblank=off: also resumes normally with compositing still enabled.

The persistent xfconf setting was kept at xpresent while these tests were performed; the alternate modes were forced only on the live XFWM process, and debug output confirmed which backend was actually active.

In an earlier reproduction I killed the hung XFWM, but XFCE respawned another GLX XFWM and that fresh process also hung, this time while waiting in glXQueryContext(). Restarting LightDM restores the graphical session.

This system already has NVreg_PreserveVideoMemoryAllocations=1, so that workaround does not prevent the issue here.

My working interpretation is therefore fairly narrow: after S3 resume, XFWM’s GLX-vblank path can enter a permanent NVIDIA GLX/X11 request-reply wait, while the X server, GPU, RandR/display path, and unrelated fresh GLX clients remain functional. I cannot tell from the client-side traces whether the root cause is in NVIDIA’s GLX implementation, the Xorg-side GLX handling, or an XFWM call sequence that exposes a driver bug, but this does not reproduce through XFWM’s XPresent vblank path.

I have full GDB thread dumps and resume diagnostics from the failed state if they would be useful.