[610.43.03][RTX 4090] Process hangs on exit in libnvidia-glcore/libGLX_nvidia after long Vulkan/OpenXR session

Graphics / Linux Linux
, , ,
1

I am seeing an intermittent but repeatedly reproduced process-exit hang in
NVIDIA userspace after long WayVR/OpenXR sessions.

WayVR accepts SIGTERM and completes all of its instrumented application and
OpenXR teardown. After that, the process remains alive with exactly one runnable
thread busy-spinning inside libnvidia-glcore, reached through libGLX_nvidia. It
makes no system calls while hung.

Environment:

  • GPU: NVIDIA GeForce RTX 4090
  • NVIDIA driver/userspace: 610.43.03
  • Kernel module: NVIDIA Open Kernel Module 610.43.03
  • Distribution: NixOS 26.11
  • Kernel: Linux 6.18.38 x86_64
  • glibc: 2.42
  • libglvnd: 1.7.0
  • Vulkan loader: 1.4.350.0
  • Wayland compositor: niri 26.04
  • OpenXR runtime: Monado
  • Application: WayVR 26.7.1, commit
    a29451ceaaa3a557c02e0ed572cac4b80f69b89b, with teardown-only
    instrumentation
  • The NVIDIA userspace libraries are unmodified. The open kernel module has a
    small local DisplayPort DSC patch needed by the VR headset.

Typical reproduction:

  1. Start Monado.
  2. Start WayVR in OpenXR mode with two Wayland desktop-monitor capture
    overlays.
  3. Run VRChat through Proton, sometimes followed by Pavlov, while using the
    monitor overlays normally: showing them, interacting with them, and
    occasionally moving or resizing them.
  4. After roughly 2.5–4 hours, send SIGTERM to WayVR while Monado and the VR
    application are still running.
  5. Intermittently, WayVR completes its own teardown but then spins indefinitely
    or for several minutes during process exit.

It does not reproduce on every long session, and short sessions usually exit
normally. However, the hung stack and behavior have been identical across
multiple reproductions.

The instrumented teardown shows:

  • SIGTERM is received.
  • OpenXR progresses through STOPPING, IDLE, and EXITING.
  • WayVR saves its state.
  • All instrumented resources are dropped successfully, including its overlays,
    swapchains/OpenXR state, and application state.
  • The final instrumented destructor finishes in about 100 ms.
  • The process then remains alive.

While hung:

  • Only one thread remains.
  • The thread is runnable and consumes CPU.
  • strace observes no system calls over repeated sampling.
  • libnvidia-glcore.so.610.43.03 and
    libGLX_nvidia.so.610.43.03 remain mapped.
  • GDB consistently reports this stack:
#0  libnvidia-glcore.so.610.43.03 + 0x108d5ec
#1  libnvidia-glcore.so.610.43.03 + 0xa4095e
#2  libGLX_nvidia.so.610.43.03    + 0x51787
#3  libGLX_nvidia.so.610.43.03    + 0x51eaf
#4  ld-linux-x86-64.so.2

Disassembly at frame 0 shows a tight loop following pointers and comparing
fields, with no calls or syscalls. I cannot identify the underlying NVIDIA
internal structure without symbols.

nvidia-bug-report-wayvr-hang-20260727-2301.log.gz (510.6 KB)

One captured instance eventually exited by itself after approximately four
minutes. In another reproduction, with
VK_LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING=1 verified in the WayVR process
environment, the same stack remained busy-spinning for over four minutes until
I killed it. Therefore that variable does not prevent this particular hang.

Some useful comparisons:

  • A freshly started WayVR process exits in roughly 0.1–0.2 seconds while the
    same Monado and VRChat processes remain running.
  • 200 separate vulkaninfo --summary executions completed normally while
    VRChat was active.
  • Scripted WayVR tests involving hundreds of OpenXR swapchain recreations,
    overlay show/hide cycles, and monitor-capture overlay recreations all exited
    normally and did not show monotonically increasing shutdown time.
  • This currently appears to require state accumulated inside a particular
    long-running WayVR process rather than merely the concurrent VRChat or Monado
    state.

This may be related to the recently reported libGLX_nvidia/Vulkan-loader unload
issue tracked as NVIDIA bug 6446591:

The symptom differs: that report describes leaked file descriptors during
repeated library unloading, and disabling dynamic library unloading avoids it.
Here, disabling unloading did not prevent the final process-exit spin. Both
issues nevertheless involve the 610.43.03 libGLX_nvidia/libnvidia-glcore
teardown path.

Attached is nvidia-bug-report-wayvr-hang-20260727-2301.log.gz, generated as
root with nvidia-bug-report.sh --extra-system-data while PID 969882 was in the
hung state. The report includes an NMI backtrace identifying the same
libnvidia-glcore instruction offset.

Could NVIDIA identify these offsets and determine whether this is covered by
the fix for bug 6446591, or whether it should be tracked separately? I can
collect another live backtrace, core dump, or other targeted diagnostics if
needed.