595.91.07: vkCreateSemaphore / vkGetSemaphoreFdKHR fail with VK_ERROR_INITIALIZATION_FAILED in Chrome's sandboxed GPU process (WebGPU)

Graphics / Linux Linux
1

Summary

Chrome’s GPU process loses its WebGPU device after roughly 10,000 to 150,000 canvas presents. Dawn (Chrome’s WebGPU implementation) reports that vkCreateSemaphore or vkGetSemaphoreFdKHR failed with VK_ERROR_INITIALIZATION_FAILED, and the driver prints one of these to stderr:

Error: NVIDIA: Failed to allocate semaphore device memory
Error: NVIDIA: Unsupported external semaphore handle type

Sometimes the GPU process aborts (SIGABRT) right after the first message instead. The attached page reproduces it every time within about three minutes, but only while Chrome’s GPU-process sandbox is on, which is Chrome’s default.

System

  • GPU: GeForce RTX 3060 12 GB (VBIOS 94.06.14.00.5E)
  • Driver: 595.91.07, open kernel module (Ubuntu package nvidia-driver-595-open 595.91.07-0ubuntu0.26.04.1)
  • OS: Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic
  • Desktop: KDE Plasma 6.6.6 (KWin 6.6.6), Wayland session
  • Vulkan loader: 1.4.341
  • Browser: Google Chrome 155.0.8059.39 (stable), default settings
  • Displays: two 2560x1440 at 180 Hz (one with HDR on) and one 1920x1080 at 60 Hz

Steps to reproduce

  1. In a Wayland session, start Chrome from a terminal so its stderr is visible:
    google-chrome-stable --user-data-dir=$(mktemp -d) file:///path/to/webgpu-semaphore-repro.html
  2. Keep the window visible. A minimized page stops rendering.
  3. Wait up to three minutes. The page clears and presents 32 small canvases every frame (about 750 presents per second here) and turns its status bar red when the device is lost. Add ?hdr=1 to the URL to use rgba16float canvases instead of rgba8unorm.

Results (32 canvases of 128x128)

Configuration Runs Outcome
rgba16float with extended tone mapping, default sandbox 6 All lost the device, after 12k, 75k, 129k, 136k, 137k and 146k presents (15 to 183 s)
rgba8unorm (SDR), default sandbox 1 Lost the device after 130k presents (172 s)
rgba16float, --disable-gpu-sandbox 2 No failure in 10 minutes (561k and 595k presents)
rgba16float, Chrome for Testing 155.0.8059.39 with --no-sandbox 1 No failure in 10 minutes (582k presents)

Observations

  • On Linux, Chrome shares each WebGPU canvas image between Dawn’s Vulkan device and its GL compositor through opaque-FD external memory and semaphores. Every present exports a new opaque-FD semaphore from Vulkan, which the compositor imports with glImportSemaphoreFdEXT, and the compositor hands a semaphore back for the next frame.
  • In steady state the GPU process opens /dev/nvidiactl about 11 times per present (strace: 1.77 million opens over 162k presents). During a traced run about 700 nvidiactl descriptors were open at any moment, and the count did not grow.
  • Under Chrome’s GPU sandbox, each of those opens is performed by a separate broker process, which passes the descriptor back over a Unix socket (SCM_RIGHTS). With the sandbox off, the GPU process opens them itself and the failure does not occur.
  • strace shows no failing syscall and no refused file access in the sandboxed GPU process during the present loop. The failure only surfaces as the driver errors above. At startup the sandbox does refuse ~/.nv/GLCache, ~/.nvidia-aftermath-rc, libnvidia-rtcore.so.595.91.07 and the Vulkan loader’s layer and ICD manifest probes.
  • The failure looks timing-dependent. With the whole browser running under strace (about 4.4 times slower), the sandboxed configuration ran 162k presents without failing.
  • In fresh sessions the failures clustered between 129k and 146k presents. The two runs started a few seconds after a previous failure failed much sooner, at 12k and 75k.
  • VRAM use stays flat at about 2 GB of 12 GB, so this is not video memory running out.
  • A Pixi.js-based WebGPU game we develop (one canvas at 180 Hz) hits the same errors in normal use, three times in about 5.5 hours of play.

Attachments

  • webgpu-semaphore-repro.zip: the repro page (webgpu-semaphore-repro.html) and Chrome’s stderr from a failing run (chrome-stderr-failing-run.log)
  • nvidia-bug-report.log.gz

Chromium issue for the same problem: [add link]

nvidia-bug-report.log.gz (403.7 KB)
webgpu-semaphore-repro.zip (4.5 KB)

2

Update: this reproduces on every driver I’ve tested, across both the 595 and 615 branches, so it doesn’t look tied to one driver version. The newest test is on 615.71.09 (nvidia-driver-615-open 615.71.09-0ubuntu0.26.04.2), with the same Chrome 155.0.8059.39 and the same repro page with ?hdr=1.

Driver Runs Outcome
595.91.07 6 All lost the device, after 12k to 146k presents
595.99.02 3 Two lost the device, after 273k and 278k presents (337 and 350 s); one ran 10 minutes without failing (407k presents)
615.71.09 3 One lost the device after 269k presents (359 s); one ran 10 minutes without failing (456k presents); one ended at 167k presents when the GPU process exited (exit_code=8704) with no NVIDIA error, which may be unrelated

The 615.71.09 failure shows the same errors as before:

vkGetSemaphoreFdKHR failed with VK_ERROR_INITIALIZATION_FAILED
Error: NVIDIA: Unsupported external semaphore handle type

595.99.02 roughly doubled how many presents it takes to fail, and 615.71.09 fails at about the same point as 595.99.02. The Chromium side is tracked at Chromium

3

Update: I found the root cause, and it isn’t in the driver. It’s a race in the Linux kernel, now reported upstream: Making sure you're not a bot!

What happens:

  1. Under Chrome’s GPU sandbox, the GPU process can’t open files itself. Each open(“/dev/nvidiactl”) the driver makes goes to a broker process, which opens the file, sends the descriptor back over a new AF_UNIX SOCK_SEQPACKET socket and closes its end.
  2. A race in the kernel’s SOCK_SEQPACKET receive path can make the GPU process’s recvmsg() return 0 (end of file) while the broker’s reply is still queued. The kernel then discards the reply, descriptor included, when the socket is closed.
  3. Chrome reports that as ENOMEM, the driver’s open() fails, and the driver reports “Failed to allocate semaphore device memory” or “Unsupported external semaphore handle type”, or the GPU process crashes.

I traced five failures with bpftrace, and every one began with an open(“/dev/nvidiactl”) failing this way. In the runs where I also recorded the broker, its open() and its sendmsg() had both succeeded. A standalone C program, with no Chrome and no NVIDIA driver involved, loses messages the same way. The kernel report has the program and the full analysis.

That accounts for everything in my original post. It needs the sandbox because only then is a broker involved. Running Chrome under strace changed the timing enough to stop it. The driver version (595.91.07, 595.99.02 and 615.71.09 all reproduce it) only changed how often it happens.

Until a kernel fix reaches users, three things on the driver side would help:

  • The driver opens /dev/nvidiactl about 11 times per present (1.77 million opens over 162,000 presents in one strace). Under Chrome’s sandbox each open is a round trip to the broker and another chance to hit the race, so reusing descriptors would make it much rarer.
  • A failed open() is reported as a semaphore allocation or handle-type error, which made this look like a driver resource leak. Reporting the failed open() itself would have pointed at the cause much sooner.
  • Retrying an open of /dev/nvidiactl that fails with ENOMEM would avoid the failure, because the race doesn’t persist.

The Chromium issue, with a suggested fix on Chrome’s side: Chromium