Periodic framing latencies found when moving from nvidia-driver 470 to 510 (realtime headless node apps)

There appears to be a periodic latency occurring every 512 frames when moving from nvidia-driver 470.199.02 to 510.108.03 (and newer). This characteristic has been observed for RTX 2080, 3060, & 4090 GPUs running on a Rocky Linux 9 workstation.

The testcase is a simple headless node application running in a console window (without X, Wayland, or a GDM running). The period latency has been observed under both OpenGL (using EGL) and Vulkan programs. The CPU running the OpenGL/Vulkan programs has been isolated and executed as root using a realtime priority.

The periodic latencies become noticable when running applications above 600 Hz (or so).

My use-case is for a deterministic realtime scene generation application, which is having problems with the newer 500-series of Nvidia-drivers. Any insight and suggested workarounds that can be provided will be greatly appreciated.

nvidia-bug-report.log (681.8 KB)