Overlapping memcpy and compute is serialized on some machines

[CU_Steve]

See these two somewhat related posts:

Fewer concurrent kernels with Hardware Accelerated GPU Scheduling (HAGS)

I program in C++ using CUDA 12.2, Windows 10, and Visual Studio. I wrote a program to explore CUDA kernel concurrency. It launches long-running kernels (100 ms) into 48 different streams, and the kernels record the maximum number of concurrent kernels they observe. I added code to explore the effects of recording and querying events, querying streams, etc. With WDDM, my program reports a maximum of 32 concurrent kernels, which is to be expected for my GPU, a GTX 1070 Ti. When I add event re…

Increased time to synchronize streams via event record/wait when HAGS is enabled

Using Windows 10 with HAGS disabled leads to inter-kernel gaps of ~2 microseconds when the kernels are in different streams and synchronized via event record/waits. When I enable HAGS, I measure 20 to 30 microseconds between the end of one kernel and the start of another kernel when the kernels are in different streams and the streams are synchronized via events. This seems excessive for something that could be done entirely on the device. Is it to be expected? My host program alternates betw…