Hi,
I am trying to understand the behavior of Nsight Compute PM Sampling when multiple CUDA kernels are active on the same GPU. In my experiment, I have two CUDA kernels launched on the same time.
When the second kernel is profiled, the Nsight Compute GUI PM Sampling timeline appears to show that the profiled context runs continuously until the profiled kernel finishes. I expected that the GPU periodically switches between the two contexts, but in the Nsight Compute report, the PM Sampling timeline looks like one uninterrupted active region for the profiled workload rather than showing context-switch slots between the two CUDA kernels. Does my profiling process look normal? Is there any Nsight Compute option to allow context switching between different CUDA kernels while PM Sampling is active?
I would also expect the profiling duration of a single kernel to be different when the kernel is profiled while other kernels are active on the GPU. However, in the Nsight Compute GUI, I do not observe any noticeable difference between profiling the kernel alone and profiling it while another kernel is already running.
My setup is:
GPU: NVIDIA GeForce RTX 4070 Laptop GPU
Driver version: 580.159.03
CUDA version: 12.4
Nsight Compute version: 2024.1.1.0
As detailed in the documentation, Nsight Compute collects a context switch trace for PM sampling.
The timeline documentation also explains the options available in the timeline’s context (right click) menu, such as Enable/Disable Context Switch Filter.
Thanks for the comment.
I understand that Nsight Compute allows the context-switch filter to be enabled or disabled, but my question is different. I do not observe any context switches occurring during the execution of the kernel that is being profiled. Does Nsight Compute serialize the execution of the profiled kernel? As shown in the attached image, there are no visible context-switch intervals during the profiled kernel execution.
If this question is for two kernels launched from the same process under ncu - yes, ncu serializes by default execution of kernels in kernel replay mode, so that each kernel can be profiled and replayed deterministically, see here.
If you want to profile both kernels at the same time, you need to use a range-based replay mode, which also implies that you need to define the range in your source via cu(da)ProfilerStart/Stop or NVTX.
I mean that, for two kernels launched from different processes/CUDA contexts, does Nsight Compute prevent context switching by serializing the execution of each kernel, or is some other mechanism involved?
To the best of my knowledge, CUDA context switching is controlled by the GPU driver/scheduler. However, in my experiment, while one kernel is being profiled with Nsight Compute, no other CUDA context appears to be scheduled during its execution. It looks as if Nsight Compute prevents another context from running until the profiled kernel finishes.
