Reason for warp stall not clear or unknown

Developer Tools Nsight Compute
,
1

Hey there,

I have recently done a CUDA programming course where we were guided through some example CUDA code, like matrix multiplication. When analyzing the kernels with Nsight Compute, I noticed that ncu reports a common warp stall position that I don’t understand. I would like some help with either identifying the specific architectural reason for the stall (which I cannot see) or understanding why ncu reports it like this.

Here is a screenshot of the reported information in Nsight Compute, focused on the relevant code:

image
image1920×1048 321 KB

All the instructions on the right panel correspond to the computation of the multiplication and addition of the highlighted line on the left panel. Therefore, you can see at the bottom of the right panel the two float instructions (FMUL and FADD), and all the previous instructions correspond to the load of the two operands (with the previous calculation of the indexing address).

There are two clear spots where Nsight Compute locates warp stalls. The bottom one is the most obvious to me: The FMUL gets stalled because it needs to wait for the loading of the second operand (previous instruction LD.E), which is stored in R2. But I don’t see the reason for the first one, on the highlighted line on the right panel.

To my understanding, that specific line (instr. IMAD) starts the calculation of the address to access the second operand, so the previous instruction is the load of the first operand (stored in R18; we can see how then FMUL uses it). With this rationale, there isn’t any clear dependency between these two instructions. However, the IMAD instruction reports a very high number of warp stalls on a “Long Scoreboard”, and when you click on the input (bottom pane), the stalls are attributed to the previous instruction: the LD.E.

  • What is causing the warps to stall at this point? Why is it saying it’s because a Scoreboard originated on the previous instruction, if there is no data dependency?

After thinking a bit, I found a possible reason that still leaves me with many doubts. Could this be a Write-after-Read dependency on R2? I see that the IMAD writes the result in R2, while the previous load seems to use R2 somehow (I don’t understand the notation, but it’s written in red). So, my thesis is that warps stall at the IMAD instruction because the result cannot be written in R2 until the previous instruction, which is a consumer of R2, finishes. If this is the scenario, I have two questions:

  1. Isn’t this something that the compiler should figure out better, and use a different register (increasing the usage of registers by one) instead of making all warps stall there? Or is this something to be controlled/adjusted by the programmer? I used CUDA 11.8, btw
  2. I would expect Nsight Compute to report this in a different way, if this is the reason. With this naming, saying it is waiting for a Long Scoreboard, I assume that the stalls are due to data dependencies. However, this WaR thing is completely different. And still, it is the main cause of warp stalls (in this toy code).

Thanks for reading this far! I wish I could share the ncu-rep or the code, but the file types are not allowed.

Everybody is welcome to join the discussion even if you don’t have an answer for my questions :)

Marc