# How to analysis the stall wait in this HMMA case **URL:** \ **Category:** Nsight Compute\ **Created:** [October 22, 2024, 8:08am UTC](https://forums.developer.nvidia.com/t/how-to-analysis-the-stall-wait-in-this-hmma-case/310727 "2024-10-22T08:08:42Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![lijinghaiwhu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lijinghaiwhu](https://forums.developer.nvidia.com/u/lijinghaiwhu)\ **Post date:** [October 22, 2024, 8:08am UTC](https://forums.developer.nvidia.com/t/how-to-analysis-the-stall-wait-in-this-hmma-case/310727/1 "2024-10-22T08:08:42Z")
I use cutlass to write a A16W4(fp16 A matrix, int4 B matrix) gemm kernel. Then I use nsight compute to analyze it. In source page, I get ![image](https://global.discourse-cdn.com/nvidia/original/4X/1/f/4/1f49d79d51e8325584c8a562260cbfc5d8a7ec22.jpeg) And I have read [this](https://forums.developer.nvidia.com/t/how-to-know-my-kernel-if-pipeline-parallel-by-nsight-compute/249106/2) to understand stalled wait, it said ![image](https://global.discourse-cdn.com/nvidia/original/4X/0/2/5/025527d60b22b915101a9046cf1aa1bd0bb95cf4.jpeg) so my question is 1. the issue rate of HMMA instructions of ampere architecture. 2. I was wondering why there are so many stalled\_waits instead of stalled\_math\_throttle between HMMA instructions since there are no data dependency between two consecutive instrs。 I can provide the ncu file if it is needed. ---
**Author:** ![Greg](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/greg/32/14043_2.png) [@Greg](https://forums.developer.nvidia.com/u/Greg)\ **Post date:** [October 22, 2024, 4:28pm UTC](https://forums.developer.nvidia.com/t/how-to-analysis-the-stall-wait-in-this-hmma-case/310727/2 "2024-10-22T16:28:38Z")
> 1. the issue rate of HMMA instructions of ampere architecture. To my knowledge the issue rate of each MMA instruction variant is not documented. From the screenshot it can be estimated for every 7 wait samples there is 1 selected sample ==\> 8 cycles for HMMA.16816.F32 on this chip. > 1. I was wondering why there are so many stalled\_waits instead of stalled\_math\_throttle between HMMA instructions since there are no data dependency between two consecutive instrs。 stalled\_math\_throttle only occurs if the kernel has multiple warp on the same sub-partition that are using the same math pipeline. cutlass often tries to have a single warp issue a sequence of MMAs then switch to a different warp to issue a sequence of MMA instructions. In this case the second warp will not be stalled on the MMA pipe. The screenshot does not have sufficient information for that conclusion. The not\_selected stall on MMA implies the MMA pipe is free; however, the warp scheduler choose a different warp to issue on that cycle. ---
**Author:** ![lijinghaiwhu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lijinghaiwhu](https://forums.developer.nvidia.com/u/lijinghaiwhu)\ **Post date:** [October 23, 2024, 1:07am UTC](https://forums.developer.nvidia.com/t/how-to-analysis-the-stall-wait-in-this-hmma-case/310727/3 "2024-10-23T01:07:50Z")
> [@Greg](#): > > To my knowledge the issue rate of each MMA instruction variant is not documented. From the screenshot it can be estimated for every 7 wait samples there is 1 selected sample ==\> 8 cycles for HMMA.16816.F32 on this chip. How was this conclusion reached? Could you describe the specific derivation process to me? And what does warp scheduler do during the 7 wait sample. > [@Greg](#): > > cutlass often tries to have a single warp issue a sequence of MMAs then switch to a different warp to issue a sequence of MMA instructions. launch config of this kernel is grid = (10, 14, 1), block = (256, 1, 1). If I understand you correctly, it has 8 warps in a block, in another word, 2 warps for each sub-partition. And to my knowledge, the hardware will decide which warp to be scheduled. So what does “cutlass often tries to” here mean? > [@Greg](#): > > In this case the second warp will not be stalled on the MMA pipe. The screenshot does not have sufficient information for that conclusion. This is my ncu file, with m =1200, n = 3548, k = 18944. Change to .log since .ncu-rep can not be uploaded. [gemm.log](https://forums.developer.nvidia.com/uploads/short-url/rhhH1jXTFr2lknB8dg2XtuVNC42.log) (20.2 MB) ---
**Author:** ![lijinghaiwhu](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lijinghaiwhu](https://forums.developer.nvidia.com/u/lijinghaiwhu)\ **Post date:** [October 31, 2024, 2:57am UTC](https://forums.developer.nvidia.com/t/how-to-analysis-the-stall-wait-in-this-hmma-case/310727/4 "2024-10-31T02:57:11Z")
Stilling waiting for you help.