A Question about how Ampere/Lovelace (RTX 3000/4000, GA10X/AD10X) cards handle Warp Dispatching

[ian.mccabe42]

I see, when searching before I found a response you provided before that sounds a bit conflicting.

About the relationship between warp and tensor_core

typically, if e.g a FMUL, FADD, or FFMA instruction is issued warp-wide, then we need 32 such calculations to satisfy the needs of the warp. Since each “cuda core” can support 1 fma per cycle, then to handle the needs of the warp for a single FFMA instruction, we would need 32 of these. If there are 32 available in a particular SMSP, then the instruction could be scheduled using all 32 of those in a single clock cycle. If the SMSP does not have 32, but has instead, 16, then it will require 2…

In such case, for the ALU performing the FP32 (aka the “CUDA core”), and in the case of Ampere gaming and later, they do “technically” have 32 capable FP32 units per SMSP. However, it’s not fully clear to me since it’s Ampere and not Turing, with Turing only having 16 of those capable units per SMSP (64 across the entire SM) meanwhile ampere gaming and later have 128 capable per SM (or 32 per SMSP).