IMMA roofline analysis in NSight Compute

[m_ali102]

As far as I understand, the SpeedOfLight_HierarchicalRoflineTensorCore section and other roofline sections are only for floating point data types.

Is there a way to plot hierarchical roofline for imma instr, i.e. int8 operations?

After looking at the section files, I’m a little confused about which metrics to use for imma.

One example of my confusion is to get both peak and actual operations/cycle in fp rooflines, Nsight multiplies the instr/cycles *512.

How does that apply to imma? Should I multiply by 512 as well or other different factor? And how does that 512 is calculated?

Thank you.

[m_ali102]

I came across this

Why the Compute Throughput's value is different from the actual Performance / Peak Performance

Thank you for pointing to the Roofline Model on NVIDIA GPUs lab. That was very helpful. But I am curious about the “512” number in the 512 x sm__inst_executed_pipe_tensor.sum FLOP calculation. I think this is specific to V100? since the h884 instruction is doing 512 FLOPs per instruction. And for A100, it should be 4096 for h16816, is this correct? Another issue I have is when I collect information for one kernel in cutlass_profiler as the following, I am getting DRAM read that is less than the…

So, I guess 512 isn’t the right factor for a100 for both fp16 and in8. Please let us know what to use for a100 for different precisions.
My estimation is 4096 and apparently the user in the mentioned post suggests the same as well.

[jmarusarz]

As of today, we don’t have a published roofline model for integer in Nsight Compute, including what factors and metrics would be needed. We have had this request before and are considering it along with other features.