Does global access to L1 cache include a shard memory component

Developer Tools Nsight Compute
1

image
According to roofline, L1 bandwidth can be measured at about 1T/s

image
image1123×334 19.5 KB

The bandwidth of global access is also 1T/s(execution time is 1ms)
image
image1025×438 22.1 KB

However, as you can see from the figure, there are 6.72 million shared memory access requests, which are separate from the global access.
So is shared memory access actually part of the global access, or else how to explain that L1’s bandwidth is 1T/s

2

The bandwidth for L1 is 128 B/cycle on almost all chips. I agree that L1 Achieved of 2.4 TFLOPS/s / 2.34 FLOP/byte is ~1 TB/s; however, this is the achieved bandwidth not the theoretical bandwidth. The memory diagram does not have the L1 bandwidth which would be two interfaces on the left side of the L1/TEX Cache.

In the Memory Table (L1) it is showing you are at 11.8% of Peak indicating a much higher peak bandwidth. The challenge is the %Peak is in L1 wavefronts which is not the same as bandwidth back to the register file.

3

Sorry,maybe I didn’t state my question clearly. From the last entry in the memory table, you can see that the total global memory access is about 1GB. The execution time of the program is 1ms, so the L1 achieved bandwidth is 1TB/S, which is consistent with the results obtained from roofline. But as you can see from the last diagram, there are 6.72m requests for shared memory, so the total data transferred by L1 should be global memory access and shared memory access, which should be greater than 1GB. So achieved bandwidth should be greater than 1TB/S. If that’s the case, it’s not consistent with roofline.

4

The NCU install has a directory “sections” that contains the section files. For L1 bytes is the return bandwidth of LSU. For measured bytes what is counted differ between each chip. For GH100 and GB100 it should be noted that the L1 does not include shared memory read by the Tensor cores as this does not increment the LSU writeback to RF.

SpeedOfLight_HierarchicalSingleRooflineChart.section

 Metrics {
   Label: "L1/TEX peak writeback cycles"
   Name: "l1tex__lsu_writeback_active.sum.peak_sustained"
   Filter {
     MaxArch: CC_72
   }
 }
 Metrics {
   Label: "L1/TEX peak local/global writeback cycles"
   Name: "l1tex__lsu_writeback_active_mem_lg.sum.peak_sustained"
   Filter {
     MinArch: CC_75
     MaxArch: CC_89
   }
   Options {
     Name: "l1tex__lsu_writeback_active_mem_lgds.sum.peak_sustained"
     Filter {
       MinArch: CC_90
     }
   }
 }

What is included in the bytes

Architectures  Included
Volta          global, local, shared
Turing         global, local
Ampere         global, local
Ada            global, local
Hopper+        global, local, shared, dsmem
5

I conducted the test on v100, so L1 bytes should include shared memory

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.