GEMM stage on ampere

571987523 · March 12, 2024, 12:30pm

[QST] Does the more numStages I set, the better performance I get?

opened 07:14AM - 08 Mar 23 UTC

closed 01:43AM - 09 Mar 23 UTC

question ? - Needs Triage

Hello! I'm working for auto tuning for examples/36. I write some scripts to gene…rate all the kernels configuration based on https://github.com/NVIDIA/cutlass/blob/master/tools/library/scripts/generator.py. According to https://github.com/NVIDIA/cutlass/issues/617, some configurations will require more shared memory than available. So I need to remove some configurations. https://github.com/NVIDIA/cutlass/issues/617 says I can calculate the amount of shared memory needed for a GEMM as follows: ```((threadblockM * threadblockK * sizeof(ElementA)) + (threadblockN * threadblockK * sizeof(ElementB))) * numStages```. I think it may be a good idea to decrease the numStages so I can still keep some of the tiling size. Does the more numStages I set, the better performance it will be (so that I can set only one numStages for each tile size and avoid a long tuning time)? If not, do you have some empirical range for numStages?

I was wondering why the minimum stage is 3 on ampere?
And can you give me a simple pseudocode example for multi-sage?

Topic		Replies	Views
Name explain about stages_64x3 GPU-Accelerated Libraries	2	219	July 20, 2024
CUTLASS: Fast Linear Algebra in CUDA C++ Technical Blog	13	2245	September 9, 2024
Understanding cutlass GEMM hierarchy GPU-Accelerated Libraries cutlass	1	3742	October 14, 2021
How to determine a good ThreadblockShape in CUTLASS CUDA Programming and Performance	0	838	November 18, 2021
my speedy SGEMM CUDA Programming and Performance	91	277231	May 29, 2013
The larger block the better? CUDA Programming and Performance	8	529	March 25, 2024
Shared memory matrix multiplication not working CUDA Programming and Performance	6	174	October 11, 2024
Warp stall reduded but performance not improved CUDA Programming and Performance nsight	0	449	October 23, 2022
Matrix Multiplication (GEMM) Register Tiling and Shared Memory Bandwidth Bound CUDA Programming and Performance kernel	4	2093	December 18, 2022
Understanding how threads work and their number in Gauss elimination method CUDA Programming and Performance	0	348	September 13, 2022

GEMM stage on ampere

Related topics