Cutlass not working in ARM-based machine

834352945 · April 11, 2023, 12:07pm

Hello, I’m running Megatron-LM in my ARM-based machine with four A100 card, But The performance is not good as X86-based machine, After collecting the performance data using Nvidia NSYS, I find out that cublas-gemm is used in ARM while cutlass-gemm is used in X86，my pytorch version is 1.11 and I don’t know why this weild thing is happen. Is there anything uncompatible with Cublas–CublasLt-cutlass in my arm machine? Can anybody help?

Best Wishes.

834352945 · April 12, 2023, 3:34am

After setting the CUBLASLT_LOG_LEVEL=5，I found that all of the heuristicResult is setting to [21] in X86-based machine，while half of the heuristicResult is setting to [0] and [1] in ARM-based machine. Can anybody help?

Topic		Replies	Views
CUTLASS 1-bit Tensorcore GEMM result error on SM86 CUDA Developer Tools	0	587	December 30, 2020
cublasSgemm error? CUDA Programming and Performance	2	5268	July 12, 2007
Bad performance of cublas for extremely small matrix multiplication? GPU-Accelerated Libraries cublas	4	937	May 1, 2024
CUBLAS performance under toolkit 4.1 CUDA Programming and Performance	2	4673	March 16, 2012
Understanding cutlass GEMM hierarchy GPU-Accelerated Libraries cutlass	1	3529	October 14, 2021
Why performance is worse with CUBLAS- than with kernel-function GPU-Accelerated Libraries	3	959	September 5, 2019
Low CuBLAS performance CUDA Programming and Performance	3	437	January 15, 2019
[Solved]Same Cublas Functions work slower on the GTX1080 from GTX 960M GPU-Accelerated Libraries	3	849	June 5, 2018
cublas problem with very big matrixes and cublasDgemm slow CUDA Programming and Performance	2	990	February 23, 2017
why cublasHgemm is slower more than cublasSgemm when I use? GPU-Accelerated Libraries	6	4312	January 22, 2019

Cutlass not working in ARM-based machine

Related topics