Disable Tensor Cores in cuBLAS functions explicity

Hi,
I’m using CUDA 11.0 on DGX system(A100), Ubuntu 20.04. Is there any way to disable the involve Tensor Cores on cublasDgemm function?
I’m noticed that the performance this function outperform the declared Peak Performance 9.7 TFLOPS. I think it’s because of the inclusion Tensor Cores on cublasDgemm by default. It seems right?
Then I need disable the behavior and disable usage Tensor Cores in cublasDgemm. It’s possible?

cublasSetMathMode(CUBLAS_PEDANTIC_MATH) not work, not any effect. I’m reading 2.1.11. Tensor Core Usage at document cuBLAS :: CUDA Toolkit Documentation - " … unless they are explicitly disabled by selecting pedantic compute modes in cuBLAS (see cublasSetMathMode(), cublasMath_t). "

Thank in advance!

Depending on matrix size, FP64 Tensor Cores should be reaching near peak performance (i.e., 19.5 TFLOPs).

This is no way to disable FP64 TCs in cuBLAS, as they are IEEE compliant.

1 Like

Okay. Thank you very much. It’c clear now.
And what can to say about SGEMM? It’s uses Tensor Cores?
Thank in advance!

SGEMM/CGEMM don’t use TCs and are also 19.5 TFLOPs peak.

2 Likes