From some information I found online, it seemed like the CUDNN library assigns a convolution algorithm (including FFT-based and Winograd algorithm) depending on the parameters of the Pytorch’s Conv2d function. I am wondering is there a way to set the CUDNN library to run only the specified algorithm every time when Conv2d function is called? Or is there a way to know which convolution algorithm is run when Conv2d function is called? I am currently running on Google Colab with Pytorch, and CUDA Version 11.2.
Also, does the CUDNN library accelerates other parts of a CNN other than the convolution algorithm? like the pooling layer, activation function, and fully connected layers.