CUDA : 10.2
cuDNN version : 7.6.5
GPU : Titan RTX & Volta 100
OS : ubuntu 18.04
I’m trying to implement Conv3D in cuDNN. I have the following questions:
Conv3D works for CUDNN_TENSOR_NCHW. But, it doesn’t run on tensor core. I have set CUDNN_TENSOR_OP_MATH, inputs are half1, multiples of 8 (which is not necessary in cuDNN 7.6.5,) and convolution algorithm is CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PRECOMP_GEMM.
Conv3D doesn’t work for CUDNN_TENSOR_NHWC. Input dim - {N, D, H, W, IN_C} and filter dim - {K, 3, 3, 3, IN_C}
For cuDNN 7.6, 3d NHWC is not fully supported. So that is expected.
cuDNN v8 have better 3D support, you can try using the latest version:
cuDNN Release Notes :: NVIDIA Deep Learning SDK Documentation
Added 3D convolutions support of NHWC and improved heuristics and kernels for Tensor Cores in NCHW resulting in performance improvements for VNet, UNet-Medical and UNet-Industrial.
OK. I understand NHWC is not supported in cuDNN 7.6.5.
What about NCHW? I read somewhere in the forum that volta gpus should be able use tensor cores for 3D convolutions. I did try it on V100 and it didn’t run on tensor cores.