Are Tensors in CUDNN Row-major or Column-major?

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

Hello, I am developing a low level library in C to manage large machine learning models. In my library i would like to be totally compatible with NVidia whenever i can to reuse CUDNN + CUBLAS calls.
in CuBlas all operators take matrices in Column-major.

In CUDNN, i am not very sure, but i saw most operators are ROW-major? Is that true?

2

That’s row major. See Section 2.5.2 of cuDNN Developer Guide below (or search “row major”).

3

Thanks a lot