Where's the documentation for cuDNN?

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

If I go to CUDA Deep Neural Network (cuDNN) | NVIDIA Developer under “references” all that’s linked are some fluff blog pieces about how great it is, but where’s the list of functions, description of what it does and explanation of parameters?

2

It’s in the download itself. You have to be a registered developer to access cuDNN, so the documentation is included in the download.

3

I’m not seeing it. My download only included a header file, a dll, and two object file libraries.

4

I was working from memory and misspoke, or else things changed.

When you are logged in as a registered developer, and are looking at this cuDNN download page:

[url]https://developer.nvidia.com/rdp/cudnn-download[/url]

You will see the following choices, from which you should select cuDNN User Guide to get the API level documentation:

cuDNN Download
NVIDIA cuDNN is a GPU-accelerated library of primitives for deep neural networks.

I Agree To the Terms of the cuDNN Software License Agreement
Download cuDNN v4 (Feb 10, 2016), for CUDA 7.0 and later.
cuDNN User Guide CLICK ON THIS LINK*****************

cuDNN Install Guide

cuDNN v4 Library for Linux

cuDNN v4 Library for Linux (IBM Power8)

cuDNN v4 Library for L4T (ARMv7)

cuDNN v4 Library for L4T (ARM64)

cuDNN v4 Library for Android (ARMv7)

cuDNN v4 Library for Android (ARM64)

cuDNN v4 Library for Windows

cuDNN v4 Library for OSX

cuDNN v4 Code Samples

cuDNN v4 Release Notes

cuDNN v4 Code Samples and User Guide (Deb)

5

Is there any code samples and User Guide for Windows? All I can see now are for Ubuntu 16.04 and Ubuntu 14.04…

6

I have install the cudnn code samples by “sudo dpkg -i libcudnn7-doc_7.3.0.29-1+cuda9.0_amd64.deb”
Then it only says,
Preparing to unpack libcudnn7-doc_7.3.0.29-1+cuda9.0_amd64.deb …
Unpacking libcudnn7-doc (7.3.0.29-1+cuda9.0) …
Setting up libcudnn7-doc (7.3.0.29-1+cuda9.0) …

and nothing else. Where is the code samples located at? How can I read the installed documentation?

7

It should be somewhere around /src/cudnn_samples_v7/ its in the cudnn installation part of the documentation.

I’m not going to lie, it isn’t the greatest source of just plain old building from scratch. Peter Goldsborough goes through some steps to set up a straight up feed forward with no back prop. (Convolutions with cuDNN – Peter Goldsborough)

For me the best thing to do was to build a toy network with a toy data set (mnist), and just play around with it.

I have a few tips assuming you are like me in that you have never used any popular frameworks like TensorFlow, but have built your own framework/network from scratch.

  1. Most of the time the alpha scalars need to be set to 1. And the beta scalars set to 0. Unless you are adding doing the cudnnConvolutionBackwardFilter function. In which you will want to set your alpha and beta to 1. I am sure there are more exceptions.
  2. Forward function output are generally y.
  3. Backward function output are generally dx.
  4. cudnnGetConvolutionForwardAlgorithm - just set the preference to CUDNN_CONVOLUTION_FWD_NO_WORKSPACE for your first time, and do the same for the backwarddata and backwardfilter.
  5. cudnnSoftmaxBackward - dy- will be the answers(labels) for the data, and alpha has to be set to -1.
  6. You will have to make your own optimization functions. For momentum you can update the weights of a layer by using cudnnOpTensor, but if you want to use something like adam. Then, you will have to make your own kernel.
2 Likes