cuDNN version incompatibility

[jelena1]

RuntimeError: cuDNN version incompatibility: PyTorch was compiled against (8, 6, 0) but found runtime version (8, 4, 1). PyTorch already comes bundled with cuDNN. One option to resolving this error is to ensure PyTorch can find the bundled cuDNN.

JetPack Version: 5.0.2-b231
PyTorch Vesion: 2.0.0+nv23.5

I follow instructions from your PyTorch for Jetson

Can you help me?

[dusty_nv]

Hi @jelena1, for JetPack 5.0.2 / L4T R35.1 you should be able to use the PyTorch wheels from here: https://developer.download.nvidia.com/compute/redist/jp/v502/pytorch/

I’ve updated that PyTorch thread to indicate that the PyTorch 2.0 wheels are built for JetPack 5.1+, sorry about that. If you would like to use those, please upgrade your version of JetPack-L4T.

[jelena1]

Thank you.
I’ve installed [torch-1.13.0a0+410ce96a.nv22.12-cp38-cp38-linux_aarch64.whl] but now I’ve got:
print('cuDNN version: ' + str(torch.backends.cudnn.version()))

cuDNN version: None

[dusty_nv]

@jelena1 can you confirm that you still have cuDNN installed? This was from a system running JetPack 5.0.2:

ls -ll /usr/lib/aarch64-linux-gnu/libcudnn*
lrwxrwxrwx 1 root root         39 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so -> /etc/alternatives/libcudnn_adv_infer_so
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8 -> libcudnn_adv_infer.so.8.4.1
-rw-r--r-- 1 root root  145413864 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_infer.so.8.4.1
-rw-r--r-- 1 root root  148002466 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_infer_static.a
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_infer_static_v8.a -> libcudnn_adv_infer_static.a
lrwxrwxrwx 1 root root         39 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so -> /etc/alternatives/libcudnn_adv_train_so
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8 -> libcudnn_adv_train.so.8.4.1
-rw-r--r-- 1 root root  109150880 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_train.so.8.4.1
-rw-r--r-- 1 root root  111646964 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_train_static.a
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_adv_train_static_v8.a -> libcudnn_adv_train_static.a
lrwxrwxrwx 1 root root         39 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so -> /etc/alternatives/libcudnn_cnn_infer_so
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8 -> libcudnn_cnn_infer.so.8.4.1
-rw-r--r-- 1 root root  843208168 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
-rw-r--r-- 1 root root 1088636654 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer_static.a
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_infer_static_v8.a -> libcudnn_cnn_infer_static.a
lrwxrwxrwx 1 root root         39 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so -> /etc/alternatives/libcudnn_cnn_train_so
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8 -> libcudnn_cnn_train.so.8.4.1
-rw-r--r-- 1 root root  106098456 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_train.so.8.4.1
-rw-r--r-- 1 root root  157062790 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_train_static.a
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_cnn_train_static_v8.a -> libcudnn_cnn_train_static.a
lrwxrwxrwx 1 root root         39 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so -> /etc/alternatives/libcudnn_ops_infer_so
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8 -> libcudnn_ops_infer.so.8.4.1
-rw-r--r-- 1 root root   96698512 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_infer.so.8.4.1
-rw-r--r-- 1 root root  100458612 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_infer_static.a
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_infer_static_v8.a -> libcudnn_ops_infer_static.a
lrwxrwxrwx 1 root root         39 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so -> /etc/alternatives/libcudnn_ops_train_so
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8 -> libcudnn_ops_train.so.8.4.1
-rw-r--r-- 1 root root   79435520 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_train.so.8.4.1
-rw-r--r-- 1 root root   80072514 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_train_static.a
lrwxrwxrwx 1 root root         27 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn_ops_train_static_v8.a -> libcudnn_ops_train_static.a
lrwxrwxrwx 1 root root         29 Jul 12  2022 /usr/lib/aarch64-linux-gnu/libcudnn.so -> /etc/alternatives/libcudnn_so
lrwxrwxrwx 1 root root         17 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn.so.8 -> libcudnn.so.8.4.1
-rw-r--r-- 1 root root     149992 May 20  2022 /usr/lib/aarch64-linux-gnu/libcudnn.so.8.4.1

If you continue having issues with it, I would recommend trying the l4t-pytorch container which already comes with this pre-installed.