Version Incompatibilities are driving me nuts

[mischmerz]

I am trying to install kokoro with CUDA. It runs fine on cpu, well not fine (slow!) but it runs. And so starts version hell. You update pyTorch for the machine, now libcuddn doesn’t work. So - you upgrade libcuddn and now libcublas complains. You install libcublas latest, but it’s version 12 - and which doesn’t seem to be compatible with libcuddn9.

BRAIN FREEZE.

Example: Installing PyTorch for Jetson Platform - NVIDIA Docs

The downloaded script bombs out, because it only supports CUDA versions until 12.4 . Easy fix. Changed it to include 12.6 Continue with the document until:

torch-2.0.0+nv23.05-cp38-cp38-linux_aarch64.whl is not a supported wheel on this platform

Thank you very much. Reading other user’s command it turns out, that those wheels were built for Python 3.8, but my machine runs 3.10.12 … ups … compatibility again. Oh well … sorry.

So PLEASE:

I need an environment in which things that depend on each other play nice. CUDA, torch, libcudnn, libcublas and all those little thingies that are supposed to work together. Orins are developer versions, but in order to develop something, one needs an environment that kinda works.

Thank you.
Michaela

[AastaLLL]

Hi,

You can find the package that supports Jetson 6.1/6.2 below:

https://pypi.jetson-ai-lab.dev/jp6/cu126

These packages are built on the Jetson directly so it’s compatible with the default libraries.
Thanks.

[mischmerz]

Thanks. I installed

torch-2.6.0rc1

without problems. Now we have this:

PyTorch was compiled against (9, 4, 0) but found runtime version (9, 3, 0). Library path points to

libass.so.9      libcudnn_adv.so.9  libcudnn_engines_precompiled.so.9       libcudnn_graph.so.9      libcudnn_ops.so.9  libgs.so.9   libunity-extras.so.9  libwacom.so.9
libcfitsio.so.9  libcudnn_cnn.so.9  libcudnn_engines_runtime_compiled.so.9  libcudnn_heuristic.so.9  libcudnn.so.9      libmtp.so.9  libunity.so.9

I even installed

tinycudann-1.7

But that didn’t make a difference. What else is required to get that thing to not complain? I guess you understand how frustrating this is?

m.

[AastaLLL]

Hi,

Manually upgrading cuDNN to 9.4.0 can fix this issue.

Since the package is wrapped with a repository configure file, you will need to upgrade the cuDNN with subitems.

Default:

$ dpkg --list | grep cudnn
ii  cudnn                                        9.3.0-1                                     arm64        NVIDIA CUDA Deep Neural Network library (cuDNN)
ii  cudnn-local-tegra-repo-ubuntu2204-9.4.0      1.0-1                                       arm64        cudnn-local-tegra repository configuration files
ii  cudnn9                                       9.3.0-1                                     arm64        NVIDIA CUDA Deep Neural Network library (cuDNN)
ii  cudnn9-cuda-12                               9.3.0.75-1                                  arm64        NVIDIA cuDNN for CUDA 12
ii  cudnn9-cuda-12-6                             9.3.0.75-1                                  arm64        NVIDIA cuDNN for CUDA 12.6
ii  libcudnn9-cuda-12                            9.3.0.75-1                                  arm64        cuDNN runtime libraries for CUDA 12.6
ii  libcudnn9-dev-cuda-12                        9.3.0.75-1                                  arm64        cuDNN development headers and symlinks for CUDA 12.6
ii  libcudnn9-samples                            9.3.0.75-1                                  all          cuDNN samples
ii  libcudnn9-static-cuda-12                     9.3.0.75-1                                  arm64        cuDNN static libraries for CUDA 12.6
ii  nvidia-cudnn                                 6.2+b77                                     arm64        NVIDIA CUDNN Meta Package
ii  nvidia-cudnn-dev                             6.2+b77                                     arm64        NVIDIA CUDNN dev Meta Package

Upgrade to cuDNN 9.4.0:

$ wget https://developer.download.nvidia.com/compute/cudnn/9.4.0/local_installers/cudnn-local-tegra-repo-ubuntu2204-9.4.0_1.0-1_arm64.deb
$ sudo dpkg -i cudnn-local-tegra-repo-ubuntu2204-9.4.0_1.0-1_arm64.deb
$ sudo cp /var/cudnn-local-tegra-repo-ubuntu2204-9.4.0/cudnn-*-keyring.gpg /usr/share/keyrings/
$ sudo apt-get update
$ sudo apt install cudnn=9.4.* cudnn9=9.4.* cudnn9-cuda-12=9.4.* cudnn9-cuda-12-6=9.4.*  libcudnn9-cuda-12=9.4.* libcudnn9-dev-cuda-12=9.4.* libcudnn9-samples=9.4.* libcudnn9-static-cuda-12=9.4.*

Then verify

$ dpkg --list | grep cudnn
ii  cudnn                                        9.4.0-1                                     arm64        NVIDIA CUDA Deep Neural Network library (cuDNN)
ii  cudnn-local-tegra-repo-ubuntu2204-9.4.0      1.0-1                                       arm64        cudnn-local-tegra repository configuration files
ii  cudnn9                                       9.4.0-1                                     arm64        NVIDIA CUDA Deep Neural Network library (cuDNN)
ii  cudnn9-cuda-12                               9.4.0.58-1                                  arm64        NVIDIA cuDNN for CUDA 12
ii  cudnn9-cuda-12-6                             9.4.0.58-1                                  arm64        NVIDIA cuDNN for CUDA 12.6
ii  libcudnn9-cuda-12                            9.4.0.58-1                                  arm64        cuDNN runtime libraries for CUDA 12.6
ii  libcudnn9-dev-cuda-12                        9.4.0.58-1                                  arm64        cuDNN development headers and symlinks for CUDA 12.6
ii  libcudnn9-samples                            9.4.0.58-1                                  all          cuDNN samples
ii  libcudnn9-static-cuda-12                     9.4.0.58-1                                  arm64        cuDNN static libraries for CUDA 12.6

After upgrading cuDNN, the PyTorch cuDNN backend can work correctly:

 python3
Python 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'2.6.0-rc1'
>>> print(torch.backends.cudnn.version())
90400

Thanks.

[mischmerz]

Thanks for your help. Everything is running now.