Cannot compile Torch C++ on Jetson Nano

[vassil17]

Hi,

I’ve been trying to get Libtorch to run on my Jetson Nano, but with no success so far. I have a docker container built from l4t-pytorch:r32.4.4-pth1.6-py3 (which matches the jetpack version on my nano).

When I try compiling my C++ code (with catkin as I’m using it for a ROS melodic package), I get an error that Torch cannot be found:

  Could not find a package configuration file provided by "Torch" with any of
  the following names:

I managed to solve this by running:

export CMAKE_PREFIX_PATH=/usr/local/lib/python3.6/dist-packages:$CMAKE_PREFIX_PATH

However, this then causes another error:

CMake Error at /usr/local/lib/python3.6/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:90 (message):
  Your installed Caffe2 version uses CUDA but I cannot find the CUDA
  libraries.  Please set the proper CUDA prefixes and / or install CUDA.

On my desktop version I had the same problem and I solved it by installing libtorch C++ API directly and adding its path to the CMAKE_PATH. How can I solve this on the Jetson?

[AastaLLL]

Hi,

Could you double-check if the CUDA library is available within the container first?
Thanks.

[vassil17]

Yes I get this:

root@3521ae3c834c:~# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_21:14:42_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

Interestingly, if I run it as

sudo docker run -it --rm --runtime nvidia --network host CONTAINER_NAME

I can compile a simple torch example. However, I then get an error that roscpp cannot be found.

If I run it normally without runtime nvidia, I can compile the ros part but I get the Caffe2 error.

[dusty_nv]

@vassil17

I can compile a simple torch example. However, I then get an error that roscpp cannot be found.

If I run it normally without runtime nvidia, I can compile the ros part but I get the Caffe2 error.

@vassil17 when you start the container, are you still sourcing the ROS overlay? i.e. with source /opt/ros/melodic/setup.bash or similar are you able to run other ROS stuff when using --runtime nvidia?

Without --runtime nvidia, the CUDA Toolkit won’t be mounted into the container. You can also set your default docker runtime to nvidia so that it’s used when you are building dockerfiles: https://github.com/dusty-nv/jetson-containers#building-the-containers

[vassil17]

@dusty_nv, yes, I can (and do) source ROS and I also run

export CMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'`:${CMAKE_PREFIX_PATH}

in order to tell catkin where to find torch.

Could the reason be that ROS melodic requires python 2 while I’m adding the python3 pytorch path to CMAKE?

EDIT: I tried building a mock ROS example (while using --runtime nvidia in the same container) but that does not depend on torch and it compiled successfully.

It seems more likely now that the problem is in the interaction between ROS melodic and pytorch. On my desktop I’ve successfully compiled the package by installing the libtorch C++ API directly and linking to it. However, as the Jetson is using an ARM64 architecture I cannot find a precompiled libtorch C++ for it (only the python wheels).

[dusty_nv]

@vassil17

However, as the Jetson is using an ARM64 architecture I cannot find a precompiled libtorch C++ for it (only the python wheels).

@vassil17 libtorch is included with the PyTorch wheel (e.g. under /usr/local/lib/python3.8/dist-packages/torch/lib)

@vassil17

Could the reason be that ROS melodic requires python 2 while I’m adding the python3 pytorch path to CMAKE?

Yea, my guess it’s having trouble finding it given that Melodic uses Python 2 whereas libtorch is installed under a Python 3 path. I’m not sure how to fix that, but presumably you could manually add the libtorch path to CMake.

It’s probably also worth mentioning that ROS Melodic is EOL, and should you be able to use Noetic, you could use dustynv/ros:noetic-pytorch-l4t-r32.7.1 from jetson-containers which already has PyTorch installed on top of ROS Noetic.

[vassil17]

@dusty_nv If I add the libtorch path manually it throws back the roscpp error (even if ros has just been sourced).

I already tried your noetic container but unfortunately unitree’s SDK only supports versions up to melodic.

For ROS I run with the -D_GLIBCXX_USE_CXX11_ABI=1 flag in my CMakeLists. Is the libtorch in the PyTorch wheel the Pre-cxx11 ABI or cxx11 ABI version , and do you think this could be causing a problem?

Thanks again for your help!

[dusty_nv]

@vassil17

If I add the libtorch path manually it throws back the roscpp error (even if ros has just been sourced).

I wonder if when you add the libtorch path, it is overwriting other paths that ROS environment may have set, as opposed to appending it? Although from this post, it appears that you are pre-pending it. Honestly I’m not sure why that happens, and I don’t personally use PyTorch with Melodic due to the aforementioned Python incompatibility.

@vassil17

Is the libtorch in the PyTorch wheel the Pre-cxx11 ABI or cxx11 ABI version , and do you think this could be causing a problem?

I believe it’s with the CXX11 ABI version (at least on my system with JetPack 4.6 and PyTorch 4.6 it was)

You can check this by checking the torch/share/cmake/Torch/TorchConfig.cmake file under the directory in which your PyTorch wheel was installed to:

cat ~/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake | grep CXX11

>>> set(TORCH_CXX_FLAGS "-D_GLIBCXX_USE_CXX11_ABI=1")