Dear 
SivaRamaKrishnaNV
I could not find aarch64 tensorrt.so for python so I just figured I’ll try the one on my Jetson Orin. So far it works, if there is an “official” way to get that lib I’d be happy to try it,
Now as far as compiling cuDNN and TensorRT code on the target. These are the steps that I followed.
-
Tar all libcudnn* and libnvinfer* in target lib/ aarch64-linux-gnu and /etc/alternative in and all cudnn.* and nvinfer* include/ aarch64-linux-gnu on the host. These are the libs and headers. The idea is to untar these on the target in /usr/include/aarch64-linux-gnu and /usr/lib/aarch64-linux-gnu and /etc/alternatives. Do not copy or scp since tar preserves all the symbolic links.
-
Once this is done then the target has all the headers and libs for compiling cuDNN and TensorRT apps.
OpenCV on Target
-
Download openCV and openCV-Contrib source, I downloaded version 4.5.4 Release OpenCV 4.5.4 · opencv/opencv · GitHub
-
Install dependencies: sudo apt install -y cmake libavcodec-dev libavformat-dev libavutil-dev libeigen3-dev libglew-dev libgtk2.0-dev libgtk-3-dev libjpeg-dev libpng-dev libpostproc-dev libswscale-dev libtbb-dev libtiff5-dev libv4l-dev libxvidcore-dev libx264-dev qt5-default zlib1g-dev python-dev python-numpy python-py python-pytest python3-dev python3-py python3-pytest libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libopenblas-dev
-
Here is part of a shell script to compile with cuDNN and CUDA on Orin drive, you may need to change the install directory
OpenCV defines
OPENCV_VERSION=4.5.4
ARCH_BIN=8.7
CUDNN_VERSION=8.6
NUM_JOBS=$(nproc)
OPENCV_DOWNLOAD_DIR=$HOME/opencv
OPENCV_DIR=opencv-$OPENCV_VERSION
OPENCV_CONTRIB_DIR=opencv_contrib-$OPENCV_VERSION
OPENCV_BUILD_DIR=$OPENCV_DIR/build
OPENCV_GITHUB_REPO=OpenCV · GitHub
OPENCV_REPO=$OPENCV_GITHUB_REPO/opencv/archive/refs/tags/$OPENCV_VERSION.tar.gz
OPENCV_CONTRIB_REPO=$OPENCV_GITHUB_REPO/opencv_contrib/archive/refs/tags/$OPENCV_VERSION.tar.gz
OPENCV_INSTALL_LOCATION=/usr
cd $OPENCV_DOWNLOAD_DIR/$OPENCV_BUILD_DIR
cmake -D CMAKE_BUILD_TYPE=RELEASE
-D CMAKE_INSTALL_PREFIX=${OPENCV_INSTALL_LOCATION}
-D WITH_CUDA=ON
-D WITH_CUDNN=ON
-D CUDNN_VERSION=${CUDNN_VERSION}
-D CUDA_ARCH_BIN=${ARCH_BIN}
-D CUDA_ARCH_PTX=“”
-D ENABLE_FAST_MATH=ON
-D CUDA_FAST_MATH=ON
-D WITH_CUBLAS=ON
-D WITH_LIBV4L=ON
-D WITH_V4L=ON
-D WITH_GSTREAMER=ON
-D WITH_GSTREAMER_0_10=OFF
-D WITH_QT=OFF
-D WITH_GTK=ON
-D WITH_GTK_2_X=ON
-D WITH_OPENGL=ON
-D BUILD_opencv_python2=ON
-D BUILD_opencv_python3=ON
-D BUILD_TESTS=OFF
-D BUILD_PERF_TESTS=OFF
-D OPENCV_GENERATE_PKGCONFIG=YES
-D PYTHON_EXECUTABLE=$(which python3)
-D OPENCV_PYTHON3_INSTALL_PATH=$(python3 -c “from distutils.sysconfig import get_python_lib; print(get_python_lib())”)
-D OPENCV_EXTRA_MODULES_PATH=…/…/${OPENCV_CONTRIB_DIR}/modules
-D CPACK_PACKAGE_VERSION=${OPENCV_VERSION}
-D EXTRAMODULES_VCSVERSION=${OPENCV_VERSION}
-D OPENCV_VCSVERSION=${OPENCV_VERSION}
…/
make -j${NUM_JOBS}
make install
Test OpenCV in python:
python3 -c “import cv2; print(‘OpenCV version:’, str(cv2.version)); print(cv2.getBuildInformation())”
Pytorch:
- Here is a script that builds the latest pytorch but it can be changed to download whatever version that you need.
Also , I think the cmake version “out of the box” on the target 3.16 is not sufficient for pytorch compile so I install cmake 3.27 from source:
Installing | CMake
The option TORCH_CXX_FLAGS=D_GLIBCXX_USE_CXX11_ABI=1 is something we need in regards to compile and link with ROS (Long story ) , you can omit that
git clone https://github.com/pytorch/pytorch.git && \
cd pytorch && \
pip3 install pyyaml==3.13 && \
pip3 install -r requirements.txt && \
sudo apt update
sudo apt-get install apt-utils
sudo apt-get install python3-setuptools
USE_OPENCV=1 \
BUILD_TORCH=ON \
CMAKE_PREFIX_PATH="/usr/bin/" \
LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:/usr/local/lib:$LD_LIBRARY_PATH \
CUDA_BIN_PATH=/usr/local/cuda-11.4/bin \
CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-11.4/ \
CUDNN_LIB_DIR=/usr/lib/aarch64-linux-gnu \
CUDNN_INCLUDE_DIR=/usr/include/aarch64-linux-gnu \
CUDNN_LIBRAY=/usr/lib/aarch64-linux-gnu/libcudnn.so \
CUDA_HOST_COMPILER=cc \
USE_CUDA=1 \
USE_CUDNN=1 \
USE_NNPACK=1 \
TORCH_CXX_FLAGS=D_GLIBCXX_USE_CXX11_ABI=1 \
CC=cc \
CXX=c++ \
TORCH_CUDA_ARCH_LIST="8.7" \
TORCH_NVCC_FLAGS="-Xfatbin -compress-all" \
python3 setup.py bdist_wheel
Install the Python wheel (includes LibTorch)
pip3 install dist/*.whl
Clean up resources
rm -fr pytorch
I also had to set up swap file on target