How to export model using tlt-converter for Jetson Nano

Thank you for your help !!

Thank you Morganh. I’ll try.

Hi Morganh,
when I run docker build -f docker/ubuntu-18.04.Dockerfile --build-arg CUDA_VERSION=10.1 --tag=tensorrt .,It occur error like:
Sending build context to Docker daemon 170.8MB
Step 1/10 : ARG CUDA_VERSION=10.1
Step 2/10 : FROM nvcr.io/nvidia/cuda:${CUDA_VERSION}-cudnn7-devel-ubuntu18.04
10.0-cudnn7-devel-ubuntu18.04: Pulling from nvidia/cuda
35c102085707: Pull complete
251f5509d51d: Pull complete
8e829fe70a46: Pull complete
6001e1789921: Pull complete
9f0a21d58e5d: Pull complete
8810fcda1e6e: Pull complete
d701a76e3193: Pull complete
34be232fb7a6: Downloading [==========> ] 103.1MB/504.4MB
7e62b1ed3410: Downloading [=======> ] 106.8MB/704.9MB
34be232fb7a6: Downloading [==========> ] 103.6MB/504.4MB
47526c6630b9: Downloading [====> ] 44.25MB/494.7MB
34be232fb7a6: Downloading [==========> ] 104.1MB/504.4MB
7e62b1ed3410: Downloading [=======> ] 107.9MB/704.9MB
34be232fb7a6: Pull complete
7e62b1ed3410: Pull complete
47526c6630b9: Pull complete
Digest: sha256:1f7bd3857100de0e7e05d35a2d1cec1ae64823096680ad5d58a9f0ae35e86f6e
Status: Downloaded newer image for nvcr.io/nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04
—> eaf2ceb9de1a
Step 3/10 : LABEL maintainer=“NVIDIA CORPORATION”
—> Running in bd2ac788df1e
Removing intermediate container bd2ac788df1e
—> db93b2546b1b
Step 4/10 : RUN apt-get update && apt-get install -y --no-install-recommends libcurl4-openssl-dev wget zlib1g-dev git pkg-config python3 python3-pip
—> Running in 69ba46b65c28
standard_init_linux.go:207: exec user process caused “exec format error”
The command ‘/bin/sh -c apt-get update && apt-get install -y --no-install-recommends libcurl4-openssl-dev wget zlib1g-dev git pkg-config python3 python3-pip’ returned a non-zero code: 1

it occur erro:standard_init_linux.go:207: exec user process caused “exec format error”

Hi 2528514903,
Please build docker via your x86_64 Linux PC instead of Xavier.

Hi Morganh
My target platform is jetson tx2,It is an ARM CPU.Do I need cross compile?
If it is needed,How I set cmake parameter?

Hi 2528514903,
Building docker and TRT OSS is in order to get the libnvinfer_plugin.so* only.
Then copy it to Xavier to replace the old one.

Please see details in https://github.com/NVIDIA-AI-IOT/deepstream_4.x_apps for the purpose.

Hi Morganh
I When I finished compiling, I get four files like:
libnvinfer_plugin.so
libnvinfer_plugin.so.5.1.5
libnvinfer_plugin.so.5.1.5.0
libnvinfer_plugin_static.a

The original four files in the system is:
libnvinfer_plugin.so
libnvinfer_plugin.so.5
libnvinfer_plugin.so.5.1.6
libnvinfer_plugin_static.a

They don’t have exactly the same file name,How do I replace them?

You can replace it since I recall it works.

Hi Morganh
The following error occurred after I replaced
error while loading shared libraries: libnvinfer_plugin.so.5: cannot open shared object file: No such file or directory

Can you paste full log including running command?

Hi Morganh
I use command and occur error like :
./deepstream-custom pgie_frcnn_uff_config.txt rtsp://admin:1111@10.0.0.155:554

(deepstream-custom:8623): GStreamer-WARNING **: 18:50:01.588: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_infer.so’: libnvinfer_plugin.so.5: cannot open shared object file: No such file or directory
One element could not be created. Exiting.

Hi 2528514903,
Could you please check below?
ldd tlt-converter file libnvinfer_plugin.so.5

My bad, please add a tooltrain file for cross compilation.

cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_BIN_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchains/cmake_aarch64.toolchain

Hi Morganh
I Set -DCMAKE_TOOLCHAIN_FILE=…/cmake/toolchains/cmake_aarch64.toolchain,But it can’t work.
when I cmake,It occur problem like:
Targeting TRT Platform: aarch64
– CUDA version set to 10.0
– cuDNN version set to 7.5
– Protobuf version set to 3.0.0
– Using libprotobuf /workspace/TensorRT/build/third_party.protobuf/lib/libprotobuf.a
– ========================= Importing and creating target nvinfer ==========================
– Looking for library nvinfer
– Library that was found /pdk_files/tensorrt/lib/libnvinfer.so
– ==========================================================================================
– ========================= Importing and creating target nvuffparser ==========================
– Looking for library nvparsers
– Library that was found /pdk_files/tensorrt/lib/libnvparsers.so
– ==========================================================================================
– Protobuf proto/trtcaffe.proto -> proto/trtcaffe.pb.cc proto/trtcaffe.pb.h
– /workspace/TensorRT/build/parsers/caffe

– ******** Summary ********
– CMake version : 3.14.4
– CMake command : /usr/local/bin/cmake
– System : Linux
– C++ compiler : /usr/bin/aarch64-linux-gnu-g++
– C++ compiler version : 7.4.0
– CXX flags : -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -Wall -Wnon-virtual-dtor
– Build type : Release
– Compile definitions : _PROTOBUF_INSTALL_DIR=/workspace/TensorRT/build/third_party.protobuf;ONNX_NAMESPACE=onnx2trt_onnx
– CMAKE_PREFIX_PATH :
– CMAKE_INSTALL_PREFIX : /pdk_files/tensorrt/lib/…
– CMAKE_MODULE_PATH :

– ONNX version : 1.3.0
– ONNX NAMESPACE : onnx2trt_onnx
– ONNX_BUILD_TESTS : OFF
– ONNX_BUILD_BENCHMARKS : OFF
– ONNX_USE_LITE_PROTO : OFF
– ONNXIFI_DUMMY_BACKEND : OFF

– Protobuf compiler :
– Protobuf includes :
– Protobuf libraries :
– BUILD_ONNX_PYTHON : OFF
– GPU_ARCH is not defined. Generating CUDA code for default SMs.
– Found TensorRT headers at /workspace/TensorRT/include
– Find TensorRT libs at /pdk_files/tensorrt/lib/libnvinfer.so;/pdk_files/tensorrt/lib/libnvinfer_plugin.so
– Configuring done
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_DEVICE_LINK_LIBRARY
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION

– Generating done
– Build files have been written to: /workspace/TensorRT/build

It can be generated.
When I make -j$(nproc),it occur error like:

[ 43%] Linking CUDA device code CMakeFiles/nvinfer_plugin.dir/cmake_device_link
[ 44%] Linking CXX shared library …/out/libnvinfer_plugin.so
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/batchedNMSPlugin/gatherNMSOutputs.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/allClassNMS.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/bboxDeltas2Proposals.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/common.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/cropAndResizeKernel.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/decodeBBoxes.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/detectionForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/extractFgScores.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/gatherTopDetections.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/generateAnchors.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/gridAnchorLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/nmsLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/normalizeLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/permuteData.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/priorBoxLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/proposalKernel.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/proposalsForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/regionForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/reorgForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/roiPooling.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/rproiInferenceFused.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/sortScoresPerClass.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/sortScoresPerImage.cu.o: No such file or directory

aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/cmake_device_link: No such file or directory
plugin/CMakeFiles/nvinfer_plugin.dir/build.make:731: recipe for target ‘out/libnvinfer_plugin.so.5.1.5.0’ failed
make[2]: *** [out/libnvinfer_plugin.so.5.1.5.0] Error 1
CMakeFiles/Makefile2:232: recipe for target ‘plugin/CMakeFiles/nvinfer_plugin.dir/all’ failed
make[1]: *** [plugin/CMakeFiles/nvinfer_plugin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs…

Hi 2528514903,
To speed up your case, I would provide you the direct way instead of cross build the libnvinfer_plugin.so .
Actually the TRT OSS can be built in Xavier directly if Xaiver is flashed with Jetpack.

$ git clone -b release/5.1 https://github.com/nvidia/TensorRT  && cd  TensorRT
$ git submodule update --init --recursive && export TRT_SOURCE=`pwd`
$ cd $TRT_SOURCE  
$ mkdir -p build && cd build
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/ && ./configure && make && sudo make install
$ /usr/local/bin/cmake .. -DTRT_BIN_DIR=`pwd`/out
$ make -j$(nproc)

If you meet issue as https://github.com/NVIDIA/TensorRT/issues/78, please use below workaround.

$ wget https://github.com/google/protobuf/releases/download/v3.0.0/protobuf-cpp-3.0.0.tar.gz
$ cp protobuf-cpp-3.0.0.tar.gz  /home/nvidia/trt-oss/TensorRT/build/third_party.protobuf/src/
$ vim CMakeFiles/third_party.protobuf.dir/build.make
comment out line90
90         #cd /home/nvidia/trt-oss/TensorRT/build/third_party.protobuf/src && /usr/local/bin/cmake -P /home/nvidia/trt-oss/TensorRT/build/third_party.protobuf/src/third_party.protobuf-stamp/download-third_party.protobuf.cmake

Hi Moganh,

Would like to know method for cross-compiling as well, as I want to use the plugin on Jetson Nano. Compiling the entire plugin on nano don’t seems like a feasible solution.

Thanks

can i copy libnvinfer_plugin.so* to xavier?

Hi sdeep038,
Could you follow https://devtalk.nvidia.com/default/topic/1064407/transfer-learning-toolkit/how-to-export-model-using-tlt-converter-for-jetson-nano/post/5393964/#5393964 instead?

i have completed above steps and in /out directory there is libnvinfer_plugin.so* libarary,
so i have one question:
QUE:To use these plugins for the samples here, complile a new libnvinfer_plugin.so* and replace your system libnvinfer_plugin.so*.

what is the meaning “replace your system libnvinfer_plugin.so*” ??

Thanks,
Deep

Hi sdeep038,
That means replacing the default libs (/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*) with the new compiled lib.

where can i found new compiled lib ??
in /opt/lib ?