How to export model using tlt-converter for Jetson Nano

Hi Morganh
The following error occurred after I replaced
error while loading shared libraries: libnvinfer_plugin.so.5: cannot open shared object file: No such file or directory

Can you paste full log including running command?

Hi Morganh
I use command and occur error like :
./deepstream-custom pgie_frcnn_uff_config.txt rtsp://admin:1111@10.0.0.155:554

(deepstream-custom:8623): GStreamer-WARNING **: 18:50:01.588: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_infer.so’: libnvinfer_plugin.so.5: cannot open shared object file: No such file or directory
One element could not be created. Exiting.

Hi 2528514903,
Could you please check below?
$ ldd tlt-converter
$ file libnvinfer_plugin.so.5

My bad, please add a tooltrain file for cross compilation.

cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_BIN_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchains/cmake_aarch64.toolchain

Hi Morganh
I Set -DCMAKE_TOOLCHAIN_FILE=…/cmake/toolchains/cmake_aarch64.toolchain,But it can’t work.
when I cmake,It occur problem like:
Targeting TRT Platform: aarch64
– CUDA version set to 10.0
– cuDNN version set to 7.5
– Protobuf version set to 3.0.0
– Using libprotobuf /workspace/TensorRT/build/third_party.protobuf/lib/libprotobuf.a
– ========================= Importing and creating target nvinfer ==========================
– Looking for library nvinfer
– Library that was found /pdk_files/tensorrt/lib/libnvinfer.so
– ==========================================================================================
– ========================= Importing and creating target nvuffparser ==========================
– Looking for library nvparsers
– Library that was found /pdk_files/tensorrt/lib/libnvparsers.so
– ==========================================================================================
– Protobuf proto/trtcaffe.proto → proto/trtcaffe.pb.cc proto/trtcaffe.pb.h
– /workspace/TensorRT/build/parsers/caffe

– ******** Summary ********
– CMake version : 3.14.4
– CMake command : /usr/local/bin/cmake
– System : Linux
– C++ compiler : /usr/bin/aarch64-linux-gnu-g++
– C++ compiler version : 7.4.0
– CXX flags : -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -Wall -Wnon-virtual-dtor
– Build type : Release
– Compile definitions : _PROTOBUF_INSTALL_DIR=/workspace/TensorRT/build/third_party.protobuf;ONNX_NAMESPACE=onnx2trt_onnx
– CMAKE_PREFIX_PATH :
– CMAKE_INSTALL_PREFIX : /pdk_files/tensorrt/lib/…
– CMAKE_MODULE_PATH :

– ONNX version : 1.3.0
– ONNX NAMESPACE : onnx2trt_onnx
– ONNX_BUILD_TESTS : OFF
– ONNX_BUILD_BENCHMARKS : OFF
– ONNX_USE_LITE_PROTO : OFF
– ONNXIFI_DUMMY_BACKEND : OFF

– Protobuf compiler :
– Protobuf includes :
– Protobuf libraries :
– BUILD_ONNX_PYTHON : OFF
– GPU_ARCH is not defined. Generating CUDA code for default SMs.
– Found TensorRT headers at /workspace/TensorRT/include
– Find TensorRT libs at /pdk_files/tensorrt/lib/libnvinfer.so;/pdk_files/tensorrt/lib/libnvinfer_plugin.so
– Configuring done
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_DEVICE_LINK_LIBRARY
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION
CMake Error: Error required internal CMake variable not set, cmake may not be built correctly.
Missing variable is:
CMAKE_CUDA_COMPILE_WHOLE_COMPILATION

– Generating done
– Build files have been written to: /workspace/TensorRT/build

It can be generated.
When I make -j$(nproc),it occur error like:

[ 43%] Linking CUDA device code CMakeFiles/nvinfer_plugin.dir/cmake_device_link
[ 44%] Linking CXX shared library …/out/libnvinfer_plugin.so
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/batchedNMSPlugin/gatherNMSOutputs.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/allClassNMS.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/bboxDeltas2Proposals.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/common.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/cropAndResizeKernel.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/decodeBBoxes.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/detectionForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/extractFgScores.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/gatherTopDetections.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/generateAnchors.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/gridAnchorLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/nmsLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/normalizeLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/permuteData.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/priorBoxLayer.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/proposalKernel.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/proposalsForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/regionForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/reorgForward.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/roiPooling.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/rproiInferenceFused.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/sortScoresPerClass.cu.o: No such file or directory
aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/common/kernels/sortScoresPerImage.cu.o: No such file or directory

aarch64-linux-gnu-g++: error: CMakeFiles/nvinfer_plugin.dir/cmake_device_link: No such file or directory
plugin/CMakeFiles/nvinfer_plugin.dir/build.make:731: recipe for target ‘out/libnvinfer_plugin.so.5.1.5.0’ failed
make[2]: *** [out/libnvinfer_plugin.so.5.1.5.0] Error 1
CMakeFiles/Makefile2:232: recipe for target ‘plugin/CMakeFiles/nvinfer_plugin.dir/all’ failed
make[1]: *** [plugin/CMakeFiles/nvinfer_plugin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs…

Hi 2528514903,
To speed up your case, I would provide you the direct way instead of cross build the libnvinfer_plugin.so .
Actually the TRT OSS can be built in Xavier directly if Xaiver is flashed with Jetpack.

$ git clone -b release/5.1 https://github.com/nvidia/TensorRT  && cd  TensorRT
$ git submodule update --init --recursive && export TRT_SOURCE=`pwd`
$ cd $TRT_SOURCE  
$ mkdir -p build && cd build
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/ && ./configure && make && sudo make install
$ /usr/local/bin/cmake .. -DTRT_BIN_DIR=`pwd`/out
$ make -j$(nproc)

If you meet issue as compile error · Issue #78 · NVIDIA/TensorRT · GitHub, please use below workaround.

$ wget https://github.com/google/protobuf/releases/download/v3.0.0/protobuf-cpp-3.0.0.tar.gz
$ cp protobuf-cpp-3.0.0.tar.gz  /home/nvidia/trt-oss/TensorRT/build/third_party.protobuf/src/
$ vim CMakeFiles/third_party.protobuf.dir/build.make
comment out line90
90         #cd /home/nvidia/trt-oss/TensorRT/build/third_party.protobuf/src && /usr/local/bin/cmake -P /home/nvidia/trt-oss/TensorRT/build/third_party.protobuf/src/third_party.protobuf-stamp/download-third_party.protobuf.cmake

Hi Moganh,

Would like to know method for cross-compiling as well, as I want to use the plugin on Jetson Nano. Compiling the entire plugin on nano don’t seems like a feasible solution.

Thanks

can i copy libnvinfer_plugin.so* to xavier?

Hi sdeep038,
Could you follow https://devtalk.nvidia.com/default/topic/1064407/transfer-learning-toolkit/how-to-export-model-using-tlt-converter-for-jetson-nano/post/5393964/#5393964 instead?

i have completed above steps and in /out directory there is libnvinfer_plugin.so* libarary,
so i have one question:
QUE:To use these plugins for the samples here, complile a new libnvinfer_plugin.so* and replace your system libnvinfer_plugin.so*.

what is the meaning “replace your system libnvinfer_plugin.so*” ??

Thanks,
Deep

Hi sdeep038,
That means replacing the default libs (/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*) with the new compiled lib.

where can i found new compiled lib ??
in /opt/lib ?

Hi sdeep038,
The lib locates at out folder when you finish https://devtalk.nvidia.com/default/topic/1064407/transfer-learning-toolkit/how-to-export-model-using-tlt-converter-for-jetson-nano/post/5393964/#5393964

$ ls pwd/out

Hi Morgan,

by mistake i have removed libnvinfer.so.5 please help me.

i am facing following error.

import tensorrt
Traceback (most recent call last):
File “”, line 1, in
File “/usr/lib/python2.7/dist-packages/tensorrt/init.py”, line 1, in
from .tensorrt import *
ImportError: libnvinfer.so.5: cannot open shared object file: No such file or directory

whai i did wrong??

where its(libnvinfer.so.5) located?

Please check if libnvinfer.so.5.1.6 is still available.

nvidia@nvidia:~$ ll /usr/lib/aarch64-linux-gnu/libnvinfer.so*
lrwxrwxrwx 1 root root        19 6月   5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer.so -> libnvinfer.so.5.1.6
lrwxrwxrwx 1 root root        19 6月   5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer.so.5 -> libnvinfer.so.5.1.6
-rw-r--r-- 1 root root 150375912 6月   5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer.so.5.1.6

No Moragan,

All libnvinfer.so* removed.

this is output:

ll /usr/lib/aarch64-linux-gnu/libnvinfer_*
lrwxrwxrwx 1 root root 26 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so → libnvinfer_plugin.so.5.1.5
lrwxrwxrwx 1 root root 28 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5 → libnvinfer_plugin.so.5.1.5.0
-rw-r–r-- 1 root root 2619200 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5.0

what should i do?

Suggest that you install TRT with Jetpack again.

is there any hack like copy .so from other machine ?

Yes, you can copy .so from the same machine as long as it is built by the same version of Jetpack.

Notice: We did not want to replace libnvinfer.so. For this topic, we are talking about replacing libnvinfer_plugin.so.

I have confusion.
then which location libnvinfer_plugin.so* lib should be available?

either /usr/local/cuda-10.0/lib64 or /usr/lib/aarch64-linux-gnu/ ???