Error while building TensorRT OSS 8.0.1

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Building for TensorRT version: 8.0.1, library version: 8
– Targeting TRT Platform: x86_64
– CUDA version set to 11.3.1
– cuDNN version set to 8.2
– Protobuf version set to 3.0.0
– Using libprotobuf /tmp/TensorRT/build/third_party.protobuf/lib/libprotobuf.a
– ========================= Importing and creating target nvinfer ============= =============
– Looking for library nvinfer
– Library that was found /usr/lib/x86_64-linux-gnu/libnvinfer.so
– ============================================================================= =============
– ========================= Importing and creating target nvuffparser ========= =================
– Looking for library nvparsers
– Library that was found /usr/lib/x86_64-linux-gnu/libnvparsers.so
– ============================================================================= =============
– GPU_ARCHS is not defined. Generating CUDA code for default SMs: 35;53;61;70;7 5;80
– Protobuf proto/trtcaffe.proto → proto/trtcaffe.pb.cc proto/trtcaffe.pb.h
– /tmp/TensorRT/build/parsers/caffe
Generated: /tmp/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx_onnx2trt_ onnx-ml.proto
Generated: /tmp/TensorRT/build/parsers/onnx/third_party/onnx/onnx/onnx-operators _onnx2trt_onnx-ml.proto

– ******** Summary ********
– CMake version : 3.17.0
– CMake command : /tmp/cmake-3.17.0/bin/cmake
– System : Linux
– C++ compiler : /usr/bin/g++
– C++ compiler version : 9.3.0
– CXX flags : -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_ oss -Wall -Wno-deprecated-declarations -Wno-unused-function -Wnon-virtual-dtor
– Build type : Release
– Compile definitions : _PROTOBUF_INSTALL_DIR=/tmp/TensorRT/build;ONNX_NAME SPACE=onnx2trt_onnx
– CMAKE_PREFIX_PATH :
– CMAKE_INSTALL_PREFIX : /usr/lib/aarch64-linux-gnu/…
– CMAKE_MODULE_PATH :

– ONNX version : 1.6.0
– ONNX NAMESPACE : onnx2trt_onnx
– ONNX_BUILD_TESTS : OFF
– ONNX_BUILD_BENCHMARKS : OFF
– ONNX_USE_LITE_PROTO : OFF
– ONNXIFI_DUMMY_BACKEND : OFF
– ONNXIFI_ENABLE_EXT : OFF

– Protobuf compiler :
– Protobuf includes :
– Protobuf libraries :
– BUILD_ONNX_PYTHON : OFF
– Found TensorRT headers at /usr/include/
– Find TensorRT libs at /usr/lib/x86_64-linux-gnu/libnvinfer.so;/usr/lib/x86_64 -linux-gnu/libnvinfer_plugin.so
– Adding new sample: sample_algorithm_selector
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_char_rnn
– - Parsers Used: uff;caffe;onnx
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_dynamic_reshape
– - Parsers Used: onnx
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_fasterRCNN
– - Parsers Used: caffe
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: sample_googlenet
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_int8
– - Parsers Used: caffe
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: sample_int8_api
– - Parsers Used: onnx
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_mlp
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_mnist
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_mnist_api
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_nmt
– - Parsers Used: none
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_onnx_mnist
– - Parsers Used: onnx
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_reformat_free_io
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_ssd
– - Parsers Used: caffe
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: sample_uff_fasterRCNN
– - Parsers Used: uff
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: sample_uff_maskRCNN
– - Parsers Used: uff
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: sample_uff_mnist
– - Parsers Used: uff
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_uff_plugin_v2_ext
– - Parsers Used: uff
– - InferPlugin Used: OFF
– - Licensing: opensource
– Adding new sample: sample_uff_ssd
– - Parsers Used: uff
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: sample_onnx_mnist_coord_conv_ac
– - Parsers Used: onnx
– - InferPlugin Used: ON
– - Licensing: opensource
– Adding new sample: trtexec
– - Parsers Used: caffe;uff;onnx
– - InferPlugin Used: OFF
– - Licensing: opensource
– Configuring done
– Generating done
– Build files have been written to: /tmp/TensorRT/build

The last lines of the make -j output are:

[ 54%] Linking CXX static library …/libnvinfer_plugin_static.a
[ 54%] Built target nvinfer_plugin_static
make: *** [Makefile:169: all] Error 2

The cMaket error log shows as follows:

Determining if the pthread_create exist failed with the following output:
Change Dir: /tmp/TensorRT/build/CMakeFiles/CMakeTmp

Run Build Command:“/usr/bin/make” “cmTC_acdb9/fast”
/usr/bin/make -f CMakeFiles/cmTC_acdb9.dir/build.make CMakeFiles/cmTC_acdb9.dir/build
make[1]: Entering directory ‘/tmp/TensorRT/build/CMakeFiles/CMakeTmp’
Building CXX object CMakeFiles/cmTC_acdb9.dir/CheckSymbolExists.cxx.o
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -std=c++11 -o CMakeFiles/cmTC_acdb9.dir/CheckSymbolExists.cxx.o -c /tmp/TensorRT/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
Linking CXX executable cmTC_acdb9
/tmp/cmake-3.13.0/bin/cmake -E cmake_link_script CMakeFiles/cmTC_acdb9.dir/link.txt --verbose=1
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss CMakeFiles/cmTC_acdb9.dir/CheckSymbolExists.cxx.o -o cmTC_acdb9
/usr/bin/ld: CMakeFiles/cmTC_acdb9.dir/CheckSymbolExists.cxx.o: in function main': CheckSymbolExists.cxx:(.text+0x1f): undefined reference to pthread_create’
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_acdb9.dir/build.make:87: cmTC_acdb9] Error 1
make[1]: Leaving directory ‘/tmp/TensorRT/build/CMakeFiles/CMakeTmp’
make: *** [Makefile:121: cmTC_acdb9/fast] Error 2

File /tmp/TensorRT/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
(void)argv;
#ifndef pthread_create
return ((int*)(&pthread_create))[argc];
#else
(void)argc;
return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /tmp/TensorRT/build/CMakeFiles/CMakeTmp

Run Build Command:“/usr/bin/make” “cmTC_34380/fast”
/usr/bin/make -f CMakeFiles/cmTC_34380.dir/build.make CMakeFiles/cmTC_34380.dir/build
make[1]: Entering directory ‘/tmp/TensorRT/build/CMakeFiles/CMakeTmp’
Building CXX object CMakeFiles/cmTC_34380.dir/CheckFunctionExists.cxx.o
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create -std=c++11 -o CMakeFiles/cmTC_34380.dir/CheckFunctionExists.cxx.o -c /tmp/TensorRT/build/CMakeFiles/CheckLibraryExists/CheckFunctionExists.cxx
Linking CXX executable cmTC_34380
/tmp/cmake-3.13.0/bin/cmake -E cmake_link_script CMakeFiles/cmTC_34380.dir/link.txt --verbose=1
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_34380.dir/CheckFunctionExists.cxx.o -o cmTC_34380 -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
make[1]: *** [CMakeFiles/cmTC_34380.dir/build.make:87: cmTC_34380] Error 1
make[1]: Leaving directory ‘/tmp/TensorRT/build/CMakeFiles/CMakeTmp’
make: *** [Makefile:121: cmTC_34380/fast] Error 2

Environment

TensorRT Version: 8.0.1.6
GPU Type: Nvidia RTX 2060
Nvidia Driver Version: 470.42.01
CUDA Version: 11.3
CUDNN Version: 8.2.1
Operating System + Version: Ubuntu 20.04 LTS
Python Version (if applicable): 3.8.10
TensorFlow Version (if applicable): 2.5
Cmake version: 3.17

Steps To Reproduce

git clone -b master GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators. TensorRT
cd TensorRT
git submodule update --init --recursive
mkdir -p build && cd build
cmake … -DTRT_LIB_DIR=/usr/lib/aarch64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CUDA_COMPILER:PATH=/usr/local/cuda/bin/nvcc
make -j$(nproc)

2

Hi,
Please refer to the installation steps from the below link if in case you are missing on anything

However suggested approach is to use TRT NGC containers to avoid any system dependency related issues.

In order to run python sample, make sure TRT python packages are installed while using NGC container.
/opt/tensorrt/python/python_setup.sh

In case, if you are trying to run custom model, please share your model and script with us, so that we can assist you better.
Thanks!

4

Thanks a lot for the quick answer. I will try using the NGC container.
I am trying to run a custom model for object detection. It´s a Resnet_50_v1 trained in tensorflow 2.0. You can find the onnx model with this link:
https://portahnos-my.sharepoint.com/:f:/g/personal/echambouleyron_porta_com_ar/El4OIcAEA9xLlqPZPuzkzdIBMilY9CaSn4m6ifAPPK-9nQ?e=QhNGyH.

I could succesfully build previuos TensorRT version but NonMaxSuppression op was not supported when trying to convert onnnx to Trt engine. Therefore, I was suggested to build the latest version which apparently does support this op.

Thanks a lot in advance.

5

@emiliochambu,

Thanks for sharing the ONNX model. We tried building engine and looks like model is using UINT8 datatype which is currently not supported by TensorRT. Please make sure model has TRT supported datatypes.
Also fyi TRT 8.0 is not yet available in NGC container, we recommend you please setup TRT latest version locally by following installation guide shared above.

Unsupported ONNX data type: UINT8 (2)
[07/08/2021-11:51:55] [E] [TRT] ModelImporter.cpp:726: ERROR: input_tensor:0:230 In function importInput:
[8] Assertion failed: convertDtype(onnxDtype.elem_type(), &trtDtype) && “Failed to convert ONNX date type to TensorRT data type.”

6

Thanks a lot for the quick answer. I did succesfully convert the datatype using onnx-graphsurgeon, but later on Non-max-supression op error showed up.

Regarding TensorRT installation, I am trying to build TensorRT OSS following this guide: GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.. As the guide suggests, I have already succesfully installed TensorRT 8.0.1.6 using the debian installation, but I am having no success in build TRT OSS which is crucial as it contains the plugin I need for converting Non max supression op to TensorRT layer.

7

Is there a difference in installing TensorRT 8.0.1.6 using debian or Tar? Should I try with the latter?

8

Here you can find the model with the datatypes changed:
https://portahnos-my.sharepoint.com/:f:/g/personal/echambouleyron_porta_com_ar/El4OIcAEA9xLlqPZPuzkzdIBMilY9CaSn4m6ifAPPK-9nQ?e=7N4IGj

9

@emiliochambu,

Thank you for sharing the model. When we tried building engine facing issue related to TopK layer.

input: “Unsqueeze__879:0”
output: “StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField/TopKV2:0”
output: “StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField/TopKV2:1”
name: “StatefulPartitionedCall/Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/SortByField/TopKV2”
op_type: “TopK”
attribute {
name: “sorted”
i: 1
type: INT
}

[E] [TRT] ModelImporter.cpp:723: — End node —
[E] [TRT] ModelImporter.cpp:726: ERROR: builtin_op_importers.cpp:4292 In function importTopK:
[8] Assertion failed: (inputs.at(1).is_weights()) && “This version of TensorRT only supports input K as an initializer.”

This is because your “K” of Topk layer in your model may be dynamic value.
TensorRT runtime only support the K value is constant. It means you must make sure your output is not dynamic.
Similar issue for your reference,

10

any revelant info for this? I´ve been struggling a lot and still with no success. @spolisetty

11

@emiliochambu,

Have you tried above and able resolve above mentioned issue ?

12

I have not tried it yet. I am still struggling with building TensorRT OSS.

13

@emiliochambu,

Could you please share complete error logs you’re facing while building TensorRT OSS.

14

Here you can find the complete outputs: https://portahnos-my.sharepoint.com/:f:/g/personal/echambouleyron_porta_com_ar/EpNALdo9XNpBn2eH7s-TSx0BeXkuOTTryjtqQYiyhXIBdA?e=338QJY

15

https://portahnos-my.sharepoint.com/:f:/g/personal/echambouleyron_porta_com_ar/EpNALdo9XNpBn2eH7s-TSx0BeXkuOTTryjtqQYiyhXIBdA?e=338QJY

16

I tried setting parsers off as suggested here with no succes. Here you can find the output and error logs

17

This seems to be the problem:
Output

bin/sh: 1: python: not found
make[2]: *** [parsers/onnx/third_party/onnx/CMakeFiles/gen_onnx_proto.dir/build.make:89: parsers/onnx/third_party/onnx/onnx/onnx_onnx2trt_onnx-ml.proto] Error 127
make[1]: *** [CMakeFiles/Makefile2:1986: parsers/onnx/third_party/onnx/CMakeFiles/gen_onnx_proto.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs…

18

Hi @emiliochambu,

Thanks for sharing logs, We are trying to analyze error logs. Looks like some dependencies issue. Not able to find python problem you mentioned above.

Have you added python3 as alias python in your environment, please let us know in case you’re trying on some docker container.

Please make sure you’ve installed all dependencies mentioned here GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.,

We also suggest you to try downgrading your python from 3.8 to 3.7 if you still face this issue.

Thank you.

19

Many thanks for answering.
I am not building on docker and I did add python3 as alias in my environment.
I´ve realized I dont have installed python-libnvinfer nor python-libnvinfer-dev in my pc. Could this be related to the python problem?

20

@spolisetty

21

I ´ve downloaded python-libnvinfer and python-libnvinfer-dev using sudo apt-get install python3-libnvinfer-dev. The files are now located at:

  • /usr/share/doc/python3-libnvinfer
  • /usr/share/doc/python3-libnvinfer-dev

But I am still getting “python not found” with this final output:
[ 54%] Linking CXX shared library …/libnvinfer_plugin.so
[ 54%] Built target nvinfer_plugin
make: *** [Makefile:172: all] Error 2