Hello experts,
I am trying to build libnvinfer plugin version 8.0.1 for using with JetPack 4.6 on Nano. I tried to follow the steps written here:
I could successfully build and install CMake following step1. But when I tried to follow step 2, upon issuing the command “$ make nvinfer_plugin -j$(nproc)” I get the following error:
Does anybody have any idea how to get pass of this?
Thanks in advance!
Which branch of TRT OSS branch did you git clone?
Thanks for your reply.
6.0 GA 4.6 TRT 8.0.1 release/8.0
This is what I tried.
I am having all sorts of trouble. Nothing is really working for me. I tried the same for older jetpack when I build plugin version 7.1.3. Everything went smoothly. Now with 8.0.1 I neither can build it on x86 nor on Jetson Nano.
Need help if anybody could succssfully build it.
Thanks,
Please check if you install CUDA correctly in your x86 or Nano.
Ok, I am now trying from scratch. I have freshly install ubuntu 18.04.
Installed Cuda toolkit 11.3 (cuda-repo-ubuntu1804-11-3-local_11.3.0-465.19.01-1_amd64.deb) which came with Nvidia driver version 465 following the steps described in CUDA Toolkit 11.3 Downloads | NVIDIA Developer
Then I install Tensor RT version 8.0.1 (TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB local repo package] (Log in | NVIDIA Developer)
After that I tried to build TensorRT OSS plugin from source code (deepstream_tao_apps/TRT-OSS/x86 at master · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub). As described in this link, I first compiled and install CMAKE 3.19.4. and then tried to build TensorRT OSS Plugin (6.0 GA TRT 8.0.1 release/8.0). However, I am having trouble when I execute $ $HOME/install/bin/cmake .. -DTRT_LIB_DIR=/usr/lib/x86_64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=pwd/out
Building for TensorRT version: 8.0.1, library version: 8
– The CXX compiler identification is GNU 7.5.0
– The CUDA compiler identification is NVIDIA 11.3.58
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/g++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Detecting CUDA compiler ABI info
– Detecting CUDA compiler ABI info - done
– Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
– Detecting CUDA compile features
– Detecting CUDA compile features - done
– Targeting TRT Platform: x86_64
– CUDA version set to 11.3.1
– cuDNN version set to 8.2
– Protobuf version set to 3.0.0
– Looking for C++ include pthread.h
– Looking for C++ include pthread.h - found
– Performing Test CMAKE_HAVE_LIBC_PTHREAD
– Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
– Looking for pthread_create in pthreads
– Looking for pthread_create in pthreads - not found
– Looking for pthread_create in pthread
– Looking for pthread_create in pthread - found
– Found Threads: TRUE
– Found PkgConfig: /usr/bin/pkg-config (found version “0.29.1”)
– Checking for one of the modules ‘zlib’
CMake Error at /home/ashiq/install/share/cmake-3.19/Modules/FindPkgConfig.cmake:805 (message):
None of the required ‘zlib’ found
Call Stack (most recent call first):
third_party/zlib.cmake:18 (pkg_search_module)
CMakeLists.txt:98 (include)
– Protobuf compiler :
– Protobuf includes :
– Protobuf libraries :
– BUILD_ONNX_PYTHON : OFF
– Found TensorRT headers at /home/ashiq/TensorRT/include
– Find TensorRT libs at /usr/lib/x86_64-linux-gnu/libnvinfer.so;/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so
– Found TENSORRT: /home/ashiq/TensorRT/include
– Adding new sample: sample_algorithm_selector
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_char_rnn
– - Parsers Used: uff;caffe;onnx
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_dynamic_reshape
– - Parsers Used: onnx
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_fasterRCNN
– - Parsers Used: caffe
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: sample_googlenet
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_int8
– - Parsers Used: caffe
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: sample_int8_api
– - Parsers Used: onnx
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_mlp
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_mnist
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_mnist_api
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_nmt
– - Parsers Used: none
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_onnx_mnist
– - Parsers Used: onnx
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_reformat_free_io
– - Parsers Used: caffe
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_ssd
– - Parsers Used: caffe
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: sample_uff_fasterRCNN
– - Parsers Used: uff
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: sample_uff_maskRCNN
– - Parsers Used: uff
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: sample_uff_mnist
– - Parsers Used: uff
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_uff_plugin_v2_ext
– - Parsers Used: uff
– - InferPlugin Used: OFF
– - Licensing: samples
– Adding new sample: sample_uff_ssd
– - Parsers Used: uff
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: sample_onnx_mnist_coord_conv_ac
– - Parsers Used: onnx
– - InferPlugin Used: ON
– - Licensing: samples
– Adding new sample: trtexec
– - Parsers Used: caffe;uff;onnx
– - InferPlugin Used: OFF
– - Licensing: samples
– Configuring incomplete, errors occurred!
See also “/home/ashiq/TensorRT/build/CMakeFiles/CMakeOutput.log”.
See also “/home/ashiq/TensorRT/build/CMakeFiles/CMakeError.log”.
Performing C++ SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make cmTC_285cc/fast && /usr/bin/make -f CMakeFiles/cmTC_285cc.dir/build.make CMakeFiles/cmTC_285cc.dir/build
make[1]: Entering directory ‘/home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp’
Building CXX object CMakeFiles/cmTC_285cc.dir/src.cxx.o
/usr/bin/g++ -DCMAKE_HAVE_LIBC_PTHREAD -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -std=c++11 -o CMakeFiles/cmTC_285cc.dir/src.cxx.o -c /home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp/src.cxx
Linking CXX executable cmTC_285cc
/home/ashiq/install/bin/cmake -E cmake_link_script CMakeFiles/cmTC_285cc.dir/link.txt --verbose=1
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss CMakeFiles/cmTC_285cc.dir/src.cxx.o -o cmTC_285cc
CMakeFiles/cmTC_285cc.dir/src.cxx.o: In function main': src.cxx:(.text+0x3e): undefined reference to pthread_create’
src.cxx:(.text+0x4a): undefined reference to pthread_detach' src.cxx:(.text+0x56): undefined reference to pthread_cancel’
src.cxx:(.text+0x67): undefined reference to pthread_join' src.cxx:(.text+0x7b): undefined reference to pthread_atfork’
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_285cc.dir/build.make:105: recipe for target ‘cmTC_285cc’ failed
make[1]: *** [cmTC_285cc] Error 1
make[1]: Leaving directory ‘/home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp’
Makefile:140: recipe for target ‘cmTC_285cc/fast’ failed
make: *** [cmTC_285cc/fast] Error 2
Source file was:
include <pthread.h>
static void* test_func(void* data)
{
return data;
}
int main(void)
{
pthread_t thread;
pthread_create(&thread, NULL, test_func, NULL);
pthread_detach(thread);
pthread_cancel(thread);
pthread_join(thread, NULL);
pthread_atfork(NULL, NULL, NULL);
pthread_exit(NULL);
return 0;
}
Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make cmTC_efdd3/fast && /usr/bin/make -f CMakeFiles/cmTC_efdd3.dir/build.make CMakeFiles/cmTC_efdd3.dir/build
make[1]: Entering directory ‘/home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp’
Building CXX object CMakeFiles/cmTC_efdd3.dir/CheckFunctionExists.cxx.o
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create -std=c++11 -o CMakeFiles/cmTC_efdd3.dir/CheckFunctionExists.cxx.o -c /home/ashiq/TensorRT/build/CMakeFiles/CheckLibraryExists/CheckFunctionExists.cxx
Linking CXX executable cmTC_efdd3
/home/ashiq/install/bin/cmake -E cmake_link_script CMakeFiles/cmTC_efdd3.dir/link.txt --verbose=1
/usr/bin/g++ -Wno-deprecated-declarations -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create CMakeFiles/cmTC_efdd3.dir/CheckFunctionExists.cxx.o -o cmTC_efdd3 -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_efdd3.dir/build.make:105: recipe for target ‘cmTC_efdd3’ failed
make[1]: *** [cmTC_efdd3] Error 1
make[1]: Leaving directory ‘/home/ashiq/TensorRT/build/CMakeFiles/CMakeTmp’
Makefile:140: recipe for target ‘cmTC_efdd3/fast’ failed
make: *** [cmTC_efdd3/fast] Error 2
Above log is from your x86 machine, right?
@Morganh , yes.
Could you add -DGPU_ARCHS=xy and retry ?
For DGPU_ARCHS, please refer to deepstream_tao_apps/TRT-OSS/x86 at master · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub
Just did. Same result. My ARCH is 61. I did not use it in the first place because it was already set as default. Anyway, rerunning it with “-DGPU_ARCHS=61” did not make any difference.
Did you install zlib?
@Morganh , Thanks for your reply. I finally could build “libnvinfer_plugin.so.8.0.1” in x86 and build the “yolo4_1_3_416_416_static.onnx” for tiny YOLO V4. After that I added “BatchedNMSPlugin” node into the ONNX model following step 2 of section 2.3 described in GitHub - NVIDIA-AI-IOT/yolov4_deepstream in my host x86 machine. This produced the “yolo4_1_3_416_416_static_onnx.nms.onnx” file.
In next step. I copied the “yolo4_1_3_416_416_static_onnx.nms.onnx” file in my Jetson Nano with JP4.6 and trying to execute and build .engine file following step 3.5 (generate Engine of fp16 mode). and I am getting the error below:
(By the way, these are the same steps that I followed when I used “libnvinfer_plugin.so.7.1.3” for earlier version of Jetpack to use YOLO V4 and it worked)
&&&& RUNNING TensorRT.sample_yolo [TensorRT v8001] # --onnx yolov4-tiny_1_3_416_416_nms.onnx
There are 0 coco images to process
[03/30/2022-11:27:56] [I] [TRT] [MemUsageChange] Init CUDA: CPU +203, GPU +0, now: CPU 226, GPU 2253 (MiB)
[03/30/2022-11:27:56] [I] Parsing ONNX file: yolov4-tiny_1_3_416_416_nms.onnx
[03/30/2022-11:27:56] [I] [TRT] ----------------------------------------------------------------
[03/30/2022-11:27:56] [I] [TRT] Input filename: yolov4-tiny_1_3_416_416_nms.onnx
[03/30/2022-11:27:56] [I] [TRT] ONNX IR version: 0.0.7
[03/30/2022-11:27:56] [I] [TRT] Opset version: 11
[03/30/2022-11:27:56] [I] [TRT] Producer name:
[03/30/2022-11:27:56] [I] [TRT] Producer version:
[03/30/2022-11:27:56] [I] [TRT] Domain:
[03/30/2022-11:27:56] [I] [TRT] Model version: 0
[03/30/2022-11:27:56] [I] [TRT] Doc string:
[03/30/2022-11:27:56] [I] [TRT] ----------------------------------------------------------------
[03/30/2022-11:27:56] [W] [TRT] onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[03/30/2022-11:27:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[03/30/2022-11:27:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[03/30/2022-11:27:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[03/30/2022-11:27:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[03/30/2022-11:27:56] [I] [TRT] No importer registered for op: BatchedNMS_TRT. Attempting to import as plugin.
[03/30/2022-11:27:56] [I] [TRT] Searching for plugin: BatchedNMS_TRT, plugin_version: 1, plugin_namespace:
[03/30/2022-11:27:56] [W] [TRT] builtin_op_importers.cpp:4552: Attribute scoreBits not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[03/30/2022-11:27:56] [I] [TRT] Successfully created plugin: BatchedNMS_TRT
[03/30/2022-11:27:56] [W] [TRT] Output type must be INT32 for shape outputs
[03/30/2022-11:27:56] [W] [TRT] Output type must be INT32 for shape outputs
[03/30/2022-11:27:56] [I] Building TensorRT engine: …/…/build/yolov4-tiny_1_3_416_416_nms.engine
[03/30/2022-11:27:56] [I] [TRT] [MemUsageSnapshot] Builder begin: CPU 251 MiB, GPU 2320 MiB
[03/30/2022-11:27:57] [I] [TRT] ---------- Layers Running on DLA ----------
[03/30/2022-11:27:57] [I] [TRT] ---------- Layers Running on GPU ----------
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 913[Constant]
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 465[Constant]
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_0
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_1
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_2
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_3
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_4
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_5
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_24
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_25
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_26
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_27
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_28
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_30
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_31
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 127 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] MaxPool_33
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_34
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_35
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_54
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_55
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_56
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_57
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_58
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_60
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_61
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 163 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] MaxPool_63
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_64
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_65
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_84
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_85
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_86
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_87
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_88
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_90
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_91
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 199 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] MaxPool_93
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_94
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_95
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_96
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_97
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_98
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_446
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_99
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_447
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_100
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_105
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_110
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_115
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_120
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_125
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_130
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_135
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_140
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_145
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_150
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_155
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_160
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Exp_217
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Cast_313
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Cast_318
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_238
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_245
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_266
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_273
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_294
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_301
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 479) [Shuffle]
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 481) [Shuffle]
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_465
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(Sigmoid_212), 361 + (Unnamed Layer* 373) [Shuffle] + Mul_214), PWN(363 + (Unnamed Layer* 396) [Shuffle], Sub_216))
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 387 + (Unnamed Layer* 399) [Shuffle] + Mul_240
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 394 + (Unnamed Layer* 402) [Shuffle] + Mul_247
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 415 + (Unnamed Layer* 405) [Shuffle] + Mul_268
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 422 + (Unnamed Layer* 408) [Shuffle] + Mul_275
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 443 + (Unnamed Layer* 411) [Shuffle] + Mul_296
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 450 + (Unnamed Layer* 414) [Shuffle] + Mul_303
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_224
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_231
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_252
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_259
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_280
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_287
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_194 + Transpose_195
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 373 + Add_226
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 380 + Add_233
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 401 + Add_254
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 408 + Add_261
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 429 + Add_282
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 436 + Add_289
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_179
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(Sigmoid_218)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_211
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 452 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 454 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 453 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 455 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_444
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Div_314
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Div_319
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(Sigmoid_219), Mul_445)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_324
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_366
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_345
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_387
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_340
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_382
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_361
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_403
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(560 + (Unnamed Layer* 502) [Shuffle], Mul_405), Sub_406)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(563 + (Unnamed Layer* 505) [Shuffle], Mul_408), Sub_409)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Expand_492
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Add_410
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Add_411
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_514
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 562 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 565 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 566 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 567 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 230 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_428
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_516
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] LeakyRelu_517
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Conv_518
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_523
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_528
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_533
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_538
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_543
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_548
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_553
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_558
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_563
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_568
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_573
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_578
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Exp_635
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Cast_731
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Cast_736
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_656
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_663
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_684
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_691
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_712
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_719
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 747) [Shuffle]
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 749) [Shuffle]
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(Sigmoid_630), 809 + (Unnamed Layer* 654) [Shuffle] + Mul_632), PWN(811 + (Unnamed Layer* 674) [Shuffle], Sub_634))
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 835 + (Unnamed Layer* 677) [Shuffle] + Mul_658
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 842 + (Unnamed Layer* 680) [Shuffle] + Mul_665
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 863 + (Unnamed Layer* 683) [Shuffle] + Mul_686
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 870 + (Unnamed Layer* 686) [Shuffle] + Mul_693
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 891 + (Unnamed Layer* 689) [Shuffle] + Mul_714
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 898 + (Unnamed Layer* 692) [Shuffle] + Mul_721
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_642
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_649
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_670
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_677
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_698
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_705
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_612 + Transpose_613
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 821 + Add_644
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 828 + Add_651
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 849 + Add_672
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 856 + Add_679
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 877 + Add_700
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 884 + Add_707
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_597
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(Sigmoid_636)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_629
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 900 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 902 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 901 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 903 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_862
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Div_732
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Div_737
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(Sigmoid_637), Mul_863)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_742
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_784
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_763
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Slice_805
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 607 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 1055 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_758
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_800
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_779
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_821
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(1008 + (Unnamed Layer* 762) [Shuffle], Mul_823), Sub_824)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] PWN(PWN(1011 + (Unnamed Layer* 765) [Shuffle], Mul_826), Sub_827)
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Add_828
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Add_829
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 1010 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 1013 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 1014 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] 1015 copy
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] Reshape_846
[03/30/2022-11:27:57] [I] [TRT] [GpuLayer] node_of_num_detections
[03/30/2022-11:27:58] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +158, GPU +241, now: CPU 410, GPU 2562 (MiB)
[03/30/2022-11:28:00] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +240, GPU +353, now: CPU 650, GPU 2915 (MiB)
[03/30/2022-11:28:00] [W] [TRT] Detected invalid timing cache, setup a local cache instead
[03/30/2022-11:33:38] [F] [TRT] [defaultAllocator.cpp::free::85] Error Code 1: Cuda Runtime (unspecified launch failure)
terminate called after throwing an instance of ‘nvinfer1::CudaRuntimeError’
what(): unspecified launch failure
Aborted (core dumped)
For tao’s model, its official guide for running inference in deepstream is GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream instead of GitHub - NVIDIA-AI-IOT/yolov4_deepstream
Users cannot converted the .etlt model to .onnx model.
So, in terms of tao, it is not related to GitHub - NVIDIA-AI-IOT/yolov4_deepstream .
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
