Yolo V4 on Jetson Nano with JP4.6

caruofc · April 30, 2022, 6:49pm

Hi,

Please refer to this link YoloV4 with OpenCV where @AastaLLL provided a solution for me on how to use YoloV4 using TensorRT.

Basically, what I am trying to do is to use Tiny Yolo V4’s pretrained weights (yolov4-tiny.weights), cfg file (yolov4-tiny.cfg) and coco names file (coco.names) for prediction in Jatson Nano using JetPack 4.6.
I have done the same in previous Jetpacks and it worked but in JetPack 4.6 I have having an issue.

NEE HELP.

So, this is what I am doing:

produce the ONNX file from weights and cfg files in a host PC: For this step I installed TensorRT 8.0.1 (https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.0.1/local_repos/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb) in a x86 Ubuntu 18.04 host PC.
Installed compatible CUDA as:
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
$ sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
$ wget https://developer.download.nvidia.com/compute/cuda/11.3.0/local_installers/cuda-repo-ubuntu1804-11-3-local_11.3.0-465.19.01-1_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1804-11-3-local_11.3.0-465.19.01-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-ubuntu1804-11-3-local/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get -y install cuda
Install ONNX runtime as:
$ pip3 install onnxruntime
Installl Cmake (>= 3.13) as:
TensorRT OSS requires cmake >= v3.13, so install cmake 3.13 if your cmake version is lower than 3.13
$ wget https://github.com/Kitware/CMake/releases/download/v3.19.4/cmake-3.19.4.tar.gz
$ tar xvf cmake-3.19.4.tar.gz
$ cd cmake-3.19.4/
$ mkdir $HOME/install
$ ./configure --prefix=$HOME/install
$ make -j$(nproc)
$ sudo make install
Build TensorRT OSS Plugin as:
$ git clone -b release/8.0 https://github.com/nvidia/TensorRT
$ cd TensorRT/
$ git submodule update --init --recursive
$ export TRT_SOURCE=pwd $ cd $TRT_SOURCE $ mkdir -p build && cd build $ export CUDACXX=/usr/local/cuda/bin/nvcc $ $HOME/install/bin/cmake … -DGPU_ARCHS=61 -DTRT_LIB_DIR=/usr/lib/x86_64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=pwd/out $ make nvinfer_plugin -j$(nproc)`
Replace libnvinfer_plugin files:
Replace the following generated files from the Tensor RT OSS “build” directory to the “lib” forlder of TensorRT but first save the original files. Issue the following 3 commands
$ sudo mv /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so* $HOME
From TennsorRT build folder execute
$ sudo cp libnvinfer_plugin.so* /usr/lib/x86_64-linux-gnu/
$ sudo ldconfig
Prepare ONNX file from Yolo weight and cfg file as:
Download Pytorch-YOlOv4 project from GitHub - Tianxiaomo/pytorch-YOLOv4: PyTorch ,ONNX and TensorRT implementation of YOLOv4
Install the pre-requisites from Pytorch-YOLOv4 directory
$ pip3 install -r requirements.txt
$ python3 demo_darknet2onnx.py yolov4-tiny.cfg coco.names yolov4-tiny.weights ./data/dog.jpg 1

Above command will create the ONNX file (yolov4_1_3_416_416_static.onnx) in Pytorch-YOLOv4 directory and a result of detection (predictions_onnx.jpg) for the sample input .jpg(./data/dog.jpg)

Add NMS filter to the ONNX file as:
Download yolov4_deepstream from GitHub - NVIDIA-AI-IOT/yolov4_deepstream and copy the generated ONNX file into “yolov4_deepstream-master/tensorrt_yolov4/source” directory and execute
$ python3 onnx_add_nms_plugin.py -f yolov4_1_3_416_416_static.onnx
Above command will generate “yolov4_1_3_416_416_static.onnx.nms.onnx”

All of the above steps worked as expected and the NMS ONNX file is generated which is now ready to be copied to Jetson Nano with Jetpack 4.6

Following steps are then performed on the target Jetson Nano board

Installl Cmake (>= 3.13) as:
TensorRT OSS requires cmake >= v3.13, so install cmake 3.13 if your cmake version is lower than 3.13
$ wget https://github.com/Kitware/CMake/releases/download/v3.19.4/cmake-3.19.4.tar.gz
$ tar xvf cmake-3.19.4.tar.gz
$ cd cmake-3.19.4/
$ mkdir $HOME/install
$ ./configure --prefix=$HOME/install
$ make -j$(nproc)
$ sudo make install
Build TensorRT OSS Plugin as:
$ git clone -b release/8.0 https://github.com/nvidia/TensorRT
$ cd TensorRT/
$ git submodule update --init --recursive
$ export TRT_SOURCE=pwd $ cd $TRT_SOURCE $ mkdir -p build && cd build $ export CUDACXX=/usr/local/cuda/bin/nvcc $HOME/install/bin/cmake … -DGPU_ARCHS=“53 62 72” -DTRT_LIB_DIR=/usr/lib/aarch64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=pwd/out $ make nvinfer_plugin -j$(nproc)`
Replace libnvinfer_plugin files as:
Replace the following generated files from the Tensor RT OSS “build” directory to the “/usr/lib/aarch64-linux-gnu/” forlder but first save the original files. Issue the following 3 commands
$ sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so* $HOME
From TennsorRT build folder execute
$ sudo cp libnvinfer_plugin.so* /usr/lib/aarch64-linux-gnu/
$ sudo ldconfig
Download yolov4_deepstream from GitHub - NVIDIA-AI-IOT/yolov4_deepstream.
Copy /usr/src/tensorrt/samples/common directory in yolov4_deepstream/tensorrt_yolov4/
Copy /usr/include/opencv4/opencv2 directory in yolov4_deepstream/tensorrt_yolov4/common
From yolov4_deepstream/tensorrt_yolov4/source execute:
$ make -j$(nproc)
Now copy the “yolov4_1_3_416_416_static.onnx.nms.onnx” file generated in the host machine into “yolov4_deepstream/tensorrt_yolov4/data” folder and rename it to “yolov4.onnx” and then execute the following command from “yolov4_deepstream/tensorrt_yolov4/source”
$ ../bin/yolov4 --fp16

The above command is expected to generate an .engine file and run the prediction on the input image “yolov4_deepstream/tensorrt_yolov4/data/demo.jpg” and produce the result in the output image (yolov4_deepstream/tensorrt_yolov4/data/demo_out.jpg).

However, when I ran the above command ($ …/bin/yolov4 --fp16), I am getting the following error.

&&&& RUNNING TensorRT.sample_yolo [TensorRT v8001] # ../bin/yolov4 --fp16
There are 0 coco images to process
[04/30/2022-12:02:54] [I] Building and running a GPU inference engine for Yolo
[04/30/2022-12:02:56] [I] [TRT] [MemUsageChange] Init CUDA: CPU +203, GPU +0, now: CPU 222, GPU 2230 (MiB)
[04/30/2022-12:02:56] [I] Parsing ONNX file: ../data/yolov4.onnx
[04/30/2022-12:02:56] [I] [TRT] ----------------------------------------------------------------
[04/30/2022-12:02:56] [I] [TRT] Input filename:   ../data/yolov4.onnx
[04/30/2022-12:02:56] [I] [TRT] ONNX IR version:  0.0.8
[04/30/2022-12:02:56] [I] [TRT] Opset version:    11
[04/30/2022-12:02:56] [I] [TRT] Producer name:    
[04/30/2022-12:02:56] [I] [TRT] Producer version: 
[04/30/2022-12:02:56] [I] [TRT] Domain:           
[04/30/2022-12:02:56] [I] [TRT] Model version:    0
[04/30/2022-12:02:56] [I] [TRT] Doc string:       
[04/30/2022-12:02:56] [I] [TRT] ----------------------------------------------------------------
[04/30/2022-12:02:56] [W] [TRT] onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[04/30/2022-12:02:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[04/30/2022-12:02:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[04/30/2022-12:02:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[04/30/2022-12:02:56] [W] [TRT] onnx2trt_utils.cpp:390: One or more weights outside the range of INT32 was clamped
[04/30/2022-12:02:57] [I] [TRT] No importer registered for op: BatchedNMS_TRT. Attempting to import as plugin.
[04/30/2022-12:02:57] [I] [TRT] Searching for plugin: BatchedNMS_TRT, plugin_version: 1, plugin_namespace: 
[04/30/2022-12:02:57] [W] [TRT] builtin_op_importers.cpp:4552: Attribute scoreBits not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[04/30/2022-12:02:57] [I] [TRT] Successfully created plugin: BatchedNMS_TRT
[04/30/2022-12:02:57] [I] Building TensorRT engine../data/yolov4.engine
[04/30/2022-12:02:57] [I] [TRT] [MemUsageSnapshot] Builder begin: CPU 247 MiB, GPU 2299 MiB
[04/30/2022-12:02:57] [I] [TRT] ---------- Layers Running on DLA ----------
[04/30/2022-12:02:57] [I] [TRT] ---------- Layers Running on GPU ----------
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 886[Constant]
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 447[Constant]
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_119
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_121
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_122
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_124
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_125
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_127
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_142
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_143
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_145
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_146
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_148
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_150
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_152
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 127 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_154
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_155
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_157
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_172
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_173
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_175
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_176
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_178
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_180
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_182
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 157 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_184
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_185
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_187
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_202
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_203
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_205
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_206
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_208
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_210
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_212
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 187 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_214
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_215
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_217
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_218
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_220
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_221
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_590
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_223
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_592
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_224
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_229
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_234
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_239
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_244
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_249
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_254
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_259
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_264
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_269
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_274
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_279
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_284
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_347
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_443
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_448
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_368
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_375
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_396
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_403
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_424
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_431
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 479) [Shuffle]
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 481) [Shuffle]
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_614
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(node_of_342), 343 + (Unnamed Layer* 373) [Shuffle] + node_of_344), PWN(345 + (Unnamed Layer* 395) [Shuffle], node_of_346))
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 369 + (Unnamed Layer* 398) [Shuffle] + node_of_370
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 376 + (Unnamed Layer* 401) [Shuffle] + node_of_377
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 397 + (Unnamed Layer* 404) [Shuffle] + node_of_398
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 404 + (Unnamed Layer* 407) [Shuffle] + node_of_405
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 425 + (Unnamed Layer* 410) [Shuffle] + node_of_426
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 432 + (Unnamed Layer* 413) [Shuffle] + node_of_433
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_354
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_361
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_382
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_389
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_410
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_417
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_322 + node_of_323
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 355 + node_of_356
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 362 + node_of_363
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 383 + node_of_384
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 390 + node_of_391
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 411 + node_of_412
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 418 + node_of_419
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_303
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_636
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(node_of_348)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_341
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_658
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 434 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 436 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 435 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 437 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_588
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 212 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_444
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_449
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(node_of_349), node_of_589)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_660
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_454
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_500
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_477
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_523
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_472
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_518
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_495
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_541
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_662
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_663
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(542 + (Unnamed Layer* 536) [Shuffle], node_of_543), node_of_544)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(545 + (Unnamed Layer* 539) [Shuffle], node_of_546), node_of_547)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_668
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_673
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_678
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_683
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_688
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_693
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_698
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_703
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_708
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_713
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_718
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_723
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_548
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_549
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 544 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 547 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 548 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 549 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_786
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_882
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_887
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_570
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_807
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_814
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_835
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_842
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_863
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_870
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 736) [Shuffle]
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] (Unnamed Layer* 738) [Shuffle]
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(PWN(node_of_781), 782 + (Unnamed Layer* 641) [Shuffle] + node_of_783), PWN(784 + (Unnamed Layer* 662) [Shuffle], node_of_785))
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 808 + (Unnamed Layer* 665) [Shuffle] + node_of_809
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 815 + (Unnamed Layer* 668) [Shuffle] + node_of_816
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 836 + (Unnamed Layer* 671) [Shuffle] + node_of_837
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 843 + (Unnamed Layer* 674) [Shuffle] + node_of_844
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 864 + (Unnamed Layer* 677) [Shuffle] + node_of_865
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 871 + (Unnamed Layer* 680) [Shuffle] + node_of_872
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_793
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_800
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_821
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_828
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_849
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_856
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_761 + node_of_762
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 794 + node_of_795
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 801 + node_of_802
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 822 + node_of_823
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 829 + node_of_830
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 850 + node_of_851
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 857 + node_of_858
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_742
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(node_of_787)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_780
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 873 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 875 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 874 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 876 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_1027
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_883
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_888
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(node_of_788), node_of_1028)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_893
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_939
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_916
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_962
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 589 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 1028 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_911
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_957
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_934
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_980
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(981 + (Unnamed Layer* 751) [Shuffle], node_of_982), node_of_983)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] PWN(PWN(984 + (Unnamed Layer* 754) [Shuffle], node_of_985), node_of_986)
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_987
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_988
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 983 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 986 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 987 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] 988 copy
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_1009
[04/30/2022-12:02:57] [I] [TRT] [GpuLayer] node_of_num_detections
[04/30/2022-12:02:58] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +158, GPU +253, now: CPU 428, GPU 2561 (MiB)
[04/30/2022-12:03:00] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +241, GPU +353, now: CPU 669, GPU 2914 (MiB)
[04/30/2022-12:03:00] [W] [TRT] Detected invalid timing cache, setup a local cache instead
[04/30/2022-12:08:33] [F] [TRT] [defaultAllocator.cpp::free::85] Error Code 1: Cuda Runtime (unspecified launch failure)
terminate called after throwing an instance of 'nvinfer1::CudaRuntimeError'
  what():  unspecified launch failure
./run.sh: line 6:  7255 Aborted                 (core dumped) ../bin/yolov4 --fp16

Need help on how to get rid of the above error. I also tried to use the prebuild “libnvinfer_plugin.so.8.0.1” downloaded from deepstream_tao_apps/TRT-OSS/Jetson/TRT8.0 at master · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub instead of the one I generated by I got the same error.

I have attcahed the “yolov4_1_3_416_416_static.onnx.nms.onnx” file below that I have generated if someone wants to try using it on Nano to see if you could generate the .engine file.

ANY HELP WOULD BE GREATELY APPRECIATED.
yolov4_1_3_416_416_static.onnx.nms.onnx (23.2 MB)

AastaLLL · May 3, 2022, 3:40am

Hi,

Do you use JetPack 4.6.1?

If yes, please noted that JetPack 4.6.1 includes TensorRT 8.2.1.
You will need to check out the release/8.2 TensorRT OSS branch instead.

$ git clone -b release/8.2 https://github.com/nvidia/TensorRT

Thanks.

system · May 25, 2022, 3:00am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.