Error in Yolov4 engine conversion,

Okay,

I am using 4.5.

**apt-cache show nvidia-jetpack**

Package: nvidia-jetpack
Version: 4.5.1-b17
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.5.1-b17), nvidia-opencv (= 4.5.1-b17), nvidia-cudnn8 (= 4.5.1-b17), nvidia-tensorrt (= 4.5.1-b17), nvidia-visionworks (= 4.5.1-b17), nvidia-container (= 4.5.1-b17), nvidia-vpi (= 4.5.1-b17), nvidia-l4t-jetson-multimedia-api (>> 32.5-0), nvidia-l4t-jetson-multimedia-api (<< 32.6-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.5.1-b17_arm64.deb
Size: 29372
SHA256: 378f7588e15c35692eb1bed6f336be74f4f396d88fad45af67c68e22b63be04b
SHA1: e41f26a3d8326e9952915eee12fa37e17de3245f
MD5sum: 31b2bd9d0f214f74acaeb3d8e4279e9d
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.5-b129
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.5-b129), nvidia-opencv (= 4.5-b129), nvidia-cudnn8 (= 4.5-b129), nvidia-tensorrt (= 4.5-b129), nvidia-visionworks (= 4.5-b129), nvidia-container (= 4.5-b129), nvidia-vpi (= 4.5-b129), nvidia-l4t-jetson-multimedia-api (>> 32.5-0), nvidia-l4t-jetson-multimedia-api (<< 32.6-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.5-b129_arm64.deb
Size: 29360
SHA256: 002646e6d81d13526ade23d7c45180014f3cd9e9f5fb0f8896b77dff85d6b9fe
SHA1: cb17547b902b2793e0df86d561809ecdbf7e401f
MD5sum: 06962c42e462f643455d6194d1a2d641
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Can you share
$ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*

Sure. My NX is also using libnvinfer_plugin.so.7.1.3.
Will update you after upgrading on TensorRT-7.2.2 as you have mentioned steps above.

Wait for a moment. I am afraid you did not replace the plugin correctly.

The expected is as below.
$ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
lrwxrwxrwx 1 root root 26 6月 6 2020 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so → libnvinfer_plugin.so.7.1.3*
lrwxrwxrwx 1 root root 26 10月 12 15:12 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7 → libnvinfer_plugin.so.7.1.3*
lrwxrwxrwx 1 root root 26 10月 12 15:12 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.0.0 → libnvinfer_plugin.so.7.1.3*
-rwxr-xr-x 1 root root 10009144 10月 12 15:06 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3*

Please follow step 4 of YOLOv4 — TAO Toolkit 3.0 documentation

Thanks @Morganh

I Followd the step 4.

But getting /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7 is not a symbolic link issue.

Suggest you backing up the default 7.1.3 plugin.
Then use "sudo ln -s xxx xxx " to generate the expected I mentioned above.

For example,
$ cd /usr/lib/aarch64-linux-gnu
$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so
$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7
$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.0.0

Yes I have done that.
Below is the output of ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*

smarg@smarg-NX:~/Documents/Pritam/Models/TensorRT/build$ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
-rwxr-xr-x 1 root root 9697336 Oct 12 13:50 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
-rwxr-xr-x 1 root root 9697336 Oct 12 13:50 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7*
lrwxrwxrwx 1 root root 26 Oct 12 14:30 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.0.0 → libnvinfer_plugin.so.7.1.3*
-rwxr-xr-x 1 root root 11028968 Oct 12 14:11 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3*

It is not expected yet.
The /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so should point to libnvinfer_plugin.so.7.1.3
The /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7 should point to libnvinfer_plugin.so.7.1.3

But with the command

$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so
$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7

I was getting

ln: failed to create symbolic link '/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so': File exists
ln: failed to create symbolic link '/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7': File exists

Just run
$ sudo rm /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so
$ sudo rm /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7

then,
$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so
$ sudo ln -s libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7

Okay Now getting.

smarg@smarg-NX:/usr/lib/aarch64-linux-gnu$ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
lrwxrwxrwx 1 root root       26 Oct 12 14:50 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so -> libnvinfer_plugin.so.7.1.3*
lrwxrwxrwx 1 root root       26 Oct 12 14:50 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7 -> libnvinfer_plugin.so.7.1.3*
lrwxrwxrwx 1 root root       26 Oct 12 14:30 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.0.0 -> libnvinfer_plugin.so.7.1.3*
-rwxr-xr-x 1 root root 11028968 Oct 12 14:36 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3*
smarg@smarg-NX:/usr/lib/aarch64-linux-gnu$ 

Now what should I do next ?
Still the same issue :

Sometime error msg invisible and some time visible.

Can you try to rebuild TRT OSS plugin?
On my side, the size is 10009144 .
-rwxr-xr-x 1 root root 10009144 10月 12 15:06 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3*

Yes Retrying.

Hi @morganh
I rebuild the TRT OSS plugin but still same issue.

Steps I followed :



    $ git clone -b 21.03 https://github.com/nvidia/TensorRT
    $ cd TensorRT/
    $ git submodule update --init --recursive
    $ export TRT_SOURCE=pwd
    $ cd $TRT_SOURCE
    $ mkdir -p build && cd build
    $ /usr/local/bin/cmake … -DGPU_ARCHS=72 -DTRT_LIB_DIR=/usr/lib/aarch64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=pwd/out

    $ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
    Previously My NX is using libnvinfer_plugin.so.7.1.3
    $ sudo cp libnvinfer_plugin.so.7.2.2 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3
    $ sudo ldconfig

    $ ./tlt-converter -k nvidia_tlt -d 3,544,960 -e trt.fp16.engine -t fp16 -p Input,1x3x544x960,1x3x544x960,1x3x544x960 yolov4_resnet18.etlt

Issue :

Can you try my plugin which is built in NX ?libnvinfer_plugin.so.7.2.2 (9.5 MB)

Sure.
I have to run only two commands

    $ sudo cp libnvinfer_plugin.so.7.2.2 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3
    $ sudo ldconfig

At the place of libnvinfer_plugin.so.7.2.2 in the above command will be your plugin ?

Can you share your yolov4 engine file too. ?

Yes, please share the result of $ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so* as well.

Below is the output $ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*

But tlt-converter again giving me the error:

smarg@smarg-NX:~/Documents/Pritam/Models/yolov4$ ./tlt-converter -k nvidia_tlt -d 3,544,960 -e trt.fp16.engine -t fp16 -p Input,1x3x544x960,1x3x544x960,1x3x544x960 yolov4_resnet18.etlt
[ERROR] UffParser: Unsupported number of graph 0
[ERROR] Failed to parse the model, please check the encoding key to make sure it's correct
[ERROR] Network must have at least one output
[ERROR] Network validation failed.
[ERROR] Unable to create engine
Segmentation fault (core dumped)