Jetson Nano ERROR nvinfer gstnvinfer.cpp:613 , while running deepstream tao 3.0

On Jetson Nano
using Jetpack 4.5
Deepstream 5.1
Ubuntu 18.04
model: SSD

I successfully built deepstream tao3.0and TensorRT OSS on jetson nano, and tried to run the python sample code with changing the config.
This is the command i run, its a code came from here:

sudo python3 deepstream_test_1.py /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264

Config file:

[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=ssd_labels.txt
#model-engine-file=../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine
tlt-encoded-model=../../models/ssd/ssd_resnet18.etlt
int8-calib-file=../../models/ssd/cal.bin
tlt-model-key=mykey
infer-dims=3;544;960
uff-input-order=0
maintain-aspect-ratio=1
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=5
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
output-blob-names=NMS
parse-bbox-func-name=NvDsInferParseCustomNMSTLT
custom-lib-path=../../post_processor/libnvds_infercustomparser_tao.so

An error occurred :

Using winsys: x11 
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
0:00:00.314522752 16636     0x23b12390 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Could not read buffer.
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.532375420 16636     0x23b12390 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault

Not sure why, is it because my deepstream tao 3.0, TensorRT OSS failed to build, or the setings in config are wrong?

Firstly, please check if you can run your tlt model successfully in nano with the official guide mentioned in https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/tree/release/tao3.0 .

It gives the same error, sorry I didn’t mention it above.

Please share the full command and full log when you run with https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/tree/release/tao3.0

This is the full command and full log

$./apps/tao_detection/ds-tao-detection -c configs/ssd_tao/pgie_ssd_tao_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264

Now playing: configs/ssd_tao/pgie_ssd_tao_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
0:00:01.580582751 21560   0x557a1ef300 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Could not read buffer.
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:05.644315085 21560   0x557a1ef300 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault

Is the tlt-model-key correct?

Yes, I changed it to my own key when I was running the code.

Is ssd_resnet18.etlt trained by yourself? What is the key when you train and get this tlt model?

I’m using the example model I download with

./download_models.sh

And for key I had tried the original one: nvidia_tlt
and mines: NHZtNnA1b2Noa2lzZWNibjV0OTY1cm9sYW46NGJmNDM3ODEtODllMy00ZWI5LWE5ZDUtODJiYzc0MWE3Zjlj
Both gave the same result.

Suggest you to git clone again and double check. Just use the default config file.

Than do I need to build TRT OSS Plugin again?

Not needed. Please backup it.

Hi, after git clone again, this is the full command and log:

$ sudo ./apps/tao_detection/ds-tao-detection -c configs/ssd_tao/pgie_ssd_tao_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264
[sudo] password for ypnano7: 
Now playing: configs/ssd_tao/pgie_ssd_tao_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_tao_apps/configs/ssd_tao/../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine open error
0:00:04.728876757 23976   0x5590aa4f00 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_tao_apps/configs/ssd_tao/../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine failed
0:00:04.728961082 23976   0x5590aa4f00 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_tao_apps/configs/ssd_tao/../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine failed, try rebuild
0:00:04.728986916 23976   0x5590aa4f00 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Validator error: FirstDimTile_4: Unsupported operation _BatchTilePlugin_TRT
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:05.315894128 23976   0x5590aa4f00 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault

Config:

[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=ssd_labels.txt
model-engine-file=../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine
tlt-encoded-model=../../models/ssd/ssd_resnet18.etlt
int8-calib-file=../../models/ssd/cal.bin
tlt-model-key=nvidia_tlt
infer-dims=3;544;960
uff-input-order=0
maintain-aspect-ratio=1
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=5
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
output-blob-names=NMS
parse-bbox-func-name=NvDsInferParseCustomNMSTLT
custom-lib-path=../../post_processor/libnvds_infercustomparser_tao.so

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

You mention that you have built the TRT OSS plugin.
Have you replaced it ? See https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/tree/release/tao3.0/TRT-OSS/Jetson#3-replace-libnvinfer_pluginso

Yes, thanks for the reminder anyway.

I am afraid there is something wrong when you replace the libnvinfer_plugin.
Refer to my post Failling in building sample from TLT-DEEPSTREAM

Refer to your post ,
I checked the command I gave:

sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3.bak
sudo cp /opt/nvidia/deepstream/deepstream-5.1/sources/TensorRT/build/out/libnvinfer_plugin.so.7.1.3  /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3
sudo ldconfig

It looks correct.

But with the ll command ou mention in the post, mine link doesn’t look right:

$ ll -sh /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
   0 lrwxrwxrwx 1 root root   26  6月  6  2020 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so -> libnvinfer_plugin.so.7.1.3*
   0 lrwxrwxrwx 1 root root   30  2月  9 13:56 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7 -> libnvinfer_plugin.so.7.1.3.bak
 14M -rwxr-xr-x 1 root root  14M  2月  9 13:56 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3*
5.4M -rw-r--r-- 1 root root 5.4M  6月  6  2020 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3.bak

Please move the original .so outside. See below.

$ sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0 ~/libnvinfer_plugin.so.7.1.0.bak

Do you mean, in my case I should move .so, .so.7, .so.7.1.3 these three file do somewhere outside, and than put the new three file that I have generate in here?

Also, because of my previous command, th original one became 7.1.3.bak, so I should fist mv that file out side?

No.
See my post Failling in building sample from TLT-DEEPSTREAM - #15 by Morganh

Please follow
$ sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0 ~/libnvinfer_plugin.so.7.1.0.bak

$ sudo cp {TRT_SOURCE}/build/out/libnvinfer_plugin.so.7.0.0.1 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0

$ sudo ldconfig