Jetson Nano ERROR nvinfer gstnvinfer.cpp:613 , while running deepstream tao 3.0

tmid0819 · February 9, 2022, 8:36am

On Jetson Nano
using Jetpack 4.5
Deepstream 5.1
Ubuntu 18.04
model: SSD

I successfully built deepstream tao3.0and TensorRT OSS on jetson nano, and tried to run the python sample code with changing the config.
This is the command i run, its a code came from here:

sudo python3 deepstream_test_1.py /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264

Config file:

[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=ssd_labels.txt
#model-engine-file=../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine
tlt-encoded-model=../../models/ssd/ssd_resnet18.etlt
int8-calib-file=../../models/ssd/cal.bin
tlt-model-key=mykey
infer-dims=3;544;960
uff-input-order=0
maintain-aspect-ratio=1
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=5
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
output-blob-names=NMS
parse-bbox-func-name=NvDsInferParseCustomNMSTLT
custom-lib-path=../../post_processor/libnvds_infercustomparser_tao.so

An error occurred :

Using winsys: x11 
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
0:00:00.314522752 16636     0x23b12390 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Could not read buffer.
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.532375420 16636     0x23b12390 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault

Not sure why, is it because my deepstream tao 3.0, TensorRT OSS failed to build, or the setings in config are wrong?

Morganh · February 9, 2022, 8:42am

Firstly, please check if you can run your tlt model successfully in nano with the official guide mentioned in GitHub - NVIDIA-AI-IOT/deepstream_tao_apps at release/tao3.0 .

tmid0819 · February 9, 2022, 8:44am

It gives the same error, sorry I didn’t mention it above.

Morganh · February 9, 2022, 8:46am

Please share the full command and full log when you run with GitHub - NVIDIA-AI-IOT/deepstream_tao_apps at release/tao3.0

tmid0819 · February 9, 2022, 8:51am

This is the full command and full log

$./apps/tao_detection/ds-tao-detection -c configs/ssd_tao/pgie_ssd_tao_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264

Now playing: configs/ssd_tao/pgie_ssd_tao_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
0:00:01.580582751 21560   0x557a1ef300 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Could not read buffer.
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:05.644315085 21560   0x557a1ef300 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault

Morganh · February 9, 2022, 9:01am

Is the tlt-model-key correct?

tmid0819 · February 9, 2022, 9:03am

Yes, I changed it to my own key when I was running the code.

Morganh · February 9, 2022, 9:05am

Is ssd_resnet18.etlt trained by yourself? What is the key when you train and get this tlt model?

tmid0819 · February 9, 2022, 9:13am

I’m using the example model I download with

./download_models.sh

And for key I had tried the original one: nvidia_tlt
and mines: NHZtNnA1b2Noa2lzZWNibjV0OTY1cm9sYW46NGJmNDM3ODEtODllMy00ZWI5LWE5ZDUtODJiYzc0MWE3Zjlj
Both gave the same result.

Morganh · February 9, 2022, 9:17am

Suggest you to git clone again and double check. Just use the default config file.

tmid0819 · February 9, 2022, 9:24am

Than do I need to build TRT OSS Plugin again?

Morganh · February 9, 2022, 9:25am

Not needed. Please backup it.

tmid0819 · February 9, 2022, 9:56am

Hi, after git clone again, this is the full command and log:

$ sudo ./apps/tao_detection/ds-tao-detection -c configs/ssd_tao/pgie_ssd_tao_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264
[sudo] password for ypnano7: 
Now playing: configs/ssd_tao/pgie_ssd_tao_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_tao_apps/configs/ssd_tao/../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine open error
0:00:04.728876757 23976   0x5590aa4f00 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_tao_apps/configs/ssd_tao/../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine failed
0:00:04.728961082 23976   0x5590aa4f00 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream_tao_apps/configs/ssd_tao/../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine failed, try rebuild
0:00:04.728986916 23976   0x5590aa4f00 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Validator error: FirstDimTile_4: Unsupported operation _BatchTilePlugin_TRT
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:05.315894128 23976   0x5590aa4f00 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault

Config:

[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=ssd_labels.txt
model-engine-file=../../models/ssd/ssd_resnet18.etlt_b1_gpu0_fp16.engine
tlt-encoded-model=../../models/ssd/ssd_resnet18.etlt
int8-calib-file=../../models/ssd/cal.bin
tlt-model-key=nvidia_tlt
infer-dims=3;544;960
uff-input-order=0
maintain-aspect-ratio=1
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=5
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
output-blob-names=NMS
parse-bbox-func-name=NvDsInferParseCustomNMSTLT
custom-lib-path=../../post_processor/libnvds_infercustomparser_tao.so

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

Morganh · February 9, 2022, 10:00am

You mention that you have built the TRT OSS plugin.
Have you replaced it ? See https://github.com/NVIDIA-AI-IOT/deepstream_tao_apps/tree/release/tao3.0/TRT-OSS/Jetson#3-replace-libnvinfer_pluginso

tmid0819 · February 9, 2022, 10:09am

Yes, thanks for the reminder anyway.

Morganh · February 9, 2022, 10:13am

I am afraid there is something wrong when you replace the libnvinfer_plugin.
Refer to my post Failling in building sample from TLT-DEEPSTREAM

tmid0819 · February 10, 2022, 6:43am

Refer to your post ,
I checked the command I gave:

sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3.bak
sudo cp /opt/nvidia/deepstream/deepstream-5.1/sources/TensorRT/build/out/libnvinfer_plugin.so.7.1.3  /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3
sudo ldconfig

It looks correct.

But with the ll command ou mention in the post, mine link doesn’t look right:

$ ll -sh /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
   0 lrwxrwxrwx 1 root root   26  6月  6  2020 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so -> libnvinfer_plugin.so.7.1.3*
   0 lrwxrwxrwx 1 root root   30  2月  9 13:56 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7 -> libnvinfer_plugin.so.7.1.3.bak
 14M -rwxr-xr-x 1 root root  14M  2月  9 13:56 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3*
5.4M -rw-r--r-- 1 root root 5.4M  6月  6  2020 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.3.bak

Morganh · February 10, 2022, 6:47am

Please move the original .so outside. See below.

$ sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0 ~/libnvinfer_plugin.so.7.1.0.bak

tmid0819 · February 10, 2022, 6:55am

Do you mean, in my case I should move .so, .so.7, .so.7.1.3 these three file do somewhere outside, and than put the new three file that I have generate in here?

Also, because of my previous command, th original one became 7.1.3.bak, so I should fist mv that file out side?

Morganh · February 10, 2022, 6:56am

No.
See my post Failling in building sample from TLT-DEEPSTREAM - #15 by Morganh

Please follow
$ sudo mv /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0 ~/libnvinfer_plugin.so.7.1.0.bak

$ sudo cp {TRT_SOURCE}/build/out/libnvinfer_plugin.so.7.0.0.1 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0

$ sudo ldconfig