TLT-deepstream sample app error

Accelerated Computing Intelligent Video Analytics TAO Toolkit
,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Titan XP
• DeepStream Version: 5.0
• TensorRT Version: 7.0
• NVIDIA GPU Driver Version (valid for GPU only): 440

Hi, I am new to Deepstream and I am trying to build the sample app from this tlt-deepstream repo.

dwang@dwang18:~/deepstream_tlt_apps$ ./deepstream-custom -c pgie_frcnn_tlt_config.txt -i $DS_SRC_PATH/samples/streams/sample_720p.h264
I met this error:

Now playing: pgie_frcnn_tlt_config.txt
0:00:00.304933936  8318 0x556ca151d870 INFO                 nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:759 FP16 not supported by platform. Using FP32 mode.
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:31 [TRT]: UffParser: UFF buffer empty
parseModel: Failed to parse UFF model
ERROR: tlt/tlt_decode.cpp:274 failed to build network since parsing model errors.
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:797 Failed to create network using custom network creation function
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:862 Failed to get cuda engine from custom library API
0:00:00.381900463  8318 0x556ca151d870 ERROR                nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1611> [UID = 1]: build engine file failed
Segmentation fault (core dumped)

I followed the steps of TensorRT installation but it looks like it failed in the TRT package. I am not sure how to fix these errors. I appreciate it if someone could help me out.

3

@wdw0908
Please provide your pgie_frcnn_tlt_config.txt. Thanks.

4 
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=./nvdsinfer_customparser_frcnn_tlt/frcnn_labels.txt
tlt-encoded-model=./models/frcnn/faster_rcnn_resnet10.etlt
tlt-model-key=tlt
uff-input-dims=3;272;480;0
uff-input-blob-name=input_image
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=5
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
output-blob-names=dense_regress_td/BiasAdd;dense_class_td/Softmax;proposal
parse-bbox-func-name=NvDsInferParseCustomFrcnnTLT
custom-lib-path=./nvdsinfer_customparser_frcnn_tlt/libnvds_infercustomparser_frcnn_tlt.so

[class-attrs-all]
pre-cluster-threshold=0.6
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0
5

So, your config file is mostly he same as https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/pgie_frcnn_tlt_config.txt

Did you follow deepstream_tao_apps/README.md at master · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub to set up environment?

Also, you mention that “I followed the steps of TensorRT installation but it looks like it failed in the TRT package.” Could you please give more details about it? Appreciate the command, logs, etc.

7

Hi, i met the same error and could you tell me how exactly did you solve this problem. I already followed the steps in deepstream_tao_apps/TRT-OSS/Jetson at master · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub. The same error happened…

8

Move into TLT forum.

9

@xiaomaozhou26
Please provide your pgie_frcnn_tlt_config.txt . Thanks.

10

hi, thanks for the quick reply. Please find the following info:

run command:
./deepstream-custom -c pgie_ssd_tlt_config.txt -i sample_720p.h264

pgie_ssd_tlt_config.txt:
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=./nvdsinfer_customparser_ssd_tlt/ssd_labels.txt
tlt-encoded-model=./models/ssd/ssd_resnet18.etlt
tlt-model-key=nvidia_tlt
uff-input-dims=3;544;960;0
uff-input-blob-name=Input
batch-size=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=4
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
output-blob-names=NMS

parse-bbox-func-name=NvDsInferParseCustomSSDTLT
custom-lib-path=./nvdsinfer_customparser_ssd_tlt/libnvds_infercustomparser_ssd_tlt.so
[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0
#########################################

Now playing: pgie_ssd_tlt_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:00.206036353 1796 0x55782a1c00 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Validator error: FirstDimTile_4: Unsupported operation _BatchTilePlugin_TRT
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.013214101 1796 0x55782a1c00 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1611> [UID = 1]: build engine file failed
Segmentation fault (core dumped)

11

What is your DS/CUDA/trt/cudnn version?
Did you ever flash your device with Jetpack 4.4 and install DS5/cuda/trt7 and cudnn8?

12

yes, I flashed my device with Jetpack4.4, which installed DS5, CUDA10.2 and Cudnn8.

13

According to error log , I think this error is because the TensorRT plugin is not updated.Please double check your step for it.

14

Which device are you using?

15

Hi, I am using jetson TX2 and I think I followed the Tensorrt installation steps: https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/tree/master/TRT-OSS/Jetson

I tried to run the tlt-converter, it failed with the same error. This means the TRT is not running correctly.

16

Please check if you were using the correct GPU_ARCHS when you build the TRT OSS plugin.
|Jetson Platform |GPU_ARCHS|
|TX2 |62 |

17

In GitHub - NVIDIA/TensorRT: TensorRT is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.
##########################
After building successfully, libnvinfer_plugin.so* will be generated under pwd /out/.

sudo cp `pwd`/out/libnvinfer_plugin.so.7.m.n  /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.x.y

##########################
After building, I have the following 3 files, but I am not sure I copied them correctly. Could you help me with this step.
libnvinfer_plugin.so libnvinfer_plugin.so.7.0.0 libnvinfer_plugin.so.7.0.0.1

18

The 3 files are the same.
Refer to Failling in building sample from TLT-DEEPSTREAM - #15 by Morganh

19

Thanks. I am using Xaiver so I think the GPU_ARCHS is 72 and it’s correct.

I ran the copying command:
sudo cp /home/dewei/TensorRT/build/out/libnvinfer_plugin.so.7.0.0.1 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0

And I got the following result:

dewei@dewei-desktop:~$ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
lrwxrwxrwx 1 root root      26 Jun 10 00:02 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.0.0 -> libnvinfer_plugin.so.7.1.0*
-rwxr-xr-x 1 root root 4652648 Jun 10 00:00 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.7.1.0*

Then, I try to ran the sample by
dewei@dewei-desktop:~/Documents/deepstream_tlt_apps$ ./deepstream-custom -c pgie_frcnn_tlt_config.txt -i $DS_SRC_PATH/samples/streams/sample_720p.h264

The error is still there as same:

(deepstream-custom:9671): GStreamer-WARNING **: 00:03:54.015: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_infer.so': libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
One element could not be created. Exiting.

Thanks

20

Please review Failling in building sample from TLT-DEEPSTREAM

21

Thanks for instruction but I am confused. The error seems still there to run the sample custom app:

(deepstream-custom:9671): GStreamer-WARNING **: 00:03:54.015: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_infer.so': libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
One element could not be created. Exiting.
22

@wdw0908
For your case, I already tell the solution in Failling in building sample from TLT-DEEPSTREAM - #15 by Morganh

Please resume the softlinks as the original, then follow my step and retry.