ERROR: [TRT]: UffParser: Validator error: FirstDimTile_1: Unsupported operation _BatchTilePlugin_TRT

Raymond_456 · June 1, 2022, 7:13am

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only) JetPack 4.6.3
• TensorRT Version 8.2.1
• NVIDIA GPU Driver Version (valid for GPU only) cuda 10.2
• Issue Type( questions, new requirements, bugs) questions

Hello, I trained in docker(nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3)and got the etlt file of the model yolov4_resnet50. When I try to deploy the file to agx, I get an error.

                                                                                                                                                                                                     (gst-plugin-scanner:9088): GStreamer-WARNING **: 22:24:24.232: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory                                                                                                                                                                                                                                                                                                                                                                                                      (gst-plugin-scanner:9088): GStreamer-WARNING **: 22:24:24.234: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory                                                                                                                                                                                                                                                                                                                                                                                                                 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***                                                                                                                                                                                                                                                                                                                                                         Opening in BLOCKING MODE                                                                                                                                                                                           gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so                                                                                                   gstnvtracker: Batch processing is ON                                                                                                                                                                               gstnvtracker: Past frame output is ON                                                                                                                                                                              [NvMultiObjectTracker] Initialized                                                                                                                                                                                 0:00:03.203432639  9087   0x7f0c0022d0 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1161> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5                                                     ERROR: Deserialize engine failed because file path: /home/py/Downloads/facenet/resnet18_detector.etlt_b1_gpu0_fp32.engine open error                                                                               0:00:05.079530093  9087   0x7f0c0022d0 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/home/py/Downloads/facenet/resnet18_detector.etlt_b1_gpu0_fp32.engine failed                                                             0:00:05.099664027  9087   0x7f0c0022d0 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/home/py/Downloads/facenet/resnet18_detector.etlt_b1_gpu0_fp32.engine failed, try rebuild                                0:00:05.099816356  9087   0x7f0c0022d0 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files                                                                                                                                                   ERROR: [TRT]: UffParser: Validator error: FirstDimTile_1: Unsupported operation _BatchTilePlugin_TRT                                                                                                               parseModel: Failed to parse UFF model                                                                                                                                                                              ERROR: Failed to build network, error in model parsing.                                                                                                                                                            ERROR: Failed to create network using custom network creation function                                                                                                                                             ERROR: Failed to get cuda engine from custom library API                                                                                                                                                           0:00:05.855356847  9087   0x7f0c0022d0 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1934> [UID = 1]: build engine file failed                                                                                                                                                                    Segmentation fault (core dumped)

Then I checked the tensorrt version of TLT Docker and AGX
TLT Docker is

AGX is

What should I do ？

Looking forward to your reply

Thankyou

Amycao · June 1, 2022, 8:36am

Please follow GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream
to build TRT OSS lib or directly use the library under TRT-OSS/Jetson/TRT8.2/

Raymond_456 · June 1, 2022, 9:08am

Deploy in TLT Docker or AGX?

Raymond_456 · June 1, 2022, 9:19am

Currently the tensorrt version on my AGX is 8.2. So I just need to replace the .so file?

Amycao · June 1, 2022, 9:46am

Yes.

Raymond_456 · June 2, 2022, 6:33am

I replaced the .SO file but still get the error

Amycao · June 2, 2022, 9:32am

How about sudo ldconfig and run again?

Raymond_456 · June 6, 2022, 1:12am

still error

yl.liu · June 6, 2022, 6:58am

AGX orin has the same error

Raymond_456 · June 6, 2022, 7:00am

Hi, what more information do I need to provide?

Amycao · June 6, 2022, 7:24am

I am not sure why it did not work on your side. can you move libnvinfer_plugin.so.8.2.1_backup out of this folder? if still not work, please build it follow the readme and try again.

Raymond_456 · June 15, 2022, 8:32am

now I can convert to .engine file from etlt file
But i have a new problem is when i run deepstream it returns me the following error

NvMMLiteOpen : Block : BlockType = 4
                                                                                    
===== NVMEDIA: NVENC =====                                                                                              
NvMMLiteBlockCreate : Block : BlockType = 4                                                                                                                                                                                                     **PERF:  FPS 0 (Avg)    FPS 1 (Avg)                                                                                     **PERF:  0.00 (0.00)    0.00 (0.00)                                                                                     **PERF:  0.00 (0.00)    0.00 (0.00)                                                                                     **PERF:  0.00 (0.00)    0.00 (0.00)                                                                                     **PERF:  0.00 (0.00)    0.00 (0.00)                                                                                     **PERF:  0.00 (0.00)    0.00 (0.00)                                                                                     
**PERF:  0.00 (0.00)    0.00 (0.00)                                                                                 
**PERF:  0.00 (0.00)    0.00 (0.00)                                                                                     
0:00:40.034610171 29666      0x16c92a0 ERROR  nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: 
Error in NvDsInferContextImpl::parseBoundingBox() <nvdsinfer_context_impl_output_parsing.cpp:59> [UID = 1]: Could not find output coverage layer for parsing objects                                                   
0:00:40.034675482 29666      0x16c92a0 ERROR                nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: 
Error in NvDsInferContextImpl::fillDetectionOutput() <nvdsinfer_context_impl_output_parsing.cpp:735> [UID = 1]: Failed to parse bboxes

Amycao · June 22, 2022, 3:05am

Sorry for the late.
Did this still be an issue?

Raymond_456 · June 23, 2022, 6:43am

Sorry for not replying to you in time, this problem has been solved, the engine file I am using now can run, but the deepstream screen only has video screen without bbox

Amycao · June 23, 2022, 9:19am

Can you get correct output by TensorRT directly?
You used GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream, right?
You also can print object bbox coordinates, to see if parsed bbox coordinates correct or not. NvDsInferParseCustomBatchedNMSTLT in nvdsinfer_custombboxparser_tao.cpp

system · July 7, 2022, 9:19am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Failed to deploy efficientdet-tf1 in deepstream DeepStream SDK	2	369	November 6, 2023
Error from deepstream sample apps DeepStream SDK	9	1008	October 12, 2021
deepstream4.0 can't run .etlt model file Jetson TX2	4	628	November 5, 2019
Deepstream Python Transfer Learning on Jetson AGX DeepStream SDK	5	243	November 23, 2022
Deepstream sample code error on Jetson Orin AGX DeepStream SDK	4	999	June 19, 2023
Cannot run deepstream-test-1 in deepstream_python_apps: Where is the ../../../../samples/ folder? DeepStream SDK	5	2270	October 12, 2021
The tlt-converter does not work well with TensorRT 6 (Jetson TX2) TAO Toolkit	7	770	October 12, 2021
Gaze Estimation App on deepstream 6.0: Engine File Error DeepStream SDK tensorrt , deepstream	5	20	October 25, 2024
Deepstream lpr test error DeepStream SDK	7	821	October 12, 2021
Error running deepstream sample apps DeepStream SDK	4	1468	September 28, 2021

ERROR: [TRT]: UffParser: Validator error: FirstDimTile_1: Unsupported operation _BatchTilePlugin_TRT

Related topics