[TLT-yolov4-Deepstream ] ERROR: [TRT]: UffParser: Unsupported number of graph 0

thuy.hoang19 · August 14, 2021, 9:40am

Please provide the following information when requesting support.

• Hardware : Xavier
• Network Type : Yolov4
• TLT Version :docker pull nvcr.io/nvidia/tlt-streamanalytics:v3.0-py3
• Training spec file:yolo_v4_retrain_resnet18_kitti.txt
random_seed: 42
yolov4_config {
big_anchor_shape: “[(18.00, 18.00), (33.00, 33.00), (52.00, 49.00)]”
mid_anchor_shape: “[(75.00, 72.00), (112.50, 102.00), (156.00, 152.00)]”
small_anchor_shape: “[(225.00, 193.00), (335.00, 260.00), (545.00, 376.00)]”
box_matching_iou: 0.25
arch: “resnet”
nlayers: 18
arch_conv_blocks: 2
loss_loc_weight: 0.8
loss_neg_obj_weights: 100.0
loss_class_weights: 0.5
label_smoothing: 0.0
big_grid_xy_extend: 0.05
mid_grid_xy_extend: 0.1
small_grid_xy_extend: 0.2
freeze_bn: false
#freeze_blocks: 0
force_relu: false
}
training_config {
batch_size_per_gpu: 8
num_epochs: 80
enable_qat: false
checkpoint_interval: 10
learning_rate {
soft_start_cosine_annealing_schedule {
min_learning_rate: 1e-7
max_learning_rate: 1e-4
soft_start: 0.3
}
}
regularizer {
type: NO_REG
weight: 3e-9
}
optimizer {
adam {
epsilon: 1e-7
beta1: 0.9
beta2: 0.999
amsgrad: false
}
}
pruned_model_path: “/workspace/yolov4/experiment_dir_pruned/yolov4_resnet18_pruned.tlt”
}
eval_config {
average_precision_mode: INTEGRATE
batch_size: 8
matching_iou_threshold: 0.7
}
nms_config {
confidence_threshold: 0.001
clustering_iou_threshold: 0.5

• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)
I am using TLT to train new model base on backbone model resnet18 using yolov4 network .
For trainning model, I followed step by step of yolov4 sameples, and it is ok until "yolo_v4 inference "
The problem occured when I tried to push etlt model to deepstream app:
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.104041869 31910 0x7f30002330 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Bus error (core dumped)

Please help to support how to debug it.

Morganh · August 14, 2021, 11:55am

Can you share the config file when you run inference with deepstream? And which command did you run?

thuy.hoang19 · August 14, 2021, 12:00pm

command : deepstream-app -c /opt/nvidia/deepstream/deepstream-5.1/samples/models/tlt-experiments_960x544/ds_configs/deepstream_app_source1_video_masknet_gpu.txt

Primary config:
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=labels_masknet.txt

GPU Engine File

#odel-engine-file=…/models/yolov4_resnet18_epoch_80.engine

DLA Engine File

#model-engine-file=/home/nvidia/detectnet_v2_models/detectnet_4K-fddb-12/resnet18_RGB960_detector_fddb_12_int8.etlt_b1_dla0_int8.engine
tlt-encoded-model=…/models/yolov4_resnet18_epoch_080.etlt
#int8-calib-file=…/models/calibration.bin
tlt-model-key=tlt_encode
infer-dims=3;544;960
uff-input-order=0
uff-input-blob-name=Input
batch-size=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0

num-detected-classes=2
cluster-mode=2
interval=0
gie-unique-id=1

is-classifier=0

output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=libnvds_infercustomparser_tlt.so

[class-attrs-0]
nms-iou-threshold=0.5
pre-cluster-threshold=0.6
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

[class-attrs-1]
nms-iou-threshold=0.5
pre-cluster-threshold=0.6
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

I saw some note that " To integrate a model trained by TLT with DeepStream, you shoud generate a device-specific optimized TensorRT engine using tlt-converter . The generated TensorRT engine file can then be ingested by DeepStream (Currently, YOLOv4 etlt files are not supported by DeepStream). " => is it the root cause?

Morganh · August 14, 2021, 2:11pm

YOLOv4 etlt files should be supported by DeepStream.

Can you try to run inference with official yolo_v4 etlt model ?

Download model from GitHub - NVIDIA-AI-IOT/deepstream_tao_apps at release/tlt3.0
→ To download sample models that are trained by trained by NVIDIA Transfer Learning Toolkit(TLT) SDK run wget https://nvidia.box.com/shared/static/i1cer4s3ox4v8svbfkuj5js8yqm3yazo.zip -O models.zip

See config file in

github.com

NVIDIA-AI-IOT/deepstream_tao_apps/blob/release/tlt3.0/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt

################################################################################
# Copyright (c) 2021, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.

This file has been truncated. show original

More info is in NVIDIA TAO Documentation

thuy.hoang19 · August 14, 2021, 2:34pm

Thank for your response! let me try

thuy.hoang19 · August 15, 2021, 4:42am

I have tried it, but the same error occured:
ERROR: Deserialize engine failed because file path: /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine open error
0:00:02.620954195 31193 0x559c656870 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine failed
0:00:02.621178678 31193 0x559c656870 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine failed, try rebuild
0:00:02.621216566 31193 0x559c656870 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API

Please give some advise for it

Morganh · August 16, 2021, 10:00am

Can you run with GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream instead of deepsteam-app ?
Please use the config file and model inside it.

thuy.hoang19 · August 16, 2021, 10:04am

Actually , the result above is outputed of GitHub - NVIDIA-AI-IOT/deepstream_tlt_apps: Sample apps to demonstrate how to deploy models trained with T.

But it seems to be that I have to use TensorRT and tlt-convert since the note in Readme said that. But I don’t know how to run tlt-convert in Jetson yet even I read the instruction

Morganh · August 16, 2021, 10:08am

No, etlt model can be also configured to run inference.
See https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt

You can comment out
# model-engine-file=…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine

Only set
tlt-encoded-model=…/…/models/yolov4/yolov4_resnet18.etlt
tlt-model-key=nvidia_tlt

More info, please see tlt user guide NVIDIA TAO Documentation

thuy.hoang19 · August 16, 2021, 10:14am

I tried to comment out
# model-engine-file=…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine
But the result is the same.
nvidia@nvidia-desktop:~/deepstream_tlt_apps/apps/tlt_detection$ ./ds-tlt-detection -c /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264 -b 2
WARNING: Overriding infer-config batch-size (1) with number of sources (2)
Now playing: /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:00.431532902 11426 0x5582230c70 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.513344708 11426 0x5582230c70 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault (core dumped)

Morganh · August 16, 2021, 10:16am

Please try to grant access.
$ sudo chown nvidia:nvidia -R /home/nvidia/deepstream_tlt_apps

thuy.hoang19 · August 16, 2021, 10:18am

This is the result:
nvidia@nvidia-desktop:~/deepstream_tlt_apps/apps/tlt_detection$ ./ds-tlt-detection -c /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264 -b 2
WARNING: Overriding infer-config batch-size (1) with number of sources (2)
Now playing: /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:00.402544942 12097 0x55b6fb8c70 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.399339990 12097 0x55b6fb8c70 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Bus error (core dumped)

thuy.hoang19 · August 16, 2021, 10:20am

This is my step to run app,

Clone deepstream_tlt_apps
2.Download model
Build post processs
Run ./ds-tlt-detection -c /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264 -b 2

Is there any steps that I missed ?

Morganh · August 16, 2021, 2:30pm

Did you ever modify /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt ?

More, refer to below searching result in forum.
https://forums.developer.nvidia.com/search?q=%22Bus%20error%22%20%23intelligent-video-analytics%3Atransfer-learning-toolkit%20order%3Alatest

thuy.hoang19 · August 16, 2021, 3:15pm

No I didn’t change anything except comment out engine path:
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=yolov4_labels.txt
#model-engine-file=…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine
int8-calib-file=…/…/models/yolov4/cal.bin
tlt-encoded-model=…/…/models/yolov4/yolov4_resnet18.etlt
tlt-model-key=nvidia_tlt
infer-dims=3;544;960
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=4
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=…/…/post_processor/libnvds_infercustomparser_tlt.so

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

thuy.hoang19 · August 17, 2021, 7:43am

It is ok after trying again with this guide :
https://docs.nvidia.com/metropolis/TLT/tlt-user-guide/text/object_detection/yolo_v4.html#generating-an-engine-using-tlt-converter

Thank for your support~

Topic		Replies	Views
UffParser: Unsupported number of graph 0 TAO Toolkit	53	1132	March 21, 2022
Benchmarking TLT YOLOv3 TAO Toolkit cuda , yolo	2	1492	October 12, 2021
Unable to deploy TAO 4.0.1 yolov4 model on deepstream6.0 TAO Toolkit deepstream	43	1079	August 18, 2023
TAO Toolkit Yolov4 DeepStream SDK	20	457	November 30, 2022
Using deepstream_python_app test3 in deepstream to load etlt model prompts configuration file parsing failure TAO Toolkit	25	952	August 17, 2021
Not able to deploy .etlt file in deepstream test app 1 TAO Toolkit	12	1818	October 12, 2021
Deepstream infrence gives no detection TAO Toolkit	28	1932	December 9, 2021
TLT-deepstream sample app error : Deepstream deplyment : Error build engine file failed TAO Toolkit	15	1398	October 12, 2021
TLT V2.0 Classification TAO Toolkit	26	2785	August 3, 2021
Convert TAO Yolov4 model to DLA engine fails TAO Toolkit	22	1667	March 1, 2022

[TLT-yolov4-Deepstream ] ERROR: [TRT]: UffParser: Unsupported number of graph 0

GPU Engine File

DLA Engine File

0=FP32, 1=INT8, 2=FP16 mode

0=FP32, 1=INT8, 2=FP16 mode

Related topics