[TLT-yolov4-Deepstream ] ERROR: [TRT]: UffParser: Unsupported number of graph 0

Please provide the following information when requesting support.

• Hardware : Xavier
• Network Type : Yolov4
• TLT Version :docker pull nvcr.io/nvidia/tlt-streamanalytics:v3.0-py3
• Training spec file:yolo_v4_retrain_resnet18_kitti.txt
random_seed: 42
yolov4_config {
big_anchor_shape: “[(18.00, 18.00), (33.00, 33.00), (52.00, 49.00)]”
mid_anchor_shape: “[(75.00, 72.00), (112.50, 102.00), (156.00, 152.00)]”
small_anchor_shape: “[(225.00, 193.00), (335.00, 260.00), (545.00, 376.00)]”
box_matching_iou: 0.25
arch: “resnet”
nlayers: 18
arch_conv_blocks: 2
loss_loc_weight: 0.8
loss_neg_obj_weights: 100.0
loss_class_weights: 0.5
label_smoothing: 0.0
big_grid_xy_extend: 0.05
mid_grid_xy_extend: 0.1
small_grid_xy_extend: 0.2
freeze_bn: false
#freeze_blocks: 0
force_relu: false
}
training_config {
batch_size_per_gpu: 8
num_epochs: 80
enable_qat: false
checkpoint_interval: 10
learning_rate {
soft_start_cosine_annealing_schedule {
min_learning_rate: 1e-7
max_learning_rate: 1e-4
soft_start: 0.3
}
}
regularizer {
type: NO_REG
weight: 3e-9
}
optimizer {
adam {
epsilon: 1e-7
beta1: 0.9
beta2: 0.999
amsgrad: false
}
}
pruned_model_path: “/workspace/yolov4/experiment_dir_pruned/yolov4_resnet18_pruned.tlt”
}
eval_config {
average_precision_mode: INTEGRATE
batch_size: 8
matching_iou_threshold: 0.7
}
nms_config {
confidence_threshold: 0.001
clustering_iou_threshold: 0.5

• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)
I am using TLT to train new model base on backbone model resnet18 using yolov4 network .
For trainning model, I followed step by step of yolov4 sameples, and it is ok until "yolo_v4 inference "
The problem occured when I tried to push etlt model to deepstream app:
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.104041869 31910 0x7f30002330 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Bus error (core dumped)

Please help to support how to debug it.

Can you share the config file when you run inference with deepstream? And which command did you run?

command : deepstream-app -c /opt/nvidia/deepstream/deepstream-5.1/samples/models/tlt-experiments_960x544/ds_configs/deepstream_app_source1_video_masknet_gpu.txt

Primary config:
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=labels_masknet.txt

GPU Engine File

#odel-engine-file=…/models/yolov4_resnet18_epoch_80.engine

DLA Engine File

#model-engine-file=/home/nvidia/detectnet_v2_models/detectnet_4K-fddb-12/resnet18_RGB960_detector_fddb_12_int8.etlt_b1_dla0_int8.engine
tlt-encoded-model=…/models/yolov4_resnet18_epoch_080.etlt
#int8-calib-file=…/models/calibration.bin
tlt-model-key=tlt_encode
infer-dims=3;544;960
uff-input-order=0
uff-input-blob-name=Input
batch-size=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0

num-detected-classes=2
cluster-mode=2
interval=0
gie-unique-id=1

is-classifier=0

output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=libnvds_infercustomparser_tlt.so

[class-attrs-0]
nms-iou-threshold=0.5
pre-cluster-threshold=0.6
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

[class-attrs-1]
nms-iou-threshold=0.5
pre-cluster-threshold=0.6
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

I saw some note that " To integrate a model trained by TLT with DeepStream, you shoud generate a device-specific optimized TensorRT engine using tlt-converter . The generated TensorRT engine file can then be ingested by DeepStream (Currently, YOLOv4 etlt files are not supported by DeepStream). " => is it the root cause?

YOLOv4 etlt files should be supported by DeepStream.

Can you try to run inference with official yolo_v4 etlt model ?

Download model from GitHub - NVIDIA-AI-IOT/deepstream_tlt_apps at release/tlt3.0
→ To download sample models that are trained by trained by NVIDIA Transfer Learning Toolkit(TLT) SDK run wget https://nvidia.box.com/shared/static/i1cer4s3ox4v8svbfkuj5js8yqm3yazo.zip -O models.zip

See config file in

More info is in YOLOv4 — Transfer Learning Toolkit 3.0 documentation

Thank for your response! let me try

I have tried it, but the same error occured:
ERROR: Deserialize engine failed because file path: /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine open error
0:00:02.620954195 31193 0x559c656870 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1691> [UID = 1]: deserialize engine from file :/home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine failed
0:00:02.621178678 31193 0x559c656870 WARN nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1798> [UID = 1]: deserialize backend context from engine from file :/home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine failed, try rebuild
0:00:02.621216566 31193 0x559c656870 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API

Please give some advise for it

Can you run with GitHub - NVIDIA-AI-IOT/deepstream_tlt_apps: Sample apps to demonstrate how to deploy models trained with T instead of deepsteam-app ?
Please use the config file and model inside it.

Actually , the result above is outputed of GitHub - NVIDIA-AI-IOT/deepstream_tlt_apps: Sample apps to demonstrate how to deploy models trained with T.

But it seems to be that I have to use TensorRT and tlt-convert since the note in Readme said that. But I don’t know how to run tlt-convert in Jetson yet even I read the instruction

No, etlt model can be also configured to run inference.
See deepstream_tlt_apps/pgie_yolov4_tlt_config.txt at master · NVIDIA-AI-IOT/deepstream_tlt_apps · GitHub

You can comment out
# model-engine-file=…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine

Only set
tlt-encoded-model=…/…/models/yolov4/yolov4_resnet18.etlt
tlt-model-key=nvidia_tlt

More info, please see tlt user guide YOLOv4 — Transfer Learning Toolkit 3.0 documentation

I tried to comment out
# model-engine-file=…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine
But the result is the same.
nvidia@nvidia-desktop:~/deepstream_tlt_apps/apps/tlt_detection$ ./ds-tlt-detection -c /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264 -b 2
WARNING: Overriding infer-config batch-size (1) with number of sources (2)
Now playing: /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:00.431532902 11426 0x5582230c70 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.513344708 11426 0x5582230c70 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Segmentation fault (core dumped)

Please try to grant access.
$ sudo chown nvidia:nvidia -R /home/nvidia/deepstream_tlt_apps

This is the result:
nvidia@nvidia-desktop:~/deepstream_tlt_apps/apps/tlt_detection$ ./ds-tlt-detection -c /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264 -b 2
WARNING: Overriding infer-config batch-size (1) with number of sources (2)
Now playing: /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:00.402544942 12097 0x55b6fb8c70 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 1]: Trying to create engine from model files
ERROR: [TRT]: UffParser: Unsupported number of graph 0
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:02.399339990 12097 0x55b6fb8c70 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger: NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 1]: build engine file failed
Bus error (core dumped)

This is my step to run app,

  1. Clone deepstream_tlt_apps
    2.Download model
  2. Build post processs
  3. Run ./ds-tlt-detection -c /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt -i /opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_720p.h264 -b 2

Is there any steps that I missed ?

Did you ever modify /home/nvidia/deepstream_tlt_apps/configs/yolov4_tlt/pgie_yolov4_tlt_config.txt ?

More, refer to below searching result in forum.
https://forums.developer.nvidia.com/search?q=%22Bus%20error%22%20%23intelligent-video-analytics:transfer-learning-toolkit%20order:latest

No I didn’t change anything except comment out engine path:
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=yolov4_labels.txt
#model-engine-file=…/…/models/yolov4/yolov4_resnet18.etlt_b1_gpu0_fp16.engine
int8-calib-file=…/…/models/yolov4/cal.bin
tlt-encoded-model=…/…/models/yolov4/yolov4_resnet18.etlt
tlt-model-key=nvidia_tlt
infer-dims=3;544;960
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=4
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=…/…/post_processor/libnvds_infercustomparser_tlt.so

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

It is ok after trying again with this guide :
https://docs.nvidia.com/metropolis/TLT/tlt-user-guide/text/object_detection/yolo_v4.html#generating-an-engine-using-tlt-converter

Thank for your support~

1 Like