Deepstream ERROR: <main:655>: Failed to set pipeline to PAUSED

hkautoplant2 · November 11, 2020, 10:37am

Hi!
I have trained a YOLOv3 network with our own dataset with images. Now I will export the model to my Jetson Xavier.
I am using a Jetson Xavier with deep-stream 5-0. I am trying to run deepstream- app but i get follwing error. I’m following this guide(option 1)
https://docs.nvidia.com/metropolis/TLT/tlt-getting-started-guide/text/deploying_to_deepstream.html#integrating-the-model-to-deepstream

I dont reallt understand what file I should run,
config_fig_infer_primary.txt or source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_tx1.txt

When I run config_fig_infer_primary.txt i get following

** ERROR: main:655: Failed to set pipeline to PAUSED
Quitting
ERROR from src_bin_muxer: Output width not set
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream/gstnvstreammux.c(2283): gst_nvstreammux_change_state (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstNvStreamMux:src_bin_muxer
App run failed

The config file I am trying to tun is following:

[property]
gpu-id=0
#net-scale-factor=0.0039215697906911373
net-scale-factor=1
#tlt-encoded-model=frcnn_kitti_resnet10_train_fp16.etlt
tlt-encoded-model=…/…/models/YOLO/yolo_resnet18_epoch_080.etlt
tlt-model-key=Z2o0aGRiaHNvcXFzNzViYWM0a3FuYW9vZzk6YWIxMDFhYmQtMDNhOS00OTYxLTg5YzMtODM4NzRmNmFlZTI0

labelfile-path=…/…/models/YOLO/labels.txt

#batch-size=4
batch-size=1
process-mode=1
model-color-format=0

0=FP32, 1=INT8, 2=FP16 mode

#network-mode=1
network-mode=0
#num-detected-classes=4
#um-detected-classes=6
interval=0
gie-unique-id=1
#output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
output-blob-names=BatchedNMS
force-implicit-batch-dim=1
parse-bbox-func-name=NvDsInferParseCustomYOLOV3TLT
custom-lib-path=/home/jetson/deepstream_tlt_apps/nvdsinfer_customparser_yolov3_tlt

0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)

#cluster-mode=1
#scaling-filter=0
#scaling-compute-hw=0

#Use these config params for group rectangles clustering mode
[class-attrs-all]
pre-cluster-threshold=0.2
group-threshold=1
eps=0.2
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

#Use the config params below for dbscan clustering mode
#[class-attrs-all]
#detected-min-w=4
#detected-min-h=4
#minBoxes=3

Per class configurations

#[class-attrs-0]
#pre-cluster-threshold=0.05
#eps=0.7
#dbscan-min-score=0.95

#[class-attrs-1]
#pre-cluster-threshold=0.05
#eps=0.7
#dbscan-min-score=0.5

#[class-attrs-2]
#pre-cluster-threshold=0.1
#eps=0.6
#dbscan-min-score=0.95

#[class-attrs-3]
#pre-cluster-threshold=0.05
#eps=0.7
#dbscan-min-score=0.5

When i run source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_tx1.txt i get follwoing error

** WARN: <parse_gie:1189>: Unknown key ‘tlt-encoded-model’ for group [primary-gie]
** WARN: <create_pipeline:1143>: Num of Tiles less than number of sources, readjusting to 8 rows, 1 columns
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
ERROR: Could not open lib: /home/jetson/deepstream_tlt_apps/nvdsinfer_customparser_yolov3_tlt, error string: /home/jetson/deepstream_tlt_apps/nvdsinfer_customparser_yolov3_tlt: cannot read file data: Is a directory
0:00:00.219964404 19013 0x19d20470 ERROR nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1139> [UID = 1]: Could not open custom lib: (null)
0:00:00.220053177 19013 0x19d20470 WARN nvinfer gstnvinfer.cpp:809:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:00.220074587 19013 0x19d20470 WARN nvinfer gstnvinfer.cpp:809:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primare_fastrcnn.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
** ERROR: main:655: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Failed to create NvDsInferContext instance
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(809): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
Config file path: /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primare_fastrcnn.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
App run failed

the source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_tx1.txt is following:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://…/…/streams/Explorer_HD1080_SN21262960_14-57-18=LEFT.png
num-sources=8
#drop-frame-interval=2
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=5
sync=1
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0

set below properties in case of RTSPStreaming

rtsp-port=8554
udp-port=5400

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=8
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1920
height=1080
#enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

If set to TRUE, system timestamp will be attached as ntp timestamp

If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached

attach-sys-ts-as-ntp=1

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
#model-engine-file=…/…/models/Primary_Detector_Nano/resnet10.caffemodel_b8_gpu0_fp16.engine
tlt-encoded-model=…/…/models/fast-rcnn/frcnn_kitti_resnet10_train_fp16.etlt
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primare_fastrcnn.txt

[tracker]
enable=1

For the case of NvDCF tracker, tracker-width and tracker-height must be a multiple of 32, respectively

tracker-width=640
tracker-height=384
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_iou.so
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_nvdcf.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
#ll-config-file required for DCF/IOU only
#ll-config-file=tracker_config.yml
#ll-config-file=config_infer_primary.txt
gpu-id=0
#enable-batch-process and enable-past-frame applicable to DCF only
enable-batch-process=1
enable-past-frame=0
display-tracking-id=1

[tests]
file-loop=0

Fiona.Chen · November 17, 2020, 3:28am

Since you want to run deepstream app with your YoloV3 model, you need to make sure DeepStreamSDK has been intalled correctly before everything start. Quickstart Guide — DeepStream 6.1.1 Release documentation
If the model is ready, you will get the encoded TLT file *.etlt, the calibration file *.bin(this is only for INT8) and label file. Put these files into a proper folder in your device.

There are two configurations needed.

You need to write the nvinfer(this plugin will load the model and run the inference according to the configuration) config file. For your case, you already wrote config_fig_infer_primary.txt which defined the parameters for the model. You may refer to Gst-nvinfer — DeepStream 6.1.1 Release documentation for the meaning of these parameters. And the /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/config_infer_primary_yoloV3.txt is also a good sample.
The error in your log may be caused by the wrong setting of custom-lib-path, it should be the path and file name of your customized parser library. But what you set is just a path of folder, no file name.
custom-lib-path=/home/jetson/deepstream_tlt_apps/nvdsinfer_customparser_yolov3_tlt
You need a deepstream application config file. I think you already wrote source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_tx1.txt for this purpose. The problem is you have not set nvinfer config file in it. Please check the “[primary-gie]” part, you are still using fastrcnn model. You need to set it to your model config file(absolute path and file name is better than relative path and file name).
config-file=config_infer_primare_fastrcnn.txt
Please refer to DeepStream Reference Application - deepstream-app — DeepStream 6.1.1 Release documentation for the meaning of the parameters.

The deepsteam-app should run with deepstream-app config file. The nvinfer config file is configured inside “[primary-gie]” part of deepstream-app config.