Fail to run deepstream with own Yolov4 model

I have a problem when trying to run deepstream with my own retrained Yolov4 model from TAO Toolkit. I am using the deepstream docker on my NVIDIA GPU and ran the test detection model succesfully. But now I am getting the following error:

(gst-plugin-scanner:17): GStreamer-WARNING **: 15:39:09.688: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so’: librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:17): GStreamer-WARNING **: 15:39:10.569: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so’: libtritonserver.so: cannot open shared object file: No such file or directory
Attributes specified for class 1 while element has been configured with num-detected-classes=1
** ERROR: <gst_nvinfer_parse_config_file:1303>: failed
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
[NvMultiObjectTracker] Initialized
0:00:05.031461080 16 0x5580324382a0 WARN nvinfer gstnvinfer.cpp:794:gst_nvinfer_start:<primary_gie> error: Configuration file parsing failed
0:00:05.031477450 16 0x5580324382a0 WARN nvinfer gstnvinfer.cpp:794:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-6.0/samples/configs/tao_pretrained_models/config_infer_primary_yolov4_lp.txt
[NvMultiObjectTracker] De-initialized
** ERROR: main:707: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie: Configuration file parsing failed
Debug info: gstnvinfer.cpp(794): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
Config file path: /opt/nvidia/deepstream/deepstream-6.0/samples/configs/tao_pretrained_models/config_infer_primary_yolov4_lp.txt
App run failed

My two config files look as following:

config_infer_primary_yolov4_lp.txt:
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=lp_labels.txt
model-engine-file=…/…/models/tao_pretrained_models/yolov4/yolov4_resnet18.etlt_b1_gpu0_int8.engine
int8-calib-file=…/…/models/tao_pretrained_models/yolov4/cal.bin
tlt-encoded-model=…/…/models/tao_pretrained_models/yolov4/yolov4_resnet18_epoch_080.etlt
tlt-model-key=
infer-dims=3;544;960
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1
##0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
num-detected-classes=1
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_infercustomparser.so

[class-attrs-all]
pre-cluster-threshold=0.3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

[class-attrs-1]
nms-iou-threshold=0.9

And deepstream_app_source1_yolov4.txt

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
num-sources=1
uri=file://…/…/streams/testvideo2.mp4
gpu-id=0

[streammux]
gpu-id=0
batch-size=1
batched-push-timeout=40000
#Set muxer output width and height
width=1280
height=720

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=1
sync=1
source-id=0
gpu-id=0

[osd]
enable=1
gpu-id=0
border-width=3
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial

[primary-gie]
enable=1
gpu-id=0
#Modify as necessary
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
#Replace the infer primary config file when you need to
#use other detection models
#config-file=config_infer_primary_frcnn.txt
#config-file=config_infer_primary_ssd.txt
#config-file=config_infer_primary_dssd.txt
#config-file=config_infer_primary_retinanet.txt
#config-file=config_infer_primary_yolov3.txt
config-file=config_infer_primary_yolov4_lp.txt
#config-file=config_infer_primary_detectnet_v2.txt

[sink1]
enable=1
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
#set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

[tracker]
enable=1
#For NvDCF and DeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
#ll-config-file required to set different tracker types
#ll-config-file=…/deepstream-app/config_tracker_IOU.yml
ll-config-file=…/deepstream-app/config_tracker_NvDCF_perf.yml
#ll-config-file=…/deepstream-app/config_tracker_NvDCF_accuracy.yml
#ll-config-file=…/deepstream-app/config_tracker_DeepSORT.yml
gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

[tests]
file-loop=0

Can someone tell me where the error might be? I already looked through this forum but could find a solution that fits me.

Please double check the access of config_infer_primary_yolov4_lp.txt.

To narrow down, you can try to replace it with other config to check if it works.

Both the config files are in the same directory so the accessibility should be no problem.

When trying to run the normal config_infer_primary_yolov4.txt I get the following:

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
[NvMultiObjectTracker] Initialized
0:00:01.300790044 62 0x55b3a0694890 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-6.0/samples/models/tao_pretrained_models/yolov4/yolov4_resnet18.etlt_b1_gpu0_int8.engine
INFO: …/nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 5
0 INPUT kFLOAT Input 3x544x960
1 OUTPUT kINT32 BatchedNMS 1
2 OUTPUT kFLOAT BatchedNMS_1 200x4
3 OUTPUT kFLOAT BatchedNMS_2 200
4 OUTPUT kFLOAT BatchedNMS_3 200

0:00:01.300859314 62 0x55b3a0694890 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2004> [UID = 1]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-6.0/samples/models/tao_pretrained_models/yolov4/yolov4_resnet18.etlt_b1_gpu0_int8.engine
0:00:01.319496242 62 0x55b3a0694890 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-6.0/samples/configs/tao_pretrained_models/config_infer_primary_yolov4.txt sucessfully
[NvMultiObjectTracker] De-initialized
** ERROR: main:707: Failed to set pipeline to PAUSED
Quitting
ERROR from source: Resource not found.
Debug info: gstfilesrc.c(533): gst_file_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
No such file “/opt/nvidia/deepstream/deepstream-6.0/samples/configs/tao_pretrained_models/…/…/streams/testvideo2.mp4”
ERROR from source: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Debug info: gstbasesrc.c(3452): gst_base_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
Failed to start
ERROR from source: Resource not found.
Debug info: gstfilesrc.c(533): gst_file_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
No such file “/opt/nvidia/deepstream/deepstream-6.0/samples/configs/tao_pretrained_models/…/…/streams/testvideo2.mp4”
ERROR from source: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Debug info: gstbasesrc.c(3452): gst_base_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
Failed to start
App run failed

Please check if the mp4 file is available.

Yes got it thank you. So now the basic configuration from NVIDIA ist running just fine.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.