DeepStream 4.0 + Custom Yolo Model - App run failed

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hey, I’m using DeepStream 4.0 with Nvidia Jetson Nano device. I followed the guide available at https://docs.nvidia.com/metropolis/deepstream/4.0/Custom_YOLO_Model_in_the_DeepStream_YOLO_App.pdf to create a custom Yolov3-tiny Model but every time I run it, it crashed and gives error.
First, I followed the README.md available inside objectDetection_Yolo/ folder and ran the model on default configuration of yolov3-tiny.cfg and yolov3-tiny.weights, it works fine but when I try my already trained custom model it crashed with the error

Using winsys: x11 
Creating LL OSD context new
0:00:00.825594078 22645      0xe89ea10 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:00.826501235 22645      0xe89ea10 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
Yolo type is not defined from config file name:yolo.cfg
0:00:00.826751710 22645      0xe89ea10 ERROR                nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Failed to create network using custom network creation function
0:00:00.826801763 22645      0xe89ea10 ERROR                nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Failed to create engine from model files
0:00:00.827080311 22645      0xe89ea10 WARN                 nvinfer gstnvinfer.cpp:692:gst_nvinfer_start:<primary_gie_classifier> error: Failed to create NvDsInferContext instance
0:00:00.827113541 22645      0xe89ea10 WARN                 nvinfer gstnvinfer.cpp:692:gst_nvinfer_start:<primary_gie_classifier> error: Config file path: /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/config_infer_primary_yoloV3_tiny.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
** ERROR: <main:651>: Failed to set pipeline to PAUSED
Quitting
ERROR from primary_gie_classifier: Failed to create NvDsInferContext instance
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(692): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie_classifier:
Config file path: /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/config_infer_primary_yoloV3_tiny.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
App run failed

deepstream-app version 4.0
DeepStreamSDK 4.0
CUDA Version: 10.0

<i><b>config_infer_primary_yoloV3_tiny.txt</b></i>

[property]
gpu-id=0
net-scale-factor=1
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=yolo.cfg
model-file=yolo_final.weights
#model-engine-file=model_b1_fp32.engine
labelfile-path=classnames.txt
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=1
num-detected-classes=114
gie-unique-id=1
is-classifier=0
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3Tiny
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so

<b><i>deepstream_app_config_yoloV3_tiny.txt</i></b>

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
uri=file://../../samples/streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_fp32.engine
labelfile-path=classnames.txt
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3_tiny.txt

[tests]
file-loop=0

I have a custom Yolo model build upon yolov3-tiny.

2

Hi,

The error message is this:

Yolo type is not defined from config file name:yolo.cfg

Have you changed the architecture of YOLOv3 tiny.
Would you mind to check if there is some layer named as “YOLO” inside your customized model?

Thanks.

3

Thank you for your reply

Yes, I have modified the architecture of Yolov3-tiny and yes there is a layer named “YOLO” in my customized model.

4

Thanks for the help, I solved the problem but now I’m facing a different problem, I’m not able to run MultiURI with custom yolo model. When I try this is the error that I get

Using winsys: x11 
Creating LL OSD context new
Deserialize yoloLayerV3 plugin: yolo_17
Deserialize yoloLayerV3 plugin: yolo_24
Deserialize yoloLayerV3 plugin: yolo_31
** ERROR: <main:651>: Failed to set pipeline to PAUSED
Quitting
ERROR from source: Resource not found.
Debug info: gstfilesrc.c(533): gst_file_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
No such file "/home/cec/Downloads/sample_video.mkv file:///home/cec/Downloads/sample_video.mkv file:///home/cec/Downloads/sample_video.mkv"
ERROR from source: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Debug info: gstbasesrc.c(3452): gst_base_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
Failed to start
ERROR from source: Resource not found.
Debug info: gstfilesrc.c(533): gst_file_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
No such file "/home/cec/Downloads/sample_video.mkv file:///home/cec/Downloads/sample_video.mkv file:///home/cec/Downloads/sample_video.mkv"
ERROR from source: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Debug info: gstbasesrc.c(3452): gst_base_src_start (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstFileSrc:source:
Failed to start
App run failed

deepstream_app_config_yolov3_tiny.txt

################################################################################
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file:///home/cec/Downloads/sample_video.mkv file:///home/cec/Downloads/sample_video.mkv file:///home/cec/Downloads/sample_video.mkv
num-sources=16
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=model_b1_fp16.engine
labelfile-path=classnames.txt
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3_tiny.txt

[tests]
file-loop=0
5

Hi,

Please noticed you will need to declare the MultiURI argument with this rule:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream_Development_Guide%2Fdeepstream_app_config.3.2.html%23wwpID0E0QB0HA

With MultiURI, the %d format specifier can also be used to specify multiple sources. The application iterates from 0 to num-sources 1 to generate the actual URIs.

uri=file:///home/ubuntu/source_%d.mp4

Thanks.

6

just solved it.
you are trying to use your customized weights right?

if this is the case, instead of changing the name of the weights file in the “config_infer_primary_yoloV3_tiny.txt” file, change the name of your yolo weight itself. so you should have a file named “yolov3-tiny.cfg” and another file named “yolov3-tiny.weights”. pay attention that these are not the files that are downloaded by running “prebuild.sh”, these are our own weights! we just changed their names!

1 Like
7

Thank you man.

8

Your config file must have yolov3-* or yolov2-* name.

9

Hi guys,
@AastaLLL @anon56193499
Does Yolov5 or Yolov4 work on Deepstream?

1 Like
10

I don’t think so DeepStream only supports till yolov3