Multiple video inference with deepstream-yolo-app

oonokzy · July 5, 2019, 4:20am

The following environment could be built to infer sample images.
However, I do not know how to perform inference on multiple movie files or camera images.
Which configuration file should I change?

#Built environment
https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/yolo

#command
</deepstream_reference_apps-master/yolo/apps/deepstream-yolo# deepstream-yolo-app Tesla /root/DeepStream_Release/samples/streams/sample_720p.h264 /root/deepstream_reference_apps-master/yolo/config/yolov2.txt>

CJR · July 8, 2019, 8:55pm

Hi,

deepstream-yolo-app is a sample which performs inference on a single stream. To perform with inference with multiple streams, you can

Use sample
https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/yolo/samples/objectDetector_YoloV3

which performs yolo v3 inference using deepstream-app.

2)Wait for DS 4.0, which will be released soon and will also have support to run all yolo models natively with deepstream-app

Add multistream support to the deepstream-yolo app. You can refer to test3 sample in the SDK which performs something similar using nvinfer plugin.

oonokzy · July 22, 2019, 12:53am

I will check it.
Thank you very much.
I will ask you questions if there is anything else.

ChrisDing · August 1, 2019, 5:23am

Deepstream 4.0 has been released. You can refer to new yoloV3 sample in 4.0 package.

oonokzy · August 5, 2019, 1:14am

Thank you for contacting me.
confirm.

ibondokji · August 18, 2019, 12:36pm

hi @ChrisDing,

i wish to test deepstream with yolo, but i dont find any yolo samples in the deepstream samples file.

i found this documentation : https://docs.nvidia.com/metropolis/deepstream/4.0/Custom_YOLO_Model_in_the_DeepStream_YOLO_App.pdf

but it is not clear for me what i should do to run my own models.

this my first time working with deepstream, would you care to give some guidance?

i would also like to ask if there is python support for deepstream?

thanks

ChrisDing · August 20, 2019, 2:05am

we will release 5.0 with python binding support.

cshah · August 20, 2019, 6:32am

Hi ibondokji,

In addition to the Yolo documentation, please refer to ‘sources/objectDetector_Yolo’ to run Yolo models with DeepStream 4.0.

ibondokji · August 20, 2019, 7:06am

Hi Cshah,

i have figured that out, thanks.
could you help me with a different issue? for some reason running the the mp4 video in your samples works, but when i run my own test video with extension mkv i get a black display. terminal results:

**PERF: FPS 0 (Avg)	
**PERF: 0.00 (0.00)	
** INFO: <bus_callback:163>: Pipeline ready

Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
**PERF: 0.00 (0.00)	
**PERF: 0.00 (0.00)	
**PERF: 0.00 (0.00)	
**PERF: 0.00 (0.00)	
**PERF: 0.00 (0.00)	
**PERF: 0.00 (0.00)	
**PERF: 0.00 (0.00)

CJR · August 20, 2019, 6:36pm

Please create a new thread with your setup information/logs/questions.

ralvarezl8d7l · August 27, 2019, 3:03am

This is great news! Do you have an update on when this would be coming out?

barzanhayati · October 22, 2019, 5:16am

Dear Chris,
I have just done what you want to do.

if you have installed deepstream4, go to the “deepstream_app_config_yoloV3_tiny.txt” file and put the following code in there. replace your own video files in the following code. if it runs correctly on your platform, it should open up a windows with 9 videos, upon each inference with yolo tinyv3 is performed.

################################################################################

Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a

copy of this software and associated documentation files (the “Software”),

to deal in the Software without restriction, including without limitation

the rights to use, copy, modify, merge, publish, distribute, sublicense,

and/or sell copies of the Software, and to permit persons to whom the

Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER

DEALINGS IN THE SOFTWARE.

################################################################################

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=3
columns=3
width=1280
height=720

#width=640
#height=480

gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/1_2016-10-08_14-16-56.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=1
gpu-id=0
nvbuf-memory-type=0

[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/1_2016-10-08_14-20-50.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=2
gpu-id=0
nvbuf-memory-type=0

[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/1_2016-10-08_14-26-23.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink3]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=3
gpu-id=0
nvbuf-memory-type=0

[source4]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/2_2016-10-08_14-32-25.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink4]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=4
gpu-id=0
nvbuf-memory-type=0

[source5]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/2_2016-10-08_15-47-14.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink5]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=5
gpu-id=0
nvbuf-memory-type=0

[source6]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/2_2016-10-10_09-00-13.avi
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink6]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=6
gpu-id=0
nvbuf-memory-type=0

[source7]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/Li165CDN.mp4
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink7]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=7
gpu-id=0
nvbuf-memory-type=0

[source8]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3

#uri=file://…/…/samples/streams/1_2016-10-08_14-15-58.avi

uri=file://…/…/samples/streams/Li165CDNNight.mp4
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink8]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=8
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1280
height=720
#enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_fp32.engine
labelfile-path=labels.txt
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3_tiny.txt

[tests]
file-loop=0

barzanhayati · October 22, 2019, 5:19am

this is how your output will look like.

https://drive.google.com/open?id=173_wOTbbtWNq2LJJA125QuwCDCCxENik

afiqlcmec · April 25, 2020, 6:27pm

hi. any update on this particular release?

Topic		Replies	Views
Multiple model pipeline DeepStream SDK	15	1823	October 12, 2021
deepstream-plugins for real time video on Xavier DeepStream SDK	10	1382	October 12, 2021
Yolo for deepstream-app DeepStream SDK	27	9181	October 12, 2021
Unable to start Yolo8 deepstream with MJpeg AVI DeepStream SDK jetson-inference , gstreamer	14	891	September 8, 2023
Run yolov9e with deepstream python bindings DeepStream SDK boot , gstreamer , deepstream	14	109	July 14, 2025
Get wrong infer results while testing yolov4 on deepstream 5.0 DeepStream SDK	46	9479	October 12, 2021
Use camera in deepstream sample yolo app DeepStream SDK	2	403	October 12, 2021
cascade yolo models DeepStream SDK	11	1237	April 27, 2020
Deep Stream SDK DeepStream SDK gstreamer	4	537	October 12, 2021
DeepStream 4.0 + Custom Yolo Model - App run failed DeepStream SDK	10	4242	October 12, 2021

Multiple video inference with deepstream-yolo-app

Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a

copy of this software and associated documentation files (the “Software”),

to deal in the Software without restriction, including without limitation

the rights to use, copy, modify, merge, publish, distribute, sublicense,

and/or sell copies of the Software, and to permit persons to whom the

Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER

DEALINGS IN THE SOFTWARE.

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

Set muxer output width and height

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

Related topics