Many frames are undetectable for DS5.0 Yolo example with a usb camera

Hi,
When I was running the Obj_Yolo deepstream app with a usb cam, I found that more than half the frames have no results. It is not the case when running with recorded videos that were .h264 encoded.

Any advice why this is happening and how I can fix it?

My settings are the following:
Jetson Xavier
DeepStream 5.0
JetPack 4.4
TensorRT 7.1.3
NVIDIA GPU Driver Version 10.2

Hey customer,

  1. better to use the latest DS5.1
  2. For your question, did you mean there is no issue with a local h264 stream file however the issue occur when you used usb cam?

Yes exactly. Also the issue does not occur when testing with videos recorded with the same usb cam and OBS software in h264 format.

Hey, customer
What’s your pipeline?

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1092
height=614
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
#uri=file://…/…/samples/streams/sample_720p.jpg
uri=file://…/…/samples/streams/2020-11-21 12-55-30.mov
num-sources=1
gpu-id=0
cudadec-memtype=0

[source1]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
camera-width=1092
camera-height=614
camera-fps-n=30
camera-fps-d=1
camera-v4l2-dev-node=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=/home/nvidia/ssd/testout.mp4
source-id=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
batched-push-timeout=40000
width=1092
height=614
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=2
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tracker]
enable=0
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

[tests]
file-loop=0

Is this file from your camera, how do you generate the mov file?

I recorded it from the same usb cam with OBS software.

So the raw camera data had been encoded via OBS solftware, what’s the OBS software?

Open Broadcaster Software

Thanks, what’s the type of your camera, have you tried other camera devices?

I tried with a video recorded with my phone, and the problem still persists. Any idea why that is the case?

Hi,
I realize this issue might be hard to replicate for others even though it seems to be a constant phenomenon for me. So can you or your co-workers give me some advice on how I can locate the issue along the pipeline?

Right now I suspect the issue might be raised during image parsing before the actual inference even begin, but I don’t how to verify this speculation. Is there any tool or procedure that can help me with that? Or maybe you can point me to the right section of the source code to take a closer look?

Any advice is appreciated, thanks!

Yeah, agree, I would suggest you to try other cameras to see if issue persist and check if any clues for camera settings

If you want to dump the image files, you can refer DeepStream SDK FAQ - #17 by mchi

Thanks for your advice.
I found that this Deepstream example only perform inference once for every three frames, and the missing results were completed with the [tracker] module.
Can you explain the necessity for such design of detection pipeline?

Actually, you can change it by setting the interval property of nvinfer

That will increase performance, basically, we consider nvinfer will consume more HW resource than tracker.

Thank you very much! There are just so many parameters to get familiar with.