Yolov3_tiny is used to track vehicle detection problems

374789127 · October 18, 2019, 10:44am

hi:
I would like to ask two questions. One is that the Primary_Detector_Nano model in deepstream is poorly identified at night. Is this just a model use case? Do not provide accurate vehicle tracking, we need to develop it ourselves? There is another problem, we use the yolov3_tiny model to track and detect the vehicle, the source stream is 1080p, I am modifying the streammux wight=1280 height=720 in the configuration file deepstream_app_config_yolov3_tiny.txt, when using deepstream-app, Error:

root@uu-desktop:/opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo# deepstream-app -c deepstream_app_config_yoloV3_tiny.txt 
** WARN: <parse_gie:940>: Unknown key 'raw_output_directory' for group [primary-gie]

Using winsys: x11 
Opening in BLOCKING MODE 
Creating LL OSD context new
Creating LL OSD context new
0:00:00.671765941  9241     0x10ac3520 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 8858734
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-leaky     3 x 416 x 416      16 x 416 x 416    496   
(2)   maxpool          16 x 416 x 416      16 x 208 x 208    496   
(3)   conv-bn-leaky    16 x 208 x 208      32 x 208 x 208    5232  
(4)   maxpool          32 x 208 x 208      32 x 104 x 104    5232  
(5)   conv-bn-leaky    32 x 104 x 104      64 x 104 x 104    23920 
(6)   maxpool          64 x 104 x 104      64 x  52 x  52    23920 
(7)   conv-bn-leaky    64 x  52 x  52     128 x  52 x  52    98160 
(8)   maxpool         128 x  52 x  52     128 x  26 x  26    98160 
(9)   conv-bn-leaky   128 x  26 x  26     256 x  26 x  26    394096
(10)  maxpool         256 x  26 x  26     256 x  13 x  13    394096
(11)  conv-bn-leaky   256 x  13 x  13     512 x  13 x  13    1575792
(12)  maxpool         512 x  13 x  13     512 x  13 x  13    1575792
(13)  conv-bn-leaky   512 x  13 x  13    1024 x  13 x  13    6298480
(14)  conv-bn-leaky  1024 x  13 x  13     256 x  13 x  13    6561648
(15)  conv-bn-leaky   256 x  13 x  13     512 x  13 x  13    7743344
(16)  conv-linear     512 x  13 x  13     255 x  13 x  13    7874159
(17)  yolo            255 x  13 x  13     255 x  13 x  13    7874159
(18)  route                  -            256 x  13 x  13    7874159
(19)  conv-bn-leaky   256 x  13 x  13     128 x  13 x  13    7907439
(20)  upsample        128 x  13 x  13     128 x  26 x  26        - 
(21)  route                  -            384 x  26 x  26    7907439
(22)  conv-bn-leaky   384 x  26 x  26     256 x  26 x  26    8793199
(23)  conv-linear     256 x  26 x  26     255 x  26 x  26    8858734
(24)  yolo            255 x  26 x  26     255 x  26 x  26    8858734
Output blob names :
yolo_17
yolo_24
Total number of layers: 50
Total number of layers on DLA: 0
Building the TensorRT Engine...
Building complete!
0:00:32.850000647  9241     0x10ac3520 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo/model_b1_fp32.engine
Deserialize yoloLayerV3 plugin: yolo_17
Deserialize yoloLayerV3 plugin: yolo_24

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume


**PERF: FPS 0 (Avg)	FPS 1 (Avg)	
**PERF: 0.00 (0.00)	0.00 (0.00)	
** INFO: <bus_callback:163>: Pipeline ready

Opening in BLOCKING MODE 
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 
** INFO: <bus_callback:149>: Pipeline running

Creating LL OSD context new
Creating LL OSD context new
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
H264: Profile = 66, Level = 0 
ERROR from sink_sub_bin_mux1: Could not multiplex stream.
Debug info: gstqtmux.c(4561): gst_qt_mux_add_buffer (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/GstQTMux:sink_sub_bin_mux1:
Buffer has no PTS.
Quitting
App run failed

配置文件:

################################################################################
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=1
columns=2
width=1920
height=1080
gpu-id=1
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://../../samples/streams/test.mp4
num-sources=1
gpu-id=1
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source1]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://../../samples/streams/test.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source2]
enable=0
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://../../samples/streams/test.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
#uri=file://../../samples/streams/test.mp4
uri=file://20191010111000.mp4

num-sources=1
gpu-id=1
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=3
container=1
codec=1
sync=0
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1
output-file=out.mp4

#[sink0]
#enable=0
##Type - 1=FakeSink 2=EglSink 3=File
#type=3
#sync=0
#source-id=1
#gpu-id=1
#nvbuf-memory-type=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=1
gpu-id=1
nvbuf-memory-type=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=2
gpu-id=0
nvbuf-memory-type=0

[sink3]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=3
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1

##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_fp32.engine
labelfile-path=labels.txt
batch-size=1
#Required by the app for OSD, not a plugin property
#bbox-border-color0=1;0;0;1
#bbox-border-color1=0;1;1;1
#bbox-border-color2=0;0;1;1
#bbox-border-color3=0;1;0;1
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3_tiny.txt
raw_output_directory=1234
[tests]
file-loop=0

AastaLLL · October 21, 2019, 6:30am

Hi,

1. To improve the detection quality at night, you can re-train the model with more low light database.
We also have a toolkit to help user refine the model:
https://developer.nvidia.com/transfer-learning-toolkit

2. I can run YOLO sample with 1280x720 streammux output.
Would you mind to give the following config a try?

################################################################################
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file://../../samples/streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=model_b1_fp32.engine
labelfile-path=labels.txt
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
interval=2
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3_tiny.txt

[tests]
file-loop=0

Thanks.

374789127 · October 23, 2019, 6:33am

hi AastaLLL：
Thank you for your reply. I want to draw the detected area in the video stream for vehicle detection. Is there a corresponding example to provide it, or provide a solution? Thank you.

AastaLLL · November 5, 2019, 8:44am

Hi,

Deepstream only draw the bounding box boundary.
Are you looking for rendering approach with overlay like this:
[url]https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md[/url]

Thanks.

374789127 · December 26, 2019, 6:15am

hi AastaLLL：
Hello, thank you for your reply. We now have the darknet algorithm. The caffee algorithm provided in deepstream is not standard. How to convert the caffee algorithm into an algorithm that can be applied in deepstream after trt acceleration.

AastaLLL · January 6, 2020, 8:33am

Hi,

In Deepstream, we have a sample to convert darknet model into TensorRT:
/opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_Yolo

You can start from this sample directly.
Thanks.

374789127 · January 6, 2020, 8:42am

hi AastaLLL：
I have seen this Darknet model. Is there a similar Darknet model for license plate detection or body color detection or vehicle type detection? I am researching using yolo interface to implement on test2.

AastaLLL · January 7, 2020, 2:16am

Hi,

We don’t have pre-trained darknet model of your requirement.
Our darknet example use the default YOLOv2(tiny), YOLOv3(tiny) released by the author.

For the license plate detection, you can give OpenALPR a try.

Although the GitHub is a little bit out-of-date(DeepStream1.5), you can still start from it.

We don’t have color detector and type detector.
But there are some color/type classifier defined in the Caffe format inside the Deepstream.
It’s used with the a car detector beforehand.
/opt/nvidia/deepstream/deepstream-4.0/samples/models/

Thanks

Topic		Replies	Views
Deepstrem 5.0 Python yolo DeepStream SDK	16	2713	October 12, 2021
Apply darknet in deepstream cuDNN	1	435	November 30, 2019
YoloV3 Deepstream SDK 4.0 performance on Jetson Nano DeepStream SDK	14	4468	October 12, 2021
Accuracy decreases when deploying darknet on jetson tx2 Jetson TX2	16	1296	October 18, 2021
Yolo for Jetson DeepStream SDK	41	6429	August 13, 2024
The performance on yolov3 in the test3 doesn't match the performance that using deepstream-app DeepStream SDK	9	588	May 6, 2020
DeepstreamSDK 4.0.1 Custom Yolov3-Tiny Error DeepStream SDK	10	1498	April 27, 2020
Yolo V3 slow DeepStream SDK yolo	23	2750	January 18, 2022
run yolov3-tiny with tensorRT model Jetson Nano	7	3455	January 4, 2020
Problem facing with Custom tiny yolov3 model on Deepstream DeepStream SDK	10	721	October 2, 2021

Yolov3_tiny is used to track vehicle detection problems

Related topics