Add tracker abilities to TinyYOLOv3 sample

zvikas · July 19, 2020, 8:27am

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Nano Jetson
• DeepStream Version
4.02
• JetPack Version (valid for Jetson only)
4.4
• TensorRT Version
7.1
• NVIDIA GPU Driver Version (valid for GPU only)

Hi,

I am running deepstream with TinyYolo sample and its running well

~/Downloads/deepstream_sdk_v4.0.2_jetson/sources/objectDetector_Yolo$ deepstream-app -c deepstream_app_config_yoloV3_tiny.txt

Now i want to add tracker abilities but I dont see it in my config file :deepstream_app_config_yoloV3_tiny.txt

My Questions:

Is there tracker abilities by default.
If not , how can i add it ?
How can i render tracker_id?

10x in advance,

Zvika

mchi · July 20, 2020, 1:39am

Hi @zvikas
Please move to DS5.0. In DS5.0, tracker is enabled with below tracker section in deepstream_app_config_yoloV3_tiny.txt configure file.

[tracker]
enable=1
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

Amycao · July 20, 2020, 2:38am

Seen inline.

zvikas · July 20, 2020, 8:05am

Hi @mchi,

I have tried to migrate my running TinyYoloV3 on DS4.02 to DS5.0 but i failed with the error:

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

Opening in BLOCKING MODE

ERROR: [TRT]: coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)

ERROR: [TRT]: INVALID_STATE: std::exception

ERROR: [TRT]: INVALID_CONFIG: Deserialize the cuda engine failed.

ERROR: Deserialize engine failed from file: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/model_b1_fp16.engine

0:00:02.617515063 1711 0x3e18c660 WARN nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1566> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/model_b1_fp16.engine failed

0:00:02.617626890 1711 0x3e18c660 WARN nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1673> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/model_b1_fp16.engine failed, try rebuild

0:00:02.617668402 1711 0x3e18c660 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files

ERROR: failed to build network since there is no model file matched.

ERROR: failed to build network.

0:00:02.617917472 1711 0x3e18c660 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1611> [UID = 1]: build engine file failed

0:00:02.617957682 1711 0x3e18c660 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1697> [UID = 1]: build backend context failed

0:00:02.617991797 1711 0x3e18c660 ERROR nvinfer gstnvinfer.cpp:596:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1024> [UID = 1]: generate backend failed, check config file settings

0:00:02.618547232 1711 0x3e18c660 WARN nvinfer gstnvinfer.cpp:781:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance

0:00:02.618592077 1711 0x3e18c660 WARN nvinfer gstnvinfer.cpp:781:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/config_infer_primary_yoloV3_tiny_5.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

** ERROR: main:651: Failed to set pipeline to PAUSED

Quitting

ERROR from primary_gie: Failed to create NvDsInferContext instance

Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(781): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:

Config file path: /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_Yolo/config_infer_primary_yoloV3_tiny_5.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED

App run failed

What can be done ?

I have done no changes except of : export CUDA_VER=10.2 & make -C nvdsinfer_custom_impl_Yolo

mchi · July 20, 2020, 10:54am

could you share mroe details about this? Did you use any file from DS4.0.2 in DS5.0?
Can you run the YoloV3 in DS5.0?

zvikas · July 20, 2020, 12:50pm

Hi ,

I have used 5 files that i have used in DS 4.02 in DS5.0 :

deepstream_app_config_yoloV3_tiny

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
#uri=file://…/…/samples/streams/sample_1080p_h264.mp4
uri=file:out.mp4
num-sources=1
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0
camera-width=1920
camera-height=1080
camera-fps-n=30
camera-v4l2-dev-node=0
camera-fps-d=1
#num-extra-surfaces=10

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=mp4 2=mkv
container=2
#1=h264 2=h265 3=mpeg4
gpu-id=0
nvbuf-memory-type=0
#only mpeg4 is supported right now.
codec=1
sync=0
bitrate=4000
#set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

config_infer_primary_yoloV3_tiny.txt:

[property]
gpu-id=0
net-scale-factor=1
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=yolov3-tiny-obj-custom.cfg

#model-file=yolov3-tiny-obj_98000.weights -with out calc anchors
model-file=yolov3-tiny-obj_83000.weights
model-engine-file=model_b1_fp16.engine
labelfile-path=labels_5.txt

0=FP32, 1=INT8, 2=FP16 mode

network-mode=2
num-detected-classes=5
gie-unique-id=1
is-classifier=0
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3Tiny
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so

labels.txt :

5 classes …

yolov3-tiny-obj-custom.cfg :

[net]

Testing

batch=1
subdivisions=1

Training

#batch=64
#subdivisions=8
width=416
#width=800
height=416
#height=800
channels=3
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1

learning_rate=0.001
burn_in=1000
max_batches = 500200
policy=steps
steps=400000,450000
scales=.1,.1

[convolutional]
batch_normalize=1
filters=16
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=2

[convolutional]
batch_normalize=1
filters=32
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=2

[convolutional]
batch_normalize=1
filters=64
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=2

[convolutional]
batch_normalize=1
filters=128
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=2

[convolutional]
batch_normalize=1
filters=256
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=2

[convolutional]
batch_normalize=1
filters=512
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=1

[convolutional]
batch_normalize=1
filters=1024
size=3
stride=1
pad=1
activation=leaky

###########

[convolutional]
batch_normalize=1
filters=256
size=1
stride=1
pad=1
activation=leaky

[convolutional]
batch_normalize=1
filters=512
size=3
stride=1
pad=1
activation=leaky

[convolutional]
size=1
stride=1
pad=1
filters=30
activation=linear

[yolo]
mask = 3,4,5
#anchors = 20, 85, 41, 43, 27, 68, 46, 75, 47,108, 70,116
anchors= 45,75,47,141,105,63,38,226,79,123,101,202
#anchors= 45,75,47,141,105,63,38,226,79,123,101,202
classes=5
num=6
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1

[route]
layers = -4

[convolutional]
batch_normalize=1
filters=128
size=1
stride=1
pad=1
activation=leaky

[upsample]
stride=2

[route]
layers = -1, 8

[convolutional]
batch_normalize=1
filters=256
size=3
stride=1
pad=1
activation=leaky

[convolutional]
size=1
stride=1
pad=1
filters=30
activation=linear

[yolo]
mask = 0,1,2
#anchors = 20, 85, 41, 43, 27, 68, 46, 75, 47,108, 70,116
anchors= 45,75,47,141,105,63,38,226,79,123,101,202
classes=5
num=6
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1

yolov3-tiny-obj_105000.weights ( custom tinyYoloV3 weights file)

zvikas · July 20, 2020, 1:25pm

Hi ,

Running the default (Runing from sample file succeeded:

deepstream-app -c deepstream_app_config_yoloV3_tiny.txt

But running from camera failed :

(deepstream-app:30073): GStreamer-CRITICAL **: 09:14:37.111: passed ‘0’ as denominator for `GstFraction’

** ERROR: <create_camera_source_bin:153>: Failed to link ‘src_elem’ (image/jpeg; video/mpeg, mpegversion=(int)4, systemstream=(boolean)false; video/mpeg, mpegversion=(int)2; video/mpegts, systemstream=(boolean)true; video/x-bayer, format=(string){ bggr, gbrg, grbg, rggb }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-dv, systemstream=(boolean)true; video/x-h263, variant=(string)itu; video/x-h264, stream-format=(string){ byte-stream, avc }, alignment=(string)au; video/x-pwc1, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-pwc2, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-sonix, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-vp8; video/x-vp9; video/x-wmv, wmvversion=(int)3, format=(string)WVC1) and ‘src_cap_filter1’ (video/x-raw, width=(int)0, height=(int)0)

** ERROR: <create_camera_source_bin:204>: create_camera_source_bin failed

** ERROR: <create_pipeline:1263>: create_pipeline failed

** ERROR: main:632: Failed to create pipeline

Quitting

App run failed

mchi · July 20, 2020, 2:24pm

Hi @zvikas

can you modified the deepstream_app_config_yoloV3_tiny.txt and config_infer_primary_yoloV3_tiny.txt based on DS5.0 config files ? You can modify the config file steps by step to narrow down which change cause the issue.
Regarding the camera issue, is it the same as previous failure? If it’s not, please file a new ticket to avoid confusion.

Thanks!