Request New features in deepstream sdk

Hi Dears,
1- I want to know Is it possible to add deep learning base tracker to deepstream in future?
2- If possible add motion detection element in your gstreamer pipeline. i.e If the current frame has above that motion rate threshold then pass the frame into other elements of pipeline and otherwise the frame directly passed into rendering element without processing elements.

1- I want to know Is it possible to add deep learning base tracker to deepstream in future?

Could you give a sample about “deep learning base tracker”?

2- If possible add motion detection element in your gstreamer pipeline. i.e If the current frame has above that motion rate threshold then pass the frame into other elements of pipeline and otherwise the frame directly passed into rendering element without processing elements.

Is below implmentation the similar as your requirement?

https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.html#wwpID0E0HDB0HA

When the plugin is operating as a secondary classifier along with the tracker, it tries to improve performance by avoiding re-inferencing on the same objects in every frame. It does this by caching the classification output in a map with the object’s unique ID as the key. The object is inferred upon only when it is first seen in a frame (based on its object ID) or when the size (bounding box area) of the object increases by 20% or more. This optimization is possible only when the tracker is added as an upstream element.

Could you give a sample about “deep learning base tracker”?

GOTURN

Have you tried GOTURN on TRT? How about the perf?

Thanks!

I didn’t test GOTURN on jetson, but It’s a deep learning base model and has high FPS on GPU about 100 FPS, In my opinion, It’s better to add this model or other deep learning base model to trackers in deepstream SDK.

Hi @LoveNvidia, what GPU did you run?

Hi @mchi,
GTX 1080.

Hi @LoveNvidia,
Thanks for sharing!
Could you help me understand the valuable use cases with GOTURN?

Hi , @mchi,
In the NvDCF tracker, it is used HOG for feature extraction ,and the GOTURN is used deep learning model for feature extraction, and this cause high accuracy.
I suggested this model as deep learning base tracker model, and your teams is better to add a good deep learning tracker real-time model in deep stream like GOTURN, and it’s supported by opencv+GPU.

Got, thanks a lot!

Discussed internally, we will consider to add this into future DS.

1 Like

@mchi, @kayccc, Hi,
It’s better to replace HOG features with lite CNN to achieve high precision. In really, deepsort is used CNN as feature for object of tracking.
What’s the high precision accuracy features in the NvDCF config? What’s name of this feature?