How to fetch NvDsTracker shadow tracking data using Python

abdurrehman.ali · July 2, 2024, 1:46pm

Please provide complete information as applicable to your setup.

• DeepStream 7.0

Hi, I am trying to use shadow tracking data generated by the NvTracker plugin from pyds.

In c++, I could do this by retrieving the NVDS_TRACKER_SHADOW_LIST_META type of user_meta.

However, it seems that NVDS_TRACKER_SHADOW_LIST_META enum is not available in the python bindings:

>>> import pyds
>>> dir(pyds.NvDsMetaType)
['NVDSINFER_SEGMENTATION_META', 'NVDSINFER_TENSOR_OUTPUT_META', 'NVDS_AUDIO_BATCH_META', 'NVDS_AUDIO_FRAME_META', 'NVDS_BATCH_META', 'NVDS_CLASSIFIER_META', 'NVDS_CROP_IMAGE_META', 'NVDS_DISPLAY_META', 'NVDS_EVENT_MSG_META', 'NVDS_FORCE32_META', 'NVDS_FRAME_META', 'NVDS_GST_CUSTOM_META', 'NVDS_INVALID_META', 'NVDS_LABEL_INFO_META', 'NVDS_LATENCY_MEASUREMENT_META', 'NVDS_OBJ_META', 'NVDS_OPTICAL_FLOW_META', 'NVDS_PAYLOAD_META', 'NVDS_RESERVED_META', 'NVDS_START_USER_META', 'NVDS_TRACKER_PAST_FRAME_META', 'NVDS_USER_META', '__class__', '__delattr__', '__dir__', '__doc__', '__entries', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__le__', '__lt__', '__members__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', 'name']

Is the functionality available in Python deepstream bindings to retrieve the shadow tracking data?

junshengy · July 3, 2024, 9:24am

You need to add the corresponding bindings. I modified deepstream_test_2.py. You can refer to it.

1.modify pyds

diff --git a/bindings/docstrings/nvdsmetadoc.h b/bindings/docstrings/nvdsmetadoc.h
index bf222a2..7d1c092 100644
--- a/bindings/docstrings/nvdsmetadoc.h
+++ b/bindings/docstrings/nvdsmetadoc.h
@@ -44,6 +44,7 @@ namespace pydsdoc
             constexpr const char* NVDSINFER_SEGMENTATION_META=R"pyds(metadata type of segmentation model output attached by gst-nvinfer. Refer :class:`NvDsInferSegmentationMeta` for details.)pyds";
             constexpr const char* NVDS_CROP_IMAGE_META=R"pyds(Specifies metadata type for JPEG-encoded object crops.See the deepstream-image-meta-test app for details.)pyds";
             constexpr const char* NVDS_TRACKER_PAST_FRAME_META=R"pyds(metadata type to be set for tracking previous frames)pyds";
+            constexpr const char* NVDS_TRACKER_SHADOW_LIST_META=R"pyds(metadata type for Shadow Tracking mode)pyds";
             constexpr const char* NVDS_AUDIO_BATCH_META=R"pyds(Specifies metadata type for formed audio batch.)pyds";
             constexpr const char* NVDS_AUDIO_FRAME_META=R"pyds(Specifies metadata type for audio frame.)pyds";
             constexpr const char* NVDS_RESERVED_META=R"pyds(Reserved field)pyds";
diff --git a/bindings/docstrings/trackermetadoc.h b/bindings/docstrings/trackermetadoc.h
index 4a35840..359c1b2 100644
--- a/bindings/docstrings/trackermetadoc.h
+++ b/bindings/docstrings/trackermetadoc.h
@@ -82,6 +82,15 @@ namespace pydsdoc
             constexpr const char* cast=R"pyds(cast given object/data to :class:`NvDsTargetMiscDataFrame`, call pyds.NvDsTargetMiscDataFrame.cast(data))pyds";
         }
 
+        namespace TRACKER_STATEDoc {
+            constexpr const char* descr = R"pyds(*Enumerator*, Specifies the data type of TRACKER_STATE.)pyds";
+            constexpr const char* EMPTY = R"pyds(EMPTY)pyds";
+            constexpr const char* ACTIVE = R"pyds(ACTIVE)pyds";
+            constexpr const char* INACTIVE = R"pyds(INACTIVE)pyds";
+            constexpr const char* TENTATIVE = R"pyds(TENTATIVE)pyds";
+            constexpr const char* PROJECTED = R"pyds(PROJECTED)pyds";
+        }
+
         namespace NvDsTargetMiscDataObjectDoc
         {
             constexpr const char* descr = R"pyds(
diff --git a/bindings/src/bindnvdsmeta.cpp b/bindings/src/bindnvdsmeta.cpp
index 95b0585..0793d4e 100644
--- a/bindings/src/bindnvdsmeta.cpp
+++ b/bindings/src/bindnvdsmeta.cpp
@@ -65,6 +65,8 @@ namespace pydeepstream {
                 .value("NVDS_TRACKER_PAST_FRAME_META",
                        NVDS_TRACKER_PAST_FRAME_META,
                        pydsdoc::nvmeta::MetaTypeDoc::NVDS_TRACKER_PAST_FRAME_META)
+                .value("NVDS_TRACKER_SHADOW_LIST_META",
+                       NVDS_TRACKER_SHADOW_LIST_META, pydsdoc::nvmeta::MetaTypeDoc::NVDS_TRACKER_SHADOW_LIST_META)
                 .value("NVDS_AUDIO_BATCH_META", NVDS_AUDIO_BATCH_META,
                        pydsdoc::nvmeta::MetaTypeDoc::NVDS_AUDIO_BATCH_META)
                 .value("NVDS_AUDIO_FRAME_META", NVDS_AUDIO_FRAME_META,
diff --git a/bindings/src/bindtrackermeta.cpp b/bindings/src/bindtrackermeta.cpp
index 672e7bb..01d6b7c 100644
--- a/bindings/src/bindtrackermeta.cpp
+++ b/bindings/src/bindtrackermeta.cpp
@@ -25,6 +25,20 @@ namespace py = pybind11;
 namespace pydeepstream {
 
     void bindtrackermeta(py::module &m) {
+        py::enum_<TRACKER_STATE>(m, "TRACKER_STATE",
+                                     pydsdoc::trackerdoc::TRACKER_STATEDoc::descr)
+                .value("EMPTY", EMPTY,
+                       pydsdoc::trackerdoc::TRACKER_STATEDoc::EMPTY)
+                .value("ACTIVE", ACTIVE,
+                       pydsdoc::trackerdoc::TRACKER_STATEDoc::ACTIVE)
+                .value("INACTIVE", INACTIVE,
+                       pydsdoc::trackerdoc::TRACKER_STATEDoc::INACTIVE)
+                .value("TENTATIVE", TENTATIVE,
+                       pydsdoc::trackerdoc::TRACKER_STATEDoc::TENTATIVE)
+                .value("PROJECTED", PROJECTED,
+                       pydsdoc::trackerdoc::TRACKER_STATEDoc::PROJECTED)
+                .export_values();
+
         /*Start of Bindings for nvds_tracker_meta.h*/
         py::class_<NvDsTargetMiscDataFrame>(m, "NvDsTargetMiscDataFrame",
                                      pydsdoc::trackerdoc::NvDsTargetMiscDataFrameDoc::descr)
@@ -33,6 +47,8 @@ namespace pydeepstream {
                 .def_readwrite("tBbox", &NvDsTargetMiscDataFrame::tBbox)
                 .def_readwrite("confidence", &NvDsTargetMiscDataFrame::confidence)
                 .def_readwrite("age", &NvDsTargetMiscDataFrame::age)
+                .def_readwrite("trackerState", &NvDsTargetMiscDataFrame::trackerState)
+                .def_readwrite("visibility", &NvDsTargetMiscDataFrame::visibility)
 
                 .def("cast",
                      [](void *data) {

I tested it on jetson, the dGPU is the same

cd /opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/bindings/build

make

pip uninstall pyds

pip3 install ./pyds-1.1.11-py3-none-linux_aarch64.whl

2.This is patch of deepstream_test_2.py and config_tracker_NvDCF_perf.yml

diff --git a/apps/deepstream-test2/config_tracker_NvDCF_perf.yml b/apps/deepstream-test2/config_tracker_NvDCF_perf.yml
index 394f04d..18ec39d 100644
--- a/apps/deepstream-test2/config_tracker_NvDCF_perf.yml
+++ b/apps/deepstream-test2/config_tracker_NvDCF_perf.yml
@@ -29,6 +29,7 @@ TargetManagement:
   probationAge: 3 # If the target's age exceeds this, the target will be considered to be valid.
   maxShadowTrackingAge: 30  # Max length of shadow tracking. If the shadowTrackingAge exceeds this limit, the tracker will be terminated.
   earlyTerminationAge: 1   # If the shadowTrackingAge reaches this threshold while in TENTATIVE period, the target will be terminated prematurely.
+  outputShadowTracks: 1
 
 TrajectoryManagement:
   useUniqueID: 0   # Use 64-bit long Unique ID when assignining tracker ID. Default is [true]
diff --git a/apps/deepstream-test2/deepstream_test_2.py b/apps/deepstream-test2/deepstream_test_2.py
index f787a7f..00d6d96 100755
--- a/apps/deepstream-test2/deepstream_test_2.py
+++ b/apps/deepstream-test2/deepstream_test_2.py
@@ -128,7 +128,7 @@ def osd_sink_pad_buffer_probe(pad,info,u_data):
             user_meta=pyds.NvDsUserMeta.cast(l_user.data)
         except StopIteration:
             break
-        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META):
+        if (user_meta and user_meta.base_meta.meta_type == pyds.NvDsMetaType.NVDS_TRACKER_SHADOW_LIST_META):
             try:
                 # Note that user_meta.user_meta_data needs a cast to pyds.NvDsTargetMiscDataBatch
                 # The casting is done by pyds.NvDsTargetMiscDataBatch.cast()
@@ -145,7 +145,6 @@ def osd_sink_pad_buffer_probe(pad,info,u_data):
                     print("numobj=",miscDataObj.numObj)
                     print("uniqueId=",miscDataObj.uniqueId)
                     print("classId=",miscDataObj.classId)
-                    print("objLabel=",miscDataObj.objLabel)
                     for miscDataFrame in pyds.NvDsTargetMiscDataObject.list(miscDataObj):
                         print('frameNum:', miscDataFrame.frameNum)
                         print('tBbox.left:', miscDataFrame.tBbox.left)
@@ -153,7 +152,8 @@ def osd_sink_pad_buffer_probe(pad,info,u_data):
                         print('tBbox.top:', miscDataFrame.tBbox.top)
                         print('tBbox.right:', miscDataFrame.tBbox.height)
                         print('confidence:', miscDataFrame.confidence)
-                        print('age:', miscDataFrame.age)
+                        print('trackerState:', miscDataFrame.trackerState)
+                        print('visibility:', miscDataFrame.visibility)
         try:
             l_user=l_user.next
         except StopIteration:

Run the sample.

python3 deepstream_test_2.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264

Similar results will be output to the terminal

numobj= 1
uniqueId= 20
classId= 0
frameNum: 30
tBbox.left: 806.1800537109375
tBbox.width: 27.789276123046875
tBbox.top: 462.8977355957031
tBbox.right: 17.30149269104004
confidence: 0.5567905902862549
trackerState: TRACKER_STATE.INACTIVE
visibility: 1.0

system · July 17, 2024, 9:24am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Unable to fetch Tracker User Meta DeepStream SDK	3	228	February 7, 2024
Test the tracker alone DeepStream SDK gstreamer	11	793	November 25, 2022
Rtsp inference and tracker DeepStream SDK ubuntu , gstreamer , python	13	250	June 29, 2024
Using nvmsgconv and RTSP output on the same DEEPSTREAM pipeline DeepStream SDK rtsp	2	1002	October 12, 2021
NvdsTracker API sample DeepStream SDK	13	2665	July 15, 2020
(Gst-nvtracker) Tracking of manually inserted objects DeepStream SDK	6	752	January 11, 2024
Deepstream 5.0 python bindings for Gst-nvdsanalytics access meta data DeepStream SDK	34	4177	October 12, 2021
RuntimeError: get_nvds_buf_Surface: Currently we only support RGBA color Format DeepStream SDK deepstream	13	455	July 2, 2024
NvDCF Jitter DeepStream SDK	22	2380	October 12, 2021
Obtain age of tracker in DeepStream DeepStream SDK	10	480	June 25, 2024

How to fetch NvDsTracker shadow tracking data using Python

Related topics