Tracking individual objects with Deepstream Python

Hi everyone,

I am using a modified version of the first Deepstream Python example to detect the objects from Logitech C920 feed. Everything looks fine so far, I am turning the coordinates to OSC messages to communicate with multimedia softwares; but there is one thing I could not find a solution.

Each time in loop, I get some coordinates, vehicle and person count; but I can’t get a unique ID for each object detected.

When I use obj_meta.object_id in my code, it returns the same number for each object type. Is there a way to parse the data to person1, person2, person3, car1, car2, car3, car4 etc? Whatever I tried, I couldn’t do it.

Hope I could explain my problem clearly. Thank you for advises in advance.

My understanding is that need to use an nvtracker element to do that, or write your own tracker, preferably using nvtracker (see “Custom Low-Level Library” at that link). Probably easiest to use the existing trackers nvtracker provides.

Thank you for your answer. I realized that I am still very ignorant to this world after checking the website you shared. I got more confused :)

I think I am using the nvtracker element without realizing; because in the code, the metadata info comes from NvDsBatchMeta. Is there a documentation of all methods or arguments of nvtracker in Python? Maybe I am missing something very easy. And writing my own tracker looks way above my knowledge.

Believe me, that’s normal. For me, it’s often easier to learn by examples. I don’t think the NvDs stuff has Python specific documentation yet, however, the basic structure is documented here (and somewhere else, but i forget the link).

Please see the test 2 app in the python examples for an example of how to use nvtracker.

Please note that the lists used in the NvDs… structs are not python lists, but rather GLib lists, which are doubly linked lists. You’ll see examples of how to iterate through them in callbacks like osd_sink_pad_buffer_probe, however you may find it easier to write your own iterator wrapper like below.

instead of this:

(from <b>test 1 </b>example)
################################################################################
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

def osd_sink_pad_buffer_probe(pad,info,u_data):
...
    while l_frame is not None:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.glist_get_nvds_frame_meta()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.glist_get_nvds_frame_meta(l_frame.data)
        except StopIteration:
            break

        frame_number=frame_meta.frame_num
        num_rects = frame_meta.num_obj_meta
        l_obj=frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                # Casting l_obj.data to pyds.NvDsObjectMeta
                obj_meta=pyds.glist_get_nvds_object_meta(l_obj.data)
            except StopIteration:
                break
            obj_counter[obj_meta.class_id] += 1
            try: 
                l_obj=l_obj.next
            except StopIteration:
                break
...

You can do this:

################################################################################
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

...

VEHICLE = 0
BICYCLE = 1
PERSON = 2
ROADSIGN = 3

# add these two generators here:
[b]def frame_meta_iterator(frame_meta_list: GLib.List
                        ) -> Iterator[pyds.NvDsFrameMeta]:
    while frame_meta_list is not None:
        yield pyds.glist_get_nvds_frame_meta(frame_meta_list.data)
        frame_meta_list = frame_meta_list.next

def obj_meta_iterator(obj_meta_list: GLib.List
                      ) -> Iterator[pyds.NvDsObjectMeta]:
    while obj_meta_list is not None:
        yield pyds.glist_get_nvds_object_meta(obj_meta_list.data)
        obj_meta_list = obj_meta_list.next[/b]

# and use them here like this to make the loops easier to read
def osd_sink_pad_buffer_probe(pad,info,u_data):
... (at the same place bolded as above,)
[b]    for frame_meta in frame_meta_iterator(batch_meta.frame_meta_list):
        for obj_meta in obj_meta_iterator(frame_meta.obj_meta_list):
            obj_counter[obj_meta.class_id] += 1[/b]

        # Acquiring a display meta object. The memory ownership remains in
        # the C code so downstream plugins can still access it. Otherwise
        # the garbage collector will claim it when this probe function exits.
        display_meta = pyds.nvds_acquire_display_meta_from_pool(batch_meta)
        display_meta.num_labels = 1
        py_nvosd_text_params = display_meta.text_params[0]

        # Setting display text to be shown on screen
        # Note that the pyds module allocates a buffer for the string, and the
        # memory will not be claimed by the garbage collector.
        # Reading the display_text field here will return the C address of the
        # allocated string. Use pyds.get_string() to get the string content.
        # (this is a setter, and reading from it will only return a pointer)
        py_nvosd_text_params.display_text = \
            f"Frame={frame_meta.frame_num} " \
            f"Objects={frame_meta.num_obj_meta} " \
            f"Vehicles={obj_counter[VEHICLE]} " \
            f"People={obj_counter[PERSON]}"

        # Now set the offsets where the string should appear
        py_nvosd_text_params.x_offset = 10
        py_nvosd_text_params.y_offset = 12

        # Font , font-color and font-size
        py_nvosd_text_params.font_params.font_name = "Serif"
        py_nvosd_text_params.font_params.font_size = 10
        # set(red, green, blue, alpha); set to White
        py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)

        # Text background color
        py_nvosd_text_params.set_bg_clr = 1
        # set(red, green, blue, alpha); set to Black
        py_nvosd_text_params.text_bg_clr.set(0.0, 0.0, 0.0, 1.0)
        # Using pyds.get_string() to get display_text as string
        pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)

    return Gst.PadProbeReturn.OK

For me, anyway, it’s easier to read that way and works just the same. There might be an even easier way to iterate through the GLib lists but I don’t think their python bindings support the iterator protocol directly so you have to write your own iterators for these lists to avoid having to. The typing hints are also completely optional, ignored at runtime, and only aid to help your IDE and readers of your code to know what objects you’re dealing with.