Find Embeddings from Infered Objects

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (GPU): Currently debugging in GTX 1080
• DeepStream Version : 7.0
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only): 535.183.01
• CUDA Version: 12.2
• Issue Type: bugs
• How to reproduce the issue ?

Run Deepstream_test_2.py, attaching one. The l_user_meta is an empty array. I was trying to find embeddings from the tracker initially but that also failed.
deepstream-test2.zip (15.6 KB)

This was added as tracker_sink_pad = tracker.get_static_pad("sink")

tracker_sink_pad.add_probe(Gst.PadProbeType.BUFFER, tracker_sink_pad_buffer_probe, 0)

def tracker_sink_pad_buffer_probe(pad,info,u_data):
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        sys.stderr.write("Unable to get GstBuffer")
        return Gst.PadProbeReturn.OK
    #sys.stdout.write("Found GstBuffer in Tracker Sink\n")
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        #sys.stdout.write("Found l_frame in Tracker Sink\n")
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            l_obj = frame_meta.obj_meta_list
            while l_obj is not None:
                #sys.stdout.write("Found l_obj in Tracker Sink\n")
                try:
                    obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
                    l_user_meta = obj_meta.obj_user_meta_list
                    print(f"l_user_meta: {[a for a in dir(l_user_meta) if not a.startswith('__')]}")
                    while l_user_meta is not None:
                        try:
                            user_meta = pyds.NvDsUserMeta.cast(l_user_meta.data)
                            if user_meta:
                                print(f"User meta type found {user_meta.base_meta.meta_type}\n")
                            l_user_meta = l_user_meta.next
                        except StopIteration:
                            break
                    l_obj = l_obj.next
                except StopIteration:
                    break
            l_frame = l_frame.next
        except StopIteration:
            break

    return Gst.PadProbeReturn.OK
3

tracker user meta is added in tracker plugin. please add probe function on tracker’s src or the plugins after tracker.

4

What do you mean by that? Is there another plugin which we need to build and add?
Also I do not see NVDS_TRACKER_OBJ_REID_META in pyds.NvDsMetaType. I built the bindings from the doc

def tracker_src_pad_buffer_probe(pad,info,u_data):
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        sys.stderr.write("Unable to get GstBuffer")
        return Gst.PadProbeReturn.OK
    #sys.stdout.write("Found GstBuffer in Tracker Sink\n")
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        #sys.stdout.write("Found l_frame in Tracker Sink\n")
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
            l_obj = frame_meta.obj_meta_list
            while l_obj is not None:
                #sys.stdout.write("Found l_obj in Tracker Sink\n")
                try:
                    obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
                    l_user_meta = obj_meta.obj_user_meta_list
                    print(f"l_user_meta: {[a for a in dir(l_user_meta) if not a.startswith('__')]}")
                    while l_user_meta is not None:
                        try:
                            user_meta = pyds.NvDsUserMeta.cast(l_user_meta.data)
                            if user_meta:
                                print(f"User meta type found {user_meta.base_meta.meta_type}\n")
                            l_user_meta = l_user_meta.next
                        except StopIteration:
                            break
                    l_obj = l_obj.next
                except StopIteration:
                    break
            l_frame = l_frame.next
        except StopIteration:
            break

    return Gst.PadProbeReturn.OK

Nothing gets printed in l_user_meta

5

I mean, the code should be corrected to tracker_sink_pad = tracker.get_static_pad(“src”) because the tracker 's meta will be added when exiting tracker plugin.

6

Yes, The change to "src" works.
But the nvDsMetaType is not known in python bindings.
For Deepstream7.0, Do I need to change the 3rd party git commit id?

Output is:

l_user_meta: ['data', 'next', 'prev']
User meta type found NvDsMetaType.???
7

python binding is opensource. you can add NVDS_TRACKER_OBJ_REID_META after referring to NVDS_TRACKER_PAST_FRAME_META adding code.

1 Like
8

Hi @fanzh ,
Could you please review the PR: Add pyBindings for NVDS_TRACKER_OBJ_REID_META by amarflybot · Pull Request #12 · NVIDIA-AI-IOT/deepstream_python_apps · GitHub

1 Like
9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

10

@amarendra.k.kumar
Thanks for your contribution, this PR has been merged to internal branch and will be publiched at next version.