Rtsp inference and tracker

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (GPU) GPU A30
• DeepStream Version 6.4
• TensorRT Version 10.0.0.6
• NVIDIA GPU Driver Version (valid for GPU only) 535.104.12
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
python rtsp.py -i uri

Like i mentioned here im trying to add tracker to my already working pipeline but i get Core dumped error

2

How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

4

This sample shows how to add nvtracker in the pipeline. deepstream_python_apps/apps/deepstream-nvdsanalytics at master · NVIDIA-AI-IOT/deepstream_python_apps (github.com)

5

Thank you for the replay.
Just by running the python script i got the error

6

Yes, i have the tracker added to the pipeline and linked, it is successfully loaded but as soon as the window is created, it closes

streammux.link(pgie)
pgie.link(tracker)
tracker.link(nvvidconv)
nvvidconv.link(nvosd)
nvosd.link(sink)

The example that you have mentioned, use queues to link the streammux and the tracker.

I tried to replicate it but still not working

srcpad.link(sinkpad)
streammux.link(queue1)
queue1.link(pgie)
pgie.link(queue2)
queue2.link(tracker)
tracker.link(nvvidconv)
nvvidconv.link(nvosd)
nvosd.link(sink)

Here the bt -full

(gdb) bt -full
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140731775706688) at ./nptl/pthread_kill.c:44
        tid = <optimized out>
        ret = 0
        pd = 0x7ffeab7fe640
        old_mask = {__val = {140732124276616, 0, 0, 140734213153990, 140731113075440, 140731775700976, 140731113075440, 0, 140731775701136, 140734213186674, 140731113262048, 1150461660618886656,
            4294967315, 0, 140731775701136, 1150461660618886656}}
        ret = <optimized out>
        pd = <optimized out>
        old_mask = <optimized out>
        ret = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        __futex = <optimized out>
        resultvar = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        __futex = <optimized out>
        __private = <optimized out>
        __oldval = <optimized out>
        result = <optimized out>
#1  __pthread_kill_internal (signo=6, threadid=140731775706688) at ./nptl/pthread_kill.c:78
No locals.
#2  __GI___pthread_kill (threadid=140731775706688, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
No locals.
#3  0x00007ffff7c42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
        ret = <optimized out>
#4  0x00007ffff7c287f3 in __GI_abort () at ./stdlib/abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {0, 0, 0, 16, 72340172838076673, 72340172838076673, 770, 0, 0, 0, 0, 0, 0, 0, 0, 0}}, sa_flags = 0,
          sa_restorer = 0x0}
        sigs = {__val = {32, 140737321261197, 0, 1150461660618886656, 140731775700976, 140731775700016, 0, 140731113064992, 0, 0, 16, 72340172838076673, 72340172838076673, 770, 0, 0}}
#5  0x00007ffff56a2692 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#6  0x00007ffff56ad89f in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#7  0x00007ffff56ad9c7 in __gxx_personality_v0 () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#8  0x00007ffff60a5fe9 in __libunwind_Unwind_Resume () from /lib/x86_64-linux-gnu/libunwind.so.8
No symbol table info available.
#9  0x00007ffec046a94f in ?? () from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#10 0x00007ffec04fb98d in cuDCFFrameTransformTexture::initialize(int, int, int, int, NvMotDataTypes::Size, unsigned int, NvBufSurfaceColorFormat, NvMotDataTypes::Size, bool, float) ()
   from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
--Type <RET> for more, q to quit, c to continue without paging--
#11 0x00007ffec0503baf in FeatureExtractor::initialize(int, int, int, int, NvMotDataTypes::Size, unsigned int, NvBufSurfaceColorFormat, NvMotDataTypes::Size, int, unsigned int, VPIBackend, unsigned long) ()
   from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#12 0x00007ffec04ed6fe in VisualTracker::initialize(SourceFrameInfo const&, unsigned int, unsigned int, std::shared_ptr<NvTrackedObjectManager> const&) ()
   from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#13 0x00007ffec047510d in NvMultiObjectTrackerBase::initializeSubModules() () from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#14 0x00007ffec04781a0 in NvMultiObjectTrackerBase::update(std::map<unsigned long, _NvMOTFrame*, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, _NvMOTFrame*> > > const&, _NvMOTTrackedObjBatch*&) () from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#15 0x00007ffec049b204 in NvMOTContext::processFrame(_NvMOTProcessParams const*, _NvMOTTrackedObjBatch*) () from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#16 0x00007ffec049bb26 in NvMOT_Process () from /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
No symbol table info available.
#17 0x00007fff49fab0ca in NvTrackerProc::processBatch() () from /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_tracker.so
No symbol table info available.
#18 0x00007ffff56dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#19 0x00007ffff7c94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737488342576, 5396937996264594240, 140731775706688, 4294967295, 140737350551504, 140737488342928, -5397405287614719168, -5396956046169547968},
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#20 0x00007ffff7d26850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
No locals.
7

Can the sample deepstream_python_apps/apps/deepstream-nvdsanalytics at master · NVIDIA-AI-IOT/deepstream_python_apps (github.com) run in your device?

8

Yes, its working

9

I have done some tests to see how it works, by default it works without problems, but if I change the model to PeopleNet it gives a Segmentation failure (core dumped).

10

I modified it, I dont need the nvvidanalytics.
Im using kafka but not using the plugin, because I want to send the tracker meta and not the user one.

This is mynvanalytics_src_pad_buffer_probe now:

def nvanalytics_src_pad_buffer_probe(pad,info,u_data):
    frame_number=0
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        print("Unable to get GstBuffer ")
        return

    # Retrieve batch metadata from the gst_buffer
    # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
    # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list

    while l_frame:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        l_obj=frame_meta.obj_meta_list
        obj_counter = {
        PGIE_CLASS_ID_PERSON:0
        }
        frame_number=frame_meta.frame_num
        while l_obj:
            try: 
                # Note that l_obj.data needs a cast to pyds.NvDsObjectMeta
                # The casting is done by pyds.NvDsObjectMeta.cast()
                obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
                obj_meta.rect_params.border_color.set(0.8, 0.149, 0.902, 1.0)
                obj_meta.rect_params.border_width = 1
            except StopIteration:
                break
            obj_counter[obj_meta.class_id] += 1

        display_meta=pyds.nvds_acquire_display_meta_from_pool(batch_meta)
        display_meta.num_labels = 1
        py_nvosd_text_params = display_meta.text_params[0]
        # Setting display text to be shown on screen
        # Note that the pyds module allocates a buffer for the string, and the
        # memory will not be claimed by the garbage collector.
        # Reading the display_text field here will return the C address of the
        # allocated string. Use pyds.get_string() to get the string content.
        py_nvosd_text_params.display_text = "Frame Number={}Person_count={}".format(frame_number, obj_counter[PGIE_CLASS_ID_PERSON])
        py_nvosd_text_params.x_offset = 10
        py_nvosd_text_params.y_offset = 12
        # Font , font-color and font-size
        py_nvosd_text_params.font_params.font_name = "Serif"
        py_nvosd_text_params.font_params.font_size = 8
        # set(red, green, blue, alpha); set to White
        py_nvosd_text_params.font_params.font_color.set(1.0, 1.0, 1.0, 1.0)
        pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)
        
        # Update frame rate through this probe
        stream_index = "stream{0}".format(frame_meta.pad_index)
        global perf_data
        perf_data.update_fps(stream_index)
        #past tracking meta data
        try:
            l_frame=l_frame.next
        except StopIteration:
            break

    return Gst.PadProbeReturn.OK

I want to access the tracker metadata like deepstream-test2.
since this is like an infinite l_frame I don’t really know how to. In order to send the important info to the topic

11

If the deepstream_python_apps/apps/deepstream-test2 at master · NVIDIA-AI-IOT/deepstream_python_apps (github.com) and deepstream_python_apps/apps/deepstream-nvdsanalytics at master · NVIDIA-AI-IOT/deepstream_python_apps (github.com) can works in your environment, there is no issue with DeepStream.

The tracker meta is in the batch user meta. Why do you mention “l_frame”? Please read the samples carefully and implement your code correctly.

12

You are right, I misunderstood about l_user and l_frame has nothing to do with it.

I just added the l_user code from deepstream-test2 in deepstream-nvdsanalytic and it works great.
Thank you.

13

Hi, im experiencing a problem with the tracker, I have added to rtsp_in_rtsp_out the tracker but the console is giving me the same error each frame.

gstnvtracker: Unable to acquire a user meta buffer. Try increasing user-meta-pool-size

The image have his inference and the id with no problem but cant acces to the meta

def pgie_src_pad_buffer_probe(pad, info, u_data):
    #Intiallizing object counter with 0.
    obj_counter = {
        PGIE_CLASS_ID_PERSON:0
    }
    gst_buffer = info.get_buffer()
    if not gst_buffer:
        print("Unable to get GstBuffer ")
        return
    stream_index = 0
    # Retrieve batch metadata from the gst_buffer
    # Note that pyds.gst_buffer_get_nvds_batch_meta() expects the
    # C address of gst_buffer as input, which is obtained with hash(gst_buffer)
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
        try:
            # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
            # The casting is done by pyds.NvDsFrameMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone.
            frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
        except StopIteration:
            break

        l_obj=frame_meta.obj_meta_list
        while l_obj is not None:
            try:
                # Casting l_obj.data to pyds.NvDsObjectMeta
                obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)
                obj_meta.rect_params.border_color.set(0.8, 0.149, 0.902, 1.0)
                obj_meta.rect_params.border_width = 1
            except StopIteration:
                break
            obj_counter[obj_meta.class_id] += 1
            try: 
                l_obj=l_obj.next
            except StopIteration:
                break

        try:
            l_frame=l_frame.next
        except StopIteration:
            break
    #past tracking meta data
    l_user=batch_meta.batch_user_meta_list
    while l_user is not None:
        try:
            # Note that l_user.data needs a cast to pyds.NvDsUserMeta
            # The casting is done by pyds.NvDsUserMeta.cast()
            # The casting also keeps ownership of the underlying memory
            # in the C code, so the Python garbage collector will leave
            # it alone
            user_meta=pyds.NvDsUserMeta.cast(l_user.data)
        except StopIteration:
            break
        if(user_meta and user_meta.base_meta.meta_type==pyds.NvDsMetaType.NVDS_TRACKER_PAST_FRAME_META):
            try:
                # Note that user_meta.user_meta_data needs a cast to pyds.NvDsTargetMiscDataBatch
                # The casting is done by pyds.NvDsTargetMiscDataBatch.cast()
                # The casting also keeps ownership of the underlying memory
                # in the C code, so the Python garbage collector will leave
                # it alone
                pPastDataBatch = pyds.NvDsTargetMiscDataBatch.cast(user_meta.user_meta_data)
            except StopIteration:
                break
            for miscDataStream in pyds.NvDsTargetMiscDataBatch.list(pPastDataBatch):
                print("streamId=",miscDataStream.streamID)
                print("surfaceStreamID=",miscDataStream.surfaceStreamID)
                for miscDataObj in pyds.NvDsTargetMiscDataStream.list(miscDataStream):
                    print("numobj=",miscDataObj.numObj)
                    print("uniqueId=",miscDataObj.uniqueId)
                    print("classId=",miscDataObj.classId)
                    print("objLabel=",miscDataObj.objLabel)
                    for miscDataFrame in pyds.NvDsTargetMiscDataObject.list(miscDataObj):
                        print('frameNum:', miscDataFrame.frameNum)
                        print('tBbox.left:', miscDataFrame.tBbox.left)
                        print('tBbox.width:', miscDataFrame.tBbox.width)
                        print('tBbox.top:', miscDataFrame.tBbox.top)
                        print('tBbox.right:', miscDataFrame.tBbox.height)
                        print('confidence:', miscDataFrame.confidence)
                        print('age:', miscDataFrame.age)
        try:
            l_user=l_user.next
        except StopIteration:
            break
    return Gst.PadProbeReturn.OK
14

Is there any way to calculate the “Tracker miscellaneous data buffer size”? So as not to allocate more than necessary.

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.