Deepstream 5.0 python bindings for Gst-nvdsanalytics access meta data

I am seen the same as @priyanshthakore and @Andrew_Smith.

I did the following as suggested by @manojy and added the py::module_local() (ref @priyanshthakore) :

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
	py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo",py::module_local())
            .def(py::init<>())
            .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
            .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
            .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
            .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
            .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
	        .def("cast",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             });
    py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta",py::module_local())
            .def(py::init<>())
            .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
            .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
            .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
            .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
            .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
            .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
            .def("cast",[](void *data){
		 	    return (NvDsAnalyticsFrameMeta *)data;
             });
}

And accessing the NvDsAnalyticsObjInfo in python through:

user_meta_list = obj_meta.obj_user_meta_list
print(user_meta_list)
while user_meta_list is not None:
     user_meta = pyds.NvDsUserMeta.cast(user_meta_list)
     print(user_meta)
      try:
           analytic_obj = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
           if analytic_obj is not None:
                print(analytic_obj)
           try: 
               user_meta_list=user_meta_list.next
           except StopIteration:
               break

I also did as suggested by @manojy and tried with:

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
     m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
		 	            return (NvDsAnalyticsObjInfo *)data;
             });
     m.def("cast_to_NvDsAnalyticsFrameMeta",[](void *data){
				    return (NvDsAnalyticsFrameMeta *)data;
	});
}

And again accessing the NvDsAnalyticsObjInfo in python through:

user_meta_list = obj_meta.obj_user_meta_list
print(user_meta_list)
        while user_meta_list is not None:
            user_meta = pyds.NvDsUserMeta.cast(user_meta_list)
            print(user_meta)
            try:
                analytic_obj = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
                if analytic_obj is not None:
                      print(analytic_obj)
            try: 
                user_meta_list=user_meta_list.next
            except StopIteration:
                break

The result is the same for both approaches and my terminal output looks like this when it freezes and stops:

user meta (obj) list <pyds.GList object at 0x7f505f29d0>
<pyds.NvDsUserMeta object at 0x7f505f2a40>
<pyds_analytics.NvDsAnalyticsObjInfo object at 0x7f505f2a08>
Segmentation fault (core dumped)

The problem is occurring due to the py::return_value_policy::automatic_reference, which is default pybind11 reference, this is causing memory dealloc problem which is resulting in segmentation fault. So changing it to py::return_value_policy::reference will work. Please use the following solution.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
	py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo",py::module_local())
            .def(py::init<>())
            .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
            .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
            .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
            .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
            .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
	        .def_static("cast",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             },py::return_value_policy::reference);
    py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta",py::module_local())
            .def(py::init<>())
            .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
            .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
            .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
            .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
            .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
            .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
            .def_static("cast",[](void *data){
		 	    return (NvDsAnalyticsFrameMeta *)data;
             },py::return_value_policy::reference);
}
1 Like

@manojy - I just tried the fix and I am still seeing a segmentation fault (core dumped). Have you tried the code yourself and if so do you have a complete working example?

Secondly, is it possible to get and see the actual python deepstream bindings (pyds)? I can only find the compiled .so (pyds.so) in the deepstream SDK.

I tried the fix and it works, no segmentation fault
however I am still attempting to access the data

user_meta_list = batch_meta.batch_user_meta_list
print(’======================================================================================’)

while user_meta_list is not None:
    user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
    if user_meta.user_meta_data:
        try:
            print('user meta',user_meta.user_meta_data)
            # user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
            user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
            print('user_meta_analytics:', user_meta_analytics)
            ============== Breaks here ==============
            print('dirStatus',user_meta_analytics.dirStatus)
            print('lcStatus',user_meta_analytics.lcStatus)
            print('ocStatus',user_meta_analytics.ocStatus)
            print('roiStatus',user_meta_analytics.roiStatus)

        except Exception as ex:
            print('exception',ex)

    try:
        user_meta_list = user_meta_list.next
    except StopIteration:
        break
    print('======================================================================================')

True, I am also able to do the cast but it fails when I access the data fields just as you

This works for me

 l_obj = frame_meta.obj_meta_list
 while l_obj is not None:
       try:
            obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
            user_meta_list = obj_meta.obj_user_meta_list
            
            while user_meta_list is not None:
                user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
                user_meta_data = user_meta.user_meta_data
                
                if user_meta.base_meta.meta_type == pyds.nvds_get_user_meta_type("NVIDIA.DSANALYTICSOBJ.USER_META"):
                    user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
                    print(user_meta_analytics.lcStatus)

for frame data

l_user = frame_meta.frame_user_meta_list
while l_user is not None:
    try:
        user_meta = pyds.NvDsUserMeta.cast(l_user.data)
        if user_meta.base_meta.meta_type != pyds.nvds_get_user_meta_type("NVIDIA.DSANALYTICSFRAME.USER_META"):
            continue
        
        user_meta_analytics = pyds_analytics.NvDsAnalyticsFrameMeta.cast(user_meta.user_meta_data)
    
        print(user_meta_analytics.objLCCumCnt)
3 Likes

@norrarat.s - Yes that works perfect, thanks!

Yes it seems to be working now
Thanks @manojy for the guiding us and @norrarat.s for adding last piece to puzzle.

Hi All,

I am still having issues accessing the metadata for individual objects. I can get all the properties for my frame no problem ie:

l_frame = batch_meta.frame_meta_list
        while l_frame is not None:
            try:
                frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
                l_user = frame_meta.frame_user_meta_list
                while l_user is not None:
                    try:
                        user_meta = pyds.NvDsUserMeta.cast(l_user.data)
                        if user_meta.base_meta.meta_type != pyds.nvds_get_user_meta_type(
                                "NVIDIA.DSANALYTICSFRAME.USER_META"):
                            continue

                        user_meta_analytics = pyds_analytics.NvDsAnalyticsFrameMeta.cast(user_meta.user_meta_data)
                        print('objCnt',user_meta_analytics.objCnt)
                        print('objInROIcnt',user_meta_analytics.objInROIcnt)
                        print('objLCCumCnt',user_meta_analytics.objLCCumCnt)
                        print('objLCCurrCnt',user_meta_analytics.objLCCurrCnt)
                        print('ocStatus',user_meta_analytics.ocStatus)
                        print('unique_id',user_meta_analytics.unique_id)
                        print(user_meta_analytics.objLCCumCnt)
                    except Exception as ex:
                        print('Exception', ex)
                    try:
                        l_user = l_user.next
                    except StopIteration:
                        break

But i cant get the other part the work, as per @norrarat.s methods

The program doesnt crash, but my user_meta_list = obj_meta.obj_user_meta_list is always None.

 l_obj = frame_meta.obj_meta_list
                while l_obj is not None:
                    try:
                        # Casting l_obj.data to pyds.NvDsObjectMeta
                        obj_meta = pyds.NvDsObjectMeta.cast(l_obj.data)
                        user_meta_list = obj_meta.obj_user_meta_list
                        # print('user_meta_list', user_meta_list)
                        # print('class id', obj_meta.class_id)
                        while user_meta_list is not None:
                            try:
                                print('user_meta_list', user_meta_list)
                                user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
                                user_meta_data = user_meta.user_meta_data
                                print('user_meta.base_meta.meta_type', user_meta.base_meta.meta_type)
                                if user_meta.base_meta.meta_type == pyds.nvds_get_user_meta_type(
                                        "NVIDIA.DSANALYTICSOBJ.USER_META"):
                                    user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(
                                        user_meta.user_meta_data)
                                    print(user_meta_analytics.lcStatus)
                            except StopIteration:
                                break
                            try:
                                print('Next Meta List')
                                user_meta_list = user_meta_list.next
                            except StopIteration:
                                break
                    except Exception as ex:
                        print('Exception', ex)
                    try:
                        l_obj = l_obj.next
                    except StopIteration:
                        break

I have added direction detection and in the OSD i can see the object being assigned a direction

@niels.kjaergaard.madsen @priyanshthakore if anyone has some help

Regards Andrew

try this

if user_meta.base_meta.meta_type != pyds.nvds_get_user_meta_type("NVIDIA.DSANALYTICSOBJ.USER_META"):
    continue
user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
print(user_meta_analytics.lcStatus)

Hello guys,

thanks for you efforts to get metadata for NvDsAnalytics. Everything works perfect!

I’ve create repo in GitHub pyds_analytics_meta, followed this one pyds_tracker_meta to simplify installation of python bindings for NvDsAnalyticsFrameMeta and NvDsAnalyticsObjInfo.

1 Like

@rostislav.etc

Thanks for the github repo I’ve just tried it.

I still have the same problem where print("----NvDsAnalytics Object Info----") contains no details

again this is because the line below is None.
user_meta_list = obj_meta.obj_user_meta_list

am I missing something in my pipeline/config ? I know for the tracker you need to enable past frame

Yes, I’ve update the source of example. It should be under first while.

There’s no any addition options you need to enable. NvDsAnalytics*Meta have information only in frames, where according events appears: direction, crossing line or inside ROI, therefore, after check base_meta.meta_type you might not see info.

So the frame meta concept I understand, it will give you information about the analytics applied to a frame, ie for a line crossing you can read how many cars have crossed that line, this all works perfectly with:

frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)

The Part I am still not getting is the NvDsAnalyticsObjInfo, from what I understand, this is the AnalyticsMeta applied to objects in the scene? I thought the NvDsAnalyticsObjInfo would contain the direction a car is traveling, or if its within the bounds of an ROI filter.

But in my case I can only access the ROI and the line crossing, so i can see there is 1 object in the ROI or 8 cars have crossed a line, but I cant loop through my objects and read their NvDsAnalyticsObjInfo to discover which car is in the ROI.

I can clearly see an object has a Direction applied to their tag in the OSD, but i cant access that metadata

Hi, Were you able to solve this issue? I am also getting none for user_meta_list = obj_meta.obj_user_meta_list
How did you solve it?

@lakshay.chhabra Use deepstream 5.1 they have fixed the issue and added sample app for the same

I m using 5.1 only

could you post a link to this sample app, please?