Deepstream 5.0 python bindings for Gst-nvdsanalytics access meta data

Hi

Is there any update for when the python bindings for the Gst-nvdsanalytics meta data will be available?

Best,
Niels

It should work on 5.0GA, refer https://docs.nvidia.com/metropolis/deepstream/python-api/NvDsAnalytics/pyanalytics.html

But how do I access the analytics meta data generated from the Gst-nvdsanalytics component? In the C++ example (deepstream-nvdsanalytics-test) they access it from the NvDsUserMeta attached to the NvDsObjectMeta:

for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
                l_obj = l_obj->next) {
            obj_meta = (NvDsObjectMeta *) (l_obj->data);

            // Access attached user meta for each object
            for (NvDsMetaList *l_user_meta = obj_meta->obj_user_meta_list; l_user_meta != NULL;
                    l_user_meta = l_user_meta->next) {
                NvDsUserMeta *user_meta = (NvDsUserMeta *) (l_user_meta->data);
                if(user_meta->base_meta.meta_type == NVDS_USER_OBJ_META_NVDSANALYTICS)
                {
                    NvDsAnalyticsObjInfo * user_meta_data =
                        (NvDsAnalyticsObjInfo *)user_meta->user_meta_data;
                    if (user_meta_data->dirStatus.length()){
                        out_string << " object " << obj_meta->object_id <<
                            " is moving in " <<  user_meta_data->dirStatus;
                    }
                }
            }
        }

I am able to iterate the user meta data in python and it contains something but I cant seem to figure out how to cast it to the NvDsAnalyticsObjInfo object such that I can access the data from python. I have tried the following in my python probe:

            user_meta_list = obj_meta.obj_user_meta_list
            while user_meta_list is not None:
                user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
                user_meta_data = user_meta.user_meta_data
                try: 
                    user_meta_list=user_meta_list.next
                except StopIteration:
                    break 

+1

+1 Would also like to see an examle

Currently, the python bindings don’t have a “cast” function for NvDsAnalyticsObjInfo, but you can generate your bindings and cast function as follows

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
	py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo")
            .def(py::init<>())
            .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
            .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
            .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
            .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
            .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
	        .def(“cast”,[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             });
    py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta)
            .def(py::init<>())
            .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
            .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
            .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
            .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
            .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
            .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
            .def(“cast”,[](void *data){
		 	    return (NvDsAnalyticsFrameMeta *)data;
             });
}

Save this file as pyds_analytics.cpp and then to generate bindings please refer to the following repo https://github.com/mrtj/pyds_tracker_meta
Replace the pyds_tracker_meta.cpp with pyds_analytics.cpp and do the same in build.sh and setup.py files, to generate the bindings. Then you can cast the user_meta.user_meta_data to NvDsAnalyticsObjInfo as follows
import pyds_analytics
ObjInfo=pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)

Ahh perfect! Thanks for the detailed answer. I will make sure to test it as soon as possible.

@manojy Thanks for the workaround info, successfully built and install so file import also working, but import in deepstream app is giving this error

Traceback (most recent call last):
  File "temp.py", line 44, in <module>
    import pyds
ImportError: generic_type: type "NvDsAnalyticsObjInfo" is already registered!

Referring to https://pybind11.readthedocs.io/en/stable/advanced/classes.html added this py::module_local() now able to import in deepstream app but ObjInfo=pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data) this causes Segmentation fault (core dumped) and exits

pyds_analytics.cpp

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include “nvds_analytics_meta.h”

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
py::class_(m,“NvDsAnalyticsObjInfo”, py::module_local())
.def(py::init<>())
.def_readwrite(“roiStatus”,&NvDsAnalyticsObjInfo::roiStatus)
.def_readwrite(“ocStatus”,&NvDsAnalyticsObjInfo::ocStatus)
.def_readwrite(“lcStatus”,&NvDsAnalyticsObjInfo::lcStatus)
.def_readwrite(“dirStatus”,&NvDsAnalyticsObjInfo::dirStatus)
.def_readwrite(“unique_id”,&NvDsAnalyticsObjInfo::unique_id)
.def(“cast”,(void data){
return (NvDsAnalyticsObjInfo
)data;
});
py::class_(m,“NvDsAnalyticsFrameMeta”, py::module_local())
.def(py::init<>())
.def_readwrite(“ocStatus”,&NvDsAnalyticsFrameMeta::ocStatus)
.def_readwrite(“objInROIcnt”,&NvDsAnalyticsFrameMeta::objInROIcnt)
.def_readwrite(“objLCCurrCnt”,&NvDsAnalyticsFrameMeta::objLCCurrCnt)
.def_readwrite(“objLCCumCnt”,&NvDsAnalyticsFrameMeta::objLCCumCnt)
.def_readwrite(“unique_id”,&NvDsAnalyticsFrameMeta::unique_id)
.def_readwrite(“objCnt”,&NvDsAnalyticsFrameMeta::objCnt)
.def(“cast”,(void *data){
return (NvDsAnalyticsFrameMeta *)data;
});
}

App Function

def nvdsanalytics_src_pad_buffer_probe(pad,info,u_data):

gst_buffer = info.get_buffer()
if not gst_buffer:
    print("Unable to get GstBuffer ")
    return

batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None:
    try:
        frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
    except StopIteration:
        break
    l_user = frame_meta.frame_user_meta_list

    while l_user is not None:
        try:
            user_meta = pyds.NvDsUserMeta.cast(l_user.data)
            ObjInfo=pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)

        except StopIteration:
            break
        try:
            l_user=l_user.next
        except sStopIteration:
            break
   
    try:
        l_frame=l_frame.next
    except StopIteration:
        break

return Gst.PadProbeReturn.OK

Can you check your code snippet format, forum support markdown syntax

1 Like

As NvDsAnalyticsObjInfo struct has already binded in pyds, you can simply add the following casting function to get NvDsAnalyticsObjInfo.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
     m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             });
}

@manojy Now really confused what to use and how to use, could you acknowledge what is wrong in above code snippets.

@bcao didn’t get you

Hello

the link to the pyds_tracker_meta bindings worked great

As for pyds_analytics_meta, I am having far more trouble.

Ive had to comment out the line
// .def_readwrite(“objCnt”,&NvDsAnalyticsFrameMeta::objCnt)
============= pyds_analytics_meta.cpp ===============

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;

PYBIND11_MODULE(pyds_tracker_meta, m) {
	m.doc() = "pybind11 wrapper to access Nvidia DeepStream NvDsAnalyticsMeta* classes";
    // m.def("cast_to_NvDsAnalyticsObjInfo"[](void *data){
	// 	return (NvDsAnalyticsObjInfo*)data;
    // });
    py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo", py::module_local())
    // py::class_(m,“NvDsAnalyticsObjInfo”, py::module_local())
            .def(py::init<>())
            .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
            .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
            .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
            .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
            .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
	        .def("cast",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             });
    py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta", py::module_local())
            .def(py::init<>())
            .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
            .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
            .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
            .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
            .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
            // .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)     <------- needed to be uncommented
            .def("cast",[](void *data){
		 	    return (NvDsAnalyticsFrameMeta *)data;
             });
}

============= nvds_analytics_meta.cpp ===============

/**
 * Copyright (c) 2020, NVIDIA CORPORATION.  All rights reserved.
 *
 * NVIDIA Corporation and its licensors retain all intellectual property
 * and proprietary rights in and to this software, related documentation
 * and any modifications thereto.  Any use, reproduction, disclosure or
 * distribution of this software and related documentation without an express
 * license agreement from NVIDIA Corporation is strictly prohibited.
 *
 */
#ifndef _NVDS_ANALYTICS_META_H_
#define _NVDS_ANALYTICS_META_H_

    #include <gst/gst.h>

    #ifdef __cplusplus
    extern "C"
    {
    #endif

    #define NVDS_USER_FRAME_META_NVDSANALYTICS (nvds_get_user_meta_type((gchar*)"NVIDIA.DSANALYTICSFRAME.USER_META"))
    #define NVDS_USER_OBJ_META_NVDSANALYTICS (nvds_get_user_meta_type((gchar*)"NVIDIA.DSANALYTICSOBJ.USER_META"))

    /**
     */
    typedef struct
    {
      std::vector <std::string> roiStatus;
      std::vector <std::string> ocStatus;
      std::vector <std::string> lcStatus;
      std::string dirStatus;
      guint unique_id;
    } NvDsAnalyticsObjInfo;

    /**
     */
    typedef struct
    {
      std::unordered_map<std::string, bool> ocStatus;
      std::unordered_map<std::string, uint32_t> objInROIcnt;
      std::unordered_map<std::string, uint64_t> objLCCurrCnt;
      std::unordered_map<std::string, uint64_t> objLCCumCnt;
      guint unique_id;

    } NvDsAnalyticsFrameMeta;

    #ifdef __cplusplus
    }
    #endif

    #endif

when I check the header file for the class it is indeed missing object count? Could this mean i am using a more out of date deepstream (5.0 jetpack 4.4)

However I can build and install it when objCnt is removed, but I cannot import the class into my deepstream application, or even an empty project, it throws the following error

import pyds_analytics_meta
Traceback (most recent call last):
File “”, line 1, in
ImportError: dynamic module does not define module export function (PyInit_pyds_analytics_meta)

Any help would be greatly appreciated

Regards Andrew

I have fixed some of my issues

Firstly i cleared some space on hdd and managed to update via the software updater (I noticed there were deepstream updates, now nvds_analytics_meta.cpp has objCnt.

I also realised "PYBIND11_MODULE(pyds_tracker_meta, m) " should have been PYBIND11_MODULE(pyds_tracker, m)

Now it successfully imports and i can use:

user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)

however the pipeline freezes and causes Segmentation fault (core dumped) Same as @priyanshthakore

@manojy please clarify this

note missing " , " after cast_to_NvDsAnalyticsObjInfo

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
     m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             });
}

then to access the NvDsAnalyticsObjInfo:

    l_user = frame_meta.frame_user_meta_list

    while l_user is not None:
        try:
            print('try cast')
            user_meta = pyds.NvDsUserMeta.cast(l_user.data)
            if user_meta.user_meta_data:
                try:
                    print('user meta', user_meta.user_meta_data)
                    user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
                    # user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
                    print('user_meta_analytics:', user_meta_analytics)
                except Exception as ex:
                    print('exception', ex)

            print('user_meta', user_meta)
        except StopIteration:
            break
        try:
            l_user = l_user.next
        except StopIteration:
            break
    print('=======================================================================================')

or

user_meta_list = batch_meta.batch_user_meta_list
    while user_meta_list is not None:
        user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
        if user_meta.user_meta_data:
            try:
                print('user meta',user_meta.user_meta_data)
                user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
                # user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
                print('user_meta_analytics:', user_meta_analytics)
            except Exception as ex:
                print('exception',ex)

Yields the following exception

exception Unable to convert function return value to a Python type! The signature was
(arg0: capsule) -> NvDsAnalyticsObjInfo

I am seen the same as @priyanshthakore and @Andrew_Smith.

I did the following as suggested by @manojy and added the py::module_local() (ref @priyanshthakore) :

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
	py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo",py::module_local())
            .def(py::init<>())
            .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
            .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
            .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
            .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
            .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
	        .def("cast",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             });
    py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta",py::module_local())
            .def(py::init<>())
            .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
            .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
            .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
            .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
            .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
            .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
            .def("cast",[](void *data){
		 	    return (NvDsAnalyticsFrameMeta *)data;
             });
}

And accessing the NvDsAnalyticsObjInfo in python through:

user_meta_list = obj_meta.obj_user_meta_list
print(user_meta_list)
while user_meta_list is not None:
     user_meta = pyds.NvDsUserMeta.cast(user_meta_list)
     print(user_meta)
      try:
           analytic_obj = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
           if analytic_obj is not None:
                print(analytic_obj)
           try: 
               user_meta_list=user_meta_list.next
           except StopIteration:
               break

I also did as suggested by @manojy and tried with:

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
     m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
		 	            return (NvDsAnalyticsObjInfo *)data;
             });
     m.def("cast_to_NvDsAnalyticsFrameMeta",[](void *data){
				    return (NvDsAnalyticsFrameMeta *)data;
	});
}

And again accessing the NvDsAnalyticsObjInfo in python through:

user_meta_list = obj_meta.obj_user_meta_list
print(user_meta_list)
        while user_meta_list is not None:
            user_meta = pyds.NvDsUserMeta.cast(user_meta_list)
            print(user_meta)
            try:
                analytic_obj = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
                if analytic_obj is not None:
                      print(analytic_obj)
            try: 
                user_meta_list=user_meta_list.next
            except StopIteration:
                break

The result is the same for both approaches and my terminal output looks like this when it freezes and stops:

user meta (obj) list <pyds.GList object at 0x7f505f29d0>
<pyds.NvDsUserMeta object at 0x7f505f2a40>
<pyds_analytics.NvDsAnalyticsObjInfo object at 0x7f505f2a08>
Segmentation fault (core dumped)

The problem is occurring due to the py::return_value_policy::automatic_reference, which is default pybind11 reference, this is causing memory dealloc problem which is resulting in segmentation fault. So changing it to py::return_value_policy::reference will work. Please use the following solution.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"

namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
	py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo",py::module_local())
            .def(py::init<>())
            .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
            .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
            .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
            .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
            .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
	        .def_static("cast",[](void *data){
		 	            return (NvDsAnalyticsObjInfo*)data;
             },py::return_value_policy::reference);
    py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta",py::module_local())
            .def(py::init<>())
            .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
            .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
            .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
            .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
            .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
            .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
            .def_static("cast",[](void *data){
		 	    return (NvDsAnalyticsFrameMeta *)data;
             },py::return_value_policy::reference);
}
1 Like

@manojy - I just tried the fix and I am still seeing a segmentation fault (core dumped). Have you tried the code yourself and if so do you have a complete working example?

Secondly, is it possible to get and see the actual python deepstream bindings (pyds)? I can only find the compiled .so (pyds.so) in the deepstream SDK.

I tried the fix and it works, no segmentation fault
however I am still attempting to access the data

user_meta_list = batch_meta.batch_user_meta_list
print(’======================================================================================’)

while user_meta_list is not None:
    user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
    if user_meta.user_meta_data:
        try:
            print('user meta',user_meta.user_meta_data)
            # user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
            user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
            print('user_meta_analytics:', user_meta_analytics)
            ============== Breaks here ==============
            print('dirStatus',user_meta_analytics.dirStatus)
            print('lcStatus',user_meta_analytics.lcStatus)
            print('ocStatus',user_meta_analytics.ocStatus)
            print('roiStatus',user_meta_analytics.roiStatus)

        except Exception as ex:
            print('exception',ex)

    try:
        user_meta_list = user_meta_list.next
    except StopIteration:
        break
    print('======================================================================================')

True, I am also able to do the cast but it fails when I access the data fields just as you