Hi
Is there any update for when the python bindings for the Gst-nvdsanalytics meta data will be available?
Best,
Niels
Hi
Is there any update for when the python bindings for the Gst-nvdsanalytics meta data will be available?
Best,
Niels
It should work on 5.0GA, refer Deepstream Python API Reference â Deepstream Deepstream Version: 6.1.1 documentation
But how do I access the analytics meta data generated from the Gst-nvdsanalytics component? In the C++ example (deepstream-nvdsanalytics-test) they access it from the NvDsUserMeta attached to the NvDsObjectMeta:
for (l_obj = frame_meta->obj_meta_list; l_obj != NULL;
l_obj = l_obj->next) {
obj_meta = (NvDsObjectMeta *) (l_obj->data);
// Access attached user meta for each object
for (NvDsMetaList *l_user_meta = obj_meta->obj_user_meta_list; l_user_meta != NULL;
l_user_meta = l_user_meta->next) {
NvDsUserMeta *user_meta = (NvDsUserMeta *) (l_user_meta->data);
if(user_meta->base_meta.meta_type == NVDS_USER_OBJ_META_NVDSANALYTICS)
{
NvDsAnalyticsObjInfo * user_meta_data =
(NvDsAnalyticsObjInfo *)user_meta->user_meta_data;
if (user_meta_data->dirStatus.length()){
out_string << " object " << obj_meta->object_id <<
" is moving in " << user_meta_data->dirStatus;
}
}
}
}
I am able to iterate the user meta data in python and it contains something but I cant seem to figure out how to cast it to the NvDsAnalyticsObjInfo object such that I can access the data from python. I have tried the following in my python probe:
user_meta_list = obj_meta.obj_user_meta_list
while user_meta_list is not None:
user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
user_meta_data = user_meta.user_meta_data
try:
user_meta_list=user_meta_list.next
except StopIteration:
break
+1 Would also like to see an examle
Currently, the python bindings donât have a âcastâ function for NvDsAnalyticsObjInfo, but you can generate your bindings and cast function as follows
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"
namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo")
.def(py::init<>())
.def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
.def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
.def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
.def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
.def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
.def(âcastâ,[](void *data){
return (NvDsAnalyticsObjInfo*)data;
});
py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta)
.def(py::init<>())
.def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
.def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
.def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
.def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
.def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
.def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
.def(âcastâ,[](void *data){
return (NvDsAnalyticsFrameMeta *)data;
});
}
Save this file as pyds_analytics.cpp and then to generate bindings please refer to the following repo GitHub - mrtj/pyds_tracker_meta: pybind11 wrapper to access Nvidia DeepStream tracker meta info from Python.
Replace the pyds_tracker_meta.cpp with pyds_analytics.cpp and do the same in build.sh and setup.py files, to generate the bindings. Then you can cast the user_meta.user_meta_data to NvDsAnalyticsObjInfo as follows
import pyds_analytics
ObjInfo=pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
Ahh perfect! Thanks for the detailed answer. I will make sure to test it as soon as possible.
@manojy Thanks for the workaround info, successfully built and install so file import also working, but import in deepstream app is giving this error
Traceback (most recent call last):
File "temp.py", line 44, in <module>
import pyds
ImportError: generic_type: type "NvDsAnalyticsObjInfo" is already registered!
Referring to Classes - pybind11 documentation added this py::module_local() now able to import in deepstream app but ObjInfo=pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data) this causes Segmentation fault (core dumped) and exits
pyds_analytics.cpp
include <pybind11/pybind11.h>
include <pybind11/stl.h>
include ânvds_analytics_meta.hânamespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
py::class_(m,âNvDsAnalyticsObjInfoâ, py::module_local())
.def(py::init<>())
.def_readwrite(âroiStatusâ,&NvDsAnalyticsObjInfo::roiStatus)
.def_readwrite(âocStatusâ,&NvDsAnalyticsObjInfo::ocStatus)
.def_readwrite(âlcStatusâ,&NvDsAnalyticsObjInfo::lcStatus)
.def_readwrite(âdirStatusâ,&NvDsAnalyticsObjInfo::dirStatus)
.def_readwrite(âunique_idâ,&NvDsAnalyticsObjInfo::unique_id)
.def(âcastâ,(void data){
return (NvDsAnalyticsObjInfo)data;
});
py::class_(m,âNvDsAnalyticsFrameMetaâ, py::module_local())
.def(py::init<>())
.def_readwrite(âocStatusâ,&NvDsAnalyticsFrameMeta::ocStatus)
.def_readwrite(âobjInROIcntâ,&NvDsAnalyticsFrameMeta::objInROIcnt)
.def_readwrite(âobjLCCurrCntâ,&NvDsAnalyticsFrameMeta::objLCCurrCnt)
.def_readwrite(âobjLCCumCntâ,&NvDsAnalyticsFrameMeta::objLCCumCnt)
.def_readwrite(âunique_idâ,&NvDsAnalyticsFrameMeta::unique_id)
.def_readwrite(âobjCntâ,&NvDsAnalyticsFrameMeta::objCnt)
.def(âcastâ,(void *data){
return (NvDsAnalyticsFrameMeta *)data;
});
}
App Function
def nvdsanalytics_src_pad_buffer_probe(pad,info,u_data):
gst_buffer = info.get_buffer() if not gst_buffer: print("Unable to get GstBuffer ") return batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer)) l_frame = batch_meta.frame_meta_list while l_frame is not None: try: frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data) except StopIteration: break l_user = frame_meta.frame_user_meta_list while l_user is not None: try: user_meta = pyds.NvDsUserMeta.cast(l_user.data) ObjInfo=pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data) except StopIteration: break try: l_user=l_user.next except sStopIteration: break try: l_frame=l_frame.next except StopIteration: break return Gst.PadProbeReturn.OK
Can you check your code snippet format, forum support markdown syntax
As NvDsAnalyticsObjInfo struct has already binded in pyds, you can simply add the following casting function to get NvDsAnalyticsObjInfo.
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"
namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
return (NvDsAnalyticsObjInfo*)data;
});
}
@manojy Now really confused what to use and how to use, could you acknowledge what is wrong in above code snippets.
@bcao didnât get you
Hello
the link to the pyds_tracker_meta bindings worked great
As for pyds_analytics_meta, I am having far more trouble.
Ive had to comment out the line
// .def_readwrite(âobjCntâ,&NvDsAnalyticsFrameMeta::objCnt)
============= pyds_analytics_meta.cpp ===============
#include <pybind11/pybind11.h> #include <pybind11/stl.h> #include "nvds_analytics_meta.h" namespace py = pybind11; PYBIND11_MODULE(pyds_tracker_meta, m) { m.doc() = "pybind11 wrapper to access Nvidia DeepStream NvDsAnalyticsMeta* classes"; // m.def("cast_to_NvDsAnalyticsObjInfo"[](void *data){ // return (NvDsAnalyticsObjInfo*)data; // }); py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo", py::module_local()) // py::class_(m,âNvDsAnalyticsObjInfoâ, py::module_local()) .def(py::init<>()) .def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus) .def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus) .def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus) .def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus) .def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id) .def("cast",[](void *data){ return (NvDsAnalyticsObjInfo*)data; }); py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta", py::module_local()) .def(py::init<>()) .def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus) .def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt) .def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt) .def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt) .def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id) // .def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt) <------- needed to be uncommented .def("cast",[](void *data){ return (NvDsAnalyticsFrameMeta *)data; }); }
============= nvds_analytics_meta.cpp ===============
/** * Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved. * * NVIDIA Corporation and its licensors retain all intellectual property * and proprietary rights in and to this software, related documentation * and any modifications thereto. Any use, reproduction, disclosure or * distribution of this software and related documentation without an express * license agreement from NVIDIA Corporation is strictly prohibited. * */ #ifndef _NVDS_ANALYTICS_META_H_ #define _NVDS_ANALYTICS_META_H_ #include <gst/gst.h> #ifdef __cplusplus extern "C" { #endif #define NVDS_USER_FRAME_META_NVDSANALYTICS (nvds_get_user_meta_type((gchar*)"NVIDIA.DSANALYTICSFRAME.USER_META")) #define NVDS_USER_OBJ_META_NVDSANALYTICS (nvds_get_user_meta_type((gchar*)"NVIDIA.DSANALYTICSOBJ.USER_META")) /** */ typedef struct { std::vector <std::string> roiStatus; std::vector <std::string> ocStatus; std::vector <std::string> lcStatus; std::string dirStatus; guint unique_id; } NvDsAnalyticsObjInfo; /** */ typedef struct { std::unordered_map<std::string, bool> ocStatus; std::unordered_map<std::string, uint32_t> objInROIcnt; std::unordered_map<std::string, uint64_t> objLCCurrCnt; std::unordered_map<std::string, uint64_t> objLCCumCnt; guint unique_id; } NvDsAnalyticsFrameMeta; #ifdef __cplusplus } #endif #endif
when I check the header file for the class it is indeed missing object count? Could this mean i am using a more out of date deepstream (5.0 jetpack 4.4)
However I can build and install it when objCnt is removed, but I cannot import the class into my deepstream application, or even an empty project, it throws the following error
import pyds_analytics_meta
Traceback (most recent call last):
File ââ, line 1, in
ImportError: dynamic module does not define module export function (PyInit_pyds_analytics_meta)
Any help would be greatly appreciated
Regards Andrew
I have fixed some of my issues
Firstly i cleared some space on hdd and managed to update via the software updater (I noticed there were deepstream updates, now nvds_analytics_meta.cpp has objCnt.
I also realised "PYBIND11_MODULE(pyds_tracker_meta, m) " should have been PYBIND11_MODULE(pyds_tracker, m)
Now it successfully imports and i can use:
user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
however the pipeline freezes and causes Segmentation fault (core dumped) Same as @priyanshthakore
@manojy please clarify this
note missing " , " after cast_to_NvDsAnalyticsObjInfo
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"
namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
return (NvDsAnalyticsObjInfo*)data;
});
}
then to access the NvDsAnalyticsObjInfo:
l_user = frame_meta.frame_user_meta_list
while l_user is not None:
try:
print('try cast')
user_meta = pyds.NvDsUserMeta.cast(l_user.data)
if user_meta.user_meta_data:
try:
print('user meta', user_meta.user_meta_data)
user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
# user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
print('user_meta_analytics:', user_meta_analytics)
except Exception as ex:
print('exception', ex)
print('user_meta', user_meta)
except StopIteration:
break
try:
l_user = l_user.next
except StopIteration:
break
print('=======================================================================================')
or
user_meta_list = batch_meta.batch_user_meta_list
while user_meta_list is not None:
user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
if user_meta.user_meta_data:
try:
print('user meta',user_meta.user_meta_data)
user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
# user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
print('user_meta_analytics:', user_meta_analytics)
except Exception as ex:
print('exception',ex)
Yields the following exception
exception Unable to convert function return value to a Python type! The signature was
(arg0: capsule) â NvDsAnalyticsObjInfo
I am seen the same as @priyanshthakore and @Andrew_Smith.
I did the following as suggested by @manojy and added the py::module_local() (ref @priyanshthakore) :
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"
namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo",py::module_local())
.def(py::init<>())
.def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
.def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
.def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
.def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
.def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
.def("cast",[](void *data){
return (NvDsAnalyticsObjInfo*)data;
});
py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta",py::module_local())
.def(py::init<>())
.def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
.def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
.def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
.def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
.def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
.def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
.def("cast",[](void *data){
return (NvDsAnalyticsFrameMeta *)data;
});
}
And accessing the NvDsAnalyticsObjInfo in python through:
user_meta_list = obj_meta.obj_user_meta_list
print(user_meta_list)
while user_meta_list is not None:
user_meta = pyds.NvDsUserMeta.cast(user_meta_list)
print(user_meta)
try:
analytic_obj = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data)
if analytic_obj is not None:
print(analytic_obj)
try:
user_meta_list=user_meta_list.next
except StopIteration:
break
I also did as suggested by @manojy and tried with:
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"
namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
m.def("cast_to_NvDsAnalyticsObjInfo",[](void *data){
return (NvDsAnalyticsObjInfo *)data;
});
m.def("cast_to_NvDsAnalyticsFrameMeta",[](void *data){
return (NvDsAnalyticsFrameMeta *)data;
});
}
And again accessing the NvDsAnalyticsObjInfo in python through:
user_meta_list = obj_meta.obj_user_meta_list
print(user_meta_list)
while user_meta_list is not None:
user_meta = pyds.NvDsUserMeta.cast(user_meta_list)
print(user_meta)
try:
analytic_obj = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data)
if analytic_obj is not None:
print(analytic_obj)
try:
user_meta_list=user_meta_list.next
except StopIteration:
break
The result is the same for both approaches and my terminal output looks like this when it freezes and stops:
user meta (obj) list <pyds.GList object at 0x7f505f29d0>
<pyds.NvDsUserMeta object at 0x7f505f2a40>
<pyds_analytics.NvDsAnalyticsObjInfo object at 0x7f505f2a08>
Segmentation fault (core dumped)
The problem is occurring due to the py::return_value_policy::automatic_reference, which is default pybind11 reference, this is causing memory dealloc problem which is resulting in segmentation fault. So changing it to py::return_value_policy::reference will work. Please use the following solution.
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvds_analytics_meta.h"
namespace py = pybind11;
PYBIND11_MODULE(pyds_analytics, m) {
py::class_<NvDsAnalyticsObjInfo>(m,"NvDsAnalyticsObjInfo",py::module_local())
.def(py::init<>())
.def_readwrite("roiStatus",&NvDsAnalyticsObjInfo::roiStatus)
.def_readwrite("ocStatus",&NvDsAnalyticsObjInfo::ocStatus)
.def_readwrite("lcStatus",&NvDsAnalyticsObjInfo::lcStatus)
.def_readwrite("dirStatus",&NvDsAnalyticsObjInfo::dirStatus)
.def_readwrite("unique_id",&NvDsAnalyticsObjInfo::unique_id)
.def_static("cast",[](void *data){
return (NvDsAnalyticsObjInfo*)data;
},py::return_value_policy::reference);
py::class_<NvDsAnalyticsFrameMeta>(m,"NvDsAnalyticsFrameMeta",py::module_local())
.def(py::init<>())
.def_readwrite("ocStatus",&NvDsAnalyticsFrameMeta::ocStatus)
.def_readwrite("objInROIcnt",&NvDsAnalyticsFrameMeta::objInROIcnt)
.def_readwrite("objLCCurrCnt",&NvDsAnalyticsFrameMeta::objLCCurrCnt)
.def_readwrite("objLCCumCnt",&NvDsAnalyticsFrameMeta::objLCCumCnt)
.def_readwrite("unique_id",&NvDsAnalyticsFrameMeta::unique_id)
.def_readwrite("objCnt",&NvDsAnalyticsFrameMeta::objCnt)
.def_static("cast",[](void *data){
return (NvDsAnalyticsFrameMeta *)data;
},py::return_value_policy::reference);
}
@manojy - I just tried the fix and I am still seeing a segmentation fault (core dumped). Have you tried the code yourself and if so do you have a complete working example?
Secondly, is it possible to get and see the actual python deepstream bindings (pyds)? I can only find the compiled .so (pyds.so) in the deepstream SDK.
I tried the fix and it works, no segmentation fault
however I am still attempting to access the data
user_meta_list = batch_meta.batch_user_meta_list
print(â======================================================================================â)while user_meta_list is not None: user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data) if user_meta.user_meta_data: try: print('user meta',user_meta.user_meta_data) # user_meta_analytics = pyds_analytics.cast_to_NvDsAnalyticsObjInfo(user_meta.user_meta_data) user_meta_analytics = pyds_analytics.NvDsAnalyticsObjInfo.cast(user_meta.user_meta_data) print('user_meta_analytics:', user_meta_analytics) ============== Breaks here ============== print('dirStatus',user_meta_analytics.dirStatus) print('lcStatus',user_meta_analytics.lcStatus) print('ocStatus',user_meta_analytics.ocStatus) print('roiStatus',user_meta_analytics.roiStatus) except Exception as ex: print('exception',ex) try: user_meta_list = user_meta_list.next except StopIteration: break print('======================================================================================')
True, I am also able to do the cast but it fails when I access the data fields just as you