Deepstream 5.0 python API

Hi,
I’m using the python bindings of Deepstream 5.0 to access frame meta data.
Looks like the python API documentation is not complete. In particular I see no documentation on how to access the information contained in:
NvDsObjectMeta.detector_bbox_info
NvDsObjectMeta.tracker_bbox_info
According to the documentation these properties are:
Holds a structure containing bounding box parameters of the object when detected by detector.
Holds a structure containing bounding box coordinates of the object when processed by tracker.

In deepstream_test_3.py sample, there is a obj_meta python object that is acquired as this.
obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)

From there I can access several attributes:
obj_meta.rect_params.left
obj_meta.rect_params.top
obj_meta.rect_params.width
obj_meta.rect_params.height
obj_meta.rect_params.object_id

However, I would like to know how to access detector and tracking object info. I try this:
det_info = obj_meta.detector_bbox_info
or this:
det_info = obj_meta.detector_bbox_info.org_bbox_coords

but I alwyas get this error:
TypeError: Unable to convert function return value to a Python type! The signature was
(self: pyds.NvDsComp_BboxInfo) -> _NvBbox_Coords

What is the way (if any) to access this info in python?

Thanks in advance

Hey, could you share your setup with us, especially which DS version are you using?

Hi, this is what I’m using:
• Hardware Platform: Ubuntu 18.04, Driver v 450.51, GPU GeForce RTX 2070 Mobile
• Cuda version: 10.2
• DeepStream Version: 5.0.1
• pyds version: 1.0.1
• TensorRT: Version 7.1.3.4

Current bindings are missing bindings for NvDsComp_BboxInfo, but you can create your own bindings by following the similar steps posted in this github page, https://github.com/mrtj/pyds_tracker_meta, please replace pyds_tracker_meta with the following code in pyds_bbox_meta.cpp and also make sure you do similarly in setup.py before making the build.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvdsmeta.h"
#include <iostream>
#include "nvll_osd_struct.h"
using namespace std;
namespace py = pybind11;

PYBIND11_MODULE(pyds_bbox_meta, m) {
    m.doc() = "pybind11 wrapper to access Nvidia DeepStream detector bbox";

#define STRING_CHAR_ARRAY(TYPE, FIELD)                             \
        [](const TYPE &self)->string {                             \
            return string(self.FIELD);                             \
        },                                                         \
        [](TYPE &self, std::string str) {                          \
            int strSize = str.size();                              \
            str.copy(self.FIELD, strSize);                         \
        },                                                         \
        py::return_value_policy::reference

	py::class_<NvDsObjectMeta>(m,"NvDsObjectMeta",py::module_local())
                .def(py::init<>())
                .def_readwrite("base_meta", &NvDsObjectMeta::base_meta)
       
                .def_readwrite("parent", &NvDsObjectMeta::parent)
        
                .def_readwrite("unique_component_id", &NvDsObjectMeta::unique_component_id)
       
                .def_readwrite("class_id", &NvDsObjectMeta::class_id)        
        
                .def_readwrite("object_id", &NvDsObjectMeta::object_id)
        
                .def_readwrite("detector_bbox_info",&NvDsObjectMeta::detector_bbox_info)

                .def_readwrite("tracker_bbox_info",&NvDsObjectMeta::tracker_bbox_info) 
            
                .def_readwrite("confidence", &NvDsObjectMeta::confidence)

                .def_readwrite("tracker_confidence",&NvDsObjectMeta::tracker_confidence)
        
                .def_readwrite("rect_params", &NvDsObjectMeta::rect_params)

                .def_readwrite("mask_params",&NvDsObjectMeta::mask_params)
	
         
                .def_readwrite("text_params", &NvDsObjectMeta::text_params)
                
                .def("cast",[](void *data) {
                        return (NvDsObjectMeta*)data;},
                        py::return_value_policy::reference)
        
                .def_property("obj_label", STRING_CHAR_ARRAY(NvDsObjectMeta, obj_label))
                .def_readwrite("classifier_meta_list", &NvDsObjectMeta::classifier_meta_list)        
     
                .def_readwrite("obj_user_meta_list", &NvDsObjectMeta::obj_user_meta_list)
                .def_property("misc_obj_info",
				[](NvDsObjectMeta &self)->py::array {
					auto dtype=py::dtype(py::format_descriptor<int>::format());
                    auto base=py::array(dtype, {MAX_USER_FIELDS}, {sizeof(int)});
                    return py::array(dtype,{MAX_USER_FIELDS}, {sizeof(int)}, self.misc_obj_info,base);
				}, [](NvDsObjectMeta& self){})
                .def_property("reserved",
				[](NvDsObjectMeta &self)->py::array {
					auto dtype=py::dtype(py::format_descriptor<int>::format());
                    auto base=py::array(dtype, {MAX_RESERVED_FIELDS}, {sizeof(int)});
                    return py::array(dtype,{MAX_RESERVED_FIELDS}, {sizeof(int)}, self.reserved,base);
				}, [](NvDsObjectMeta& self){});

	py::class_<NvOSD_RectParams>(m,"NvOSD_RectParams",py::module_local())
                .def(py::init<>())
                .def_readwrite("left",&NvOSD_RectParams::left)
                .def_readwrite("top",&NvOSD_RectParams::top)
                .def_readwrite("width",&NvOSD_RectParams::width)
                .def_readwrite("height",&NvOSD_RectParams::height)
                .def_readwrite("border_width",&NvOSD_RectParams::border_width)
                .def_readwrite("border_color",&NvOSD_RectParams::border_color)
                .def_readwrite("has_bg_color",&NvOSD_RectParams::has_bg_color)
                .def_readwrite("reserved",&NvOSD_RectParams::reserved)
                .def_readwrite("bg_color",&NvOSD_RectParams::bg_color)
                .def_readwrite("has_color_info",&NvOSD_RectParams::has_color_info)
                .def_readwrite("color_id",&NvOSD_RectParams::color_id);
	py::class_<NvDsComp_BboxInfo>(m,"NvDsComp_BboxInfo",py::module_local())
                .def(py::init<>())

                .def_readwrite("org_bbox_coords",&NvDsComp_BboxInfo::org_bbox_coords);

	 py::class_<NvBbox_Coords>(m,"NvBbox_Coords",py::module_local())
            .def(py::init<>())
            .def_readwrite("left",&NvBbox_Coords::left)
            .def_readwrite("top",&NvBbox_Coords::top)
            .def_readwrite("width",&NvBbox_Coords::width)
            .def_readwrite("height",&NvBbox_Coords::height);

Now using the built pyds_bbox_meta.so, you can get detector_bbox_info using the following code

obj_meta=pyds_bbox_meta.NvDsObjectMeta.cast(l_obj.data)
det_info_left = obj_meta.detector_bbox_info.org_bbox_coords.left
det_info_top = obj_meta.detector_bbox_info.org_bbox_coords.top

you can follow the similar procedure for creating and getting tracker bbox info as well.

1 Like

Hi,
I’m trying to do this but get the error

pyds_bbox_meta.cpp: In function ‘void pybind11_init_pyds_bbox_meta(pybind11::module_&)’:
pyds_bbox_meta.cpp:58:76: error: ‘array’ in namespace ‘py’ does not name a type
(NvDsObjectMeta &self)->py::array {

Also If I’ll be able to build this, what is it exactly that i need to do with pyds_bbox_meta.so? I’ve got some python code that runs, do i link it to something? Sorry if these are stupid questions, but new to deepstream.

UPDATE: I managed to build the above code, you need to also include

#include <pybind11/numpy.h>

and it is missing a closing }

still not sure what to do with the .so

1 Like

Either add the parent folder of your .so to your PYTHONPATH, or if you edited the setup.py with the corresponding package_data, you can run pip install /path/to/setup_py/folder.

Then, you can use the providded snippet:

import pyds_bbox_meta
...  # add rest of callback code here
obj_meta=pyds_bbox_meta.NvDsObjectMeta.cast(l_obj.data)
det_info_left = obj_meta.detector_bbox_info.org_bbox_coords.left
det_info_top = obj_meta.detector_bbox_info.org_bbox_coords.top
1 Like