Deepstream 5.0 python API

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,
I’m using the python bindings of Deepstream 5.0 to access frame meta data.
Looks like the python API documentation is not complete. In particular I see no documentation on how to access the information contained in:
NvDsObjectMeta.detector_bbox_info
NvDsObjectMeta.tracker_bbox_info
According to the documentation these properties are:
Holds a structure containing bounding box parameters of the object when detected by detector.
Holds a structure containing bounding box coordinates of the object when processed by tracker.

In deepstream_test_3.py sample, there is a obj_meta python object that is acquired as this.
obj_meta=pyds.NvDsObjectMeta.cast(l_obj.data)

From there I can access several attributes:
obj_meta.rect_params.left
obj_meta.rect_params.top
obj_meta.rect_params.width
obj_meta.rect_params.height
obj_meta.rect_params.object_id

However, I would like to know how to access detector and tracking object info. I try this:
det_info = obj_meta.detector_bbox_info
or this:
det_info = obj_meta.detector_bbox_info.org_bbox_coords

but I alwyas get this error:
TypeError: Unable to convert function return value to a Python type! The signature was
(self: pyds.NvDsComp_BboxInfo) → _NvBbox_Coords

What is the way (if any) to access this info in python?

Thanks in advance

3

Hey, could you share your setup with us, especially which DS version are you using?

4

Hi, this is what I’m using:
• Hardware Platform: Ubuntu 18.04, Driver v 450.51, GPU GeForce RTX 2070 Mobile
• Cuda version: 10.2
• DeepStream Version: 5.0.1
• pyds version: 1.0.1
• TensorRT: Version 7.1.3.4

5

Current bindings are missing bindings for NvDsComp_BboxInfo, but you can create your own bindings by following the similar steps posted in this github page, GitHub - mrtj/pyds_tracker_meta: pybind11 wrapper to access Nvidia DeepStream tracker meta info from Python., please replace pyds_tracker_meta with the following code in pyds_bbox_meta.cpp and also make sure you do similarly in setup.py before making the build.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include "nvdsmeta.h"
#include <iostream>
#include "nvll_osd_struct.h"
using namespace std;
namespace py = pybind11;

PYBIND11_MODULE(pyds_bbox_meta, m) {
    m.doc() = "pybind11 wrapper to access Nvidia DeepStream detector bbox";

#define STRING_CHAR_ARRAY(TYPE, FIELD)                             \
        [](const TYPE &self)->string {                             \
            return string(self.FIELD);                             \
        },                                                         \
        [](TYPE &self, std::string str) {                          \
            int strSize = str.size();                              \
            str.copy(self.FIELD, strSize);                         \
        },                                                         \
        py::return_value_policy::reference

	py::class_<NvDsObjectMeta>(m,"NvDsObjectMeta",py::module_local())
                .def(py::init<>())
                .def_readwrite("base_meta", &NvDsObjectMeta::base_meta)
       
                .def_readwrite("parent", &NvDsObjectMeta::parent)
        
                .def_readwrite("unique_component_id", &NvDsObjectMeta::unique_component_id)
       
                .def_readwrite("class_id", &NvDsObjectMeta::class_id)        
        
                .def_readwrite("object_id", &NvDsObjectMeta::object_id)
        
                .def_readwrite("detector_bbox_info",&NvDsObjectMeta::detector_bbox_info)

                .def_readwrite("tracker_bbox_info",&NvDsObjectMeta::tracker_bbox_info) 
            
                .def_readwrite("confidence", &NvDsObjectMeta::confidence)

                .def_readwrite("tracker_confidence",&NvDsObjectMeta::tracker_confidence)
        
                .def_readwrite("rect_params", &NvDsObjectMeta::rect_params)

                .def_readwrite("mask_params",&NvDsObjectMeta::mask_params)
	
         
                .def_readwrite("text_params", &NvDsObjectMeta::text_params)
                
                .def("cast",[](void *data) {
                        return (NvDsObjectMeta*)data;},
                        py::return_value_policy::reference)
        
                .def_property("obj_label", STRING_CHAR_ARRAY(NvDsObjectMeta, obj_label))
                .def_readwrite("classifier_meta_list", &NvDsObjectMeta::classifier_meta_list)        
     
                .def_readwrite("obj_user_meta_list", &NvDsObjectMeta::obj_user_meta_list)
                .def_property("misc_obj_info",
				[](NvDsObjectMeta &self)->py::array {
					auto dtype=py::dtype(py::format_descriptor<int>::format());
                    auto base=py::array(dtype, {MAX_USER_FIELDS}, {sizeof(int)});
                    return py::array(dtype,{MAX_USER_FIELDS}, {sizeof(int)}, self.misc_obj_info,base);
				}, [](NvDsObjectMeta& self){})
                .def_property("reserved",
				[](NvDsObjectMeta &self)->py::array {
					auto dtype=py::dtype(py::format_descriptor<int>::format());
                    auto base=py::array(dtype, {MAX_RESERVED_FIELDS}, {sizeof(int)});
                    return py::array(dtype,{MAX_RESERVED_FIELDS}, {sizeof(int)}, self.reserved,base);
				}, [](NvDsObjectMeta& self){});

	py::class_<NvOSD_RectParams>(m,"NvOSD_RectParams",py::module_local())
                .def(py::init<>())
                .def_readwrite("left",&NvOSD_RectParams::left)
                .def_readwrite("top",&NvOSD_RectParams::top)
                .def_readwrite("width",&NvOSD_RectParams::width)
                .def_readwrite("height",&NvOSD_RectParams::height)
                .def_readwrite("border_width",&NvOSD_RectParams::border_width)
                .def_readwrite("border_color",&NvOSD_RectParams::border_color)
                .def_readwrite("has_bg_color",&NvOSD_RectParams::has_bg_color)
                .def_readwrite("reserved",&NvOSD_RectParams::reserved)
                .def_readwrite("bg_color",&NvOSD_RectParams::bg_color)
                .def_readwrite("has_color_info",&NvOSD_RectParams::has_color_info)
                .def_readwrite("color_id",&NvOSD_RectParams::color_id);
	py::class_<NvDsComp_BboxInfo>(m,"NvDsComp_BboxInfo",py::module_local())
                .def(py::init<>())

                .def_readwrite("org_bbox_coords",&NvDsComp_BboxInfo::org_bbox_coords);

	 py::class_<NvBbox_Coords>(m,"NvBbox_Coords",py::module_local())
            .def(py::init<>())
            .def_readwrite("left",&NvBbox_Coords::left)
            .def_readwrite("top",&NvBbox_Coords::top)
            .def_readwrite("width",&NvBbox_Coords::width)
            .def_readwrite("height",&NvBbox_Coords::height);

Now using the built pyds_bbox_meta.so, you can get detector_bbox_info using the following code

obj_meta=pyds_bbox_meta.NvDsObjectMeta.cast(l_obj.data)
det_info_left = obj_meta.detector_bbox_info.org_bbox_coords.left
det_info_top = obj_meta.detector_bbox_info.org_bbox_coords.top

you can follow the similar procedure for creating and getting tracker bbox info as well.

1 Like
7

Hi,
I’m trying to do this but get the error

pyds_bbox_meta.cpp: In function ‘void pybind11_init_pyds_bbox_meta(pybind11::module_&)’:
pyds_bbox_meta.cpp:58:76: error: ‘array’ in namespace ‘py’ does not name a type
(NvDsObjectMeta &self)->py::array {

Also If I’ll be able to build this, what is it exactly that i need to do with pyds_bbox_meta.so? I’ve got some python code that runs, do i link it to something? Sorry if these are stupid questions, but new to deepstream.

UPDATE: I managed to build the above code, you need to also include

include <pybind11/numpy.h>

and it is missing a closing }

still not sure what to do with the .so

1 Like
9

Either add the parent folder of your .so to your PYTHONPATH, or if you edited the setup.py with the corresponding package_data, you can run pip install /path/to/setup_py/folder.

Then, you can use the providded snippet:

import pyds_bbox_meta
...  # add rest of callback code here
obj_meta=pyds_bbox_meta.NvDsObjectMeta.cast(l_obj.data)
det_info_left = obj_meta.detector_bbox_info.org_bbox_coords.left
det_info_top = obj_meta.detector_bbox_info.org_bbox_coords.top
2 Likes