Getting user_meta through pyds

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I need to pass two values calculated in a custom-plugin I’ve made and fetch them with a sink pad buffer probe.

Currently I’m modifying the gst-dsdirection plugin and saving a user_meta to frame_meta. The output is the same DsDirectionOutput as in the gst-dsdirection example.

static void
attach_metadata_frame (GstDsDirection * dsdirection, NvDsFrameMeta * frame_meta,
    DsDirectionOutput * output)
{

  NvDsBatchMeta *batch_meta = frame_meta->base_meta.batch_meta;

  // Attach - DsDirection MetaData
  NvDsUserMeta *user_meta = nvds_acquire_user_meta_from_pool (batch_meta);
  NvDsMetaType user_meta_type = NVDS_DIRECTION_USER_META;

  user_meta->user_meta_data = output;
  user_meta->base_meta.meta_type = user_meta_type;
  user_meta->base_meta.copy_func = copy_ds_direction_meta;
  user_meta->base_meta.release_func = release_ds_direction_meta;

  nvds_add_user_meta_to_frame (frame_meta, user_meta);

}

In the python application I’m fetching it like this:

batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None:
    try:
        # Note that l_frame.data needs a cast to pyds.NvDsFrameMeta
        # The casting is done by pyds.glist_get_nvds_frame_meta()
        # The casting also keeps ownership of the underlying memory
        # in the C code, so the Python garbage collector will leave
        # it alone.
        #frame_meta = pyds.glist_get_nvds_frame_meta(l_frame.data)
        frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
    except StopIteration:
        break

    l_user = frame_meta.frame_user_meta_list
    print(l_user)
    user_meta = pyds.NvDsUserMeta.cast(l_user.data)

    print('user_meta:', user_meta)
    print('user_meta.user_meta_data:', user_meta.user_meta_data)
    print('user_meta.base_meta:', user_meta.base_meta)

The output from this is:

...
<pyds.GList object at 0x7f2d8fc10030>
user_meta: <pyds.NvDsUserMeta object at 0x7f2d8fc10ab0>
user_meta.user_meta_data: <capsule object NULL at 0x7f2d8fc6a300>
user_meta.base_meta: <pyds.NvDsBaseMeta object at 0x7f2d8fc100a0>
...

Any idea how to properly add it to the frame_meta and/or fetch the values with the python bindings? Should I maybe save this as a different object structure?

Thanks

3

Hey, so you are trying to add user meta data to the frame_meta using python app, right?

4

I’m saving this

typedef struct
{
  float flowx;
  float flowy;
  char direction[MAX_LABEL_SIZE];
} DsDirectionObject;

structure to the frame_meta via

nvds_add_user_meta_to_frame (frame_meta, user_meta);

But the main thing is that I’m having trouble fetching the values with a python app (probe).

I’m wondering what is the best way to do this?
I just need to save two floats in my custom plug-in to each frame, and then be able to fetch it in my python app.

5

Got it, you need to add your own python bindings to access your own c data structure. You can refer GitHub - mrtj/pyds_tracker_meta: pybind11 wrapper to access Nvidia DeepStream tracker meta info from Python. for how to add your own binding

6

Great, thanks!

7

Still having trouble with this.
I’m saving this structure

typedef struct
{
  float flowx;
  float flowy;
} DsDirectionObject2;

with this function

static void
attach_metadata_frame (GstDsDirection * dsdirection, NvDsFrameMeta * frame_meta,
    DsDirectionObject2 * output, NvDsBatchMeta * batch_meta)
{

  // Attach - DsDirection MetaData
  NvDsUserMeta *user_meta = nvds_acquire_user_meta_from_pool (batch_meta);
  NvDsMetaType user_meta_type = NVDS_DIRECTION_USER_META;

  //output_object  = output->object;

  std::cout << "meanx :  " << output->flowx << std::endl;
  std::cout << "meany :  " << output->flowy << std::endl;
  std::cout << "user_meta_type :  " << user_meta_type << std::endl;

  user_meta->user_meta_data = output;
  user_meta->base_meta.meta_type = user_meta_type;
  user_meta->base_meta.copy_func = copy_ds_direction_meta;
  user_meta->base_meta.release_func = release_ds_direction_meta;

  nvds_add_user_meta_to_frame (frame_meta, user_meta);

}

which prints, e.g.:

meanx :  30.3153
meany :  -8.39673
user_meta_type :  10773

However I’m not able to cast it successfully to python with this (added in pyds_tracker_meta.cpp)

m.def("DsDirectionObject2_cast",
    [](void *obj) { return (DsDirectionObject2*)(obj); },
    "Cast given object/data to DsDirectionObject2",
    py::arg("obj"),
    py::return_value_policy::reference);

I always get the same error when I try to print the type and cast it to python

    while user_meta_list is not None:
        user_meta = pyds.NvDsUserMeta.cast(user_meta_list.data)
    
        try:
            print('user_meta: ', user_meta.base_meta.meta_type)
            if str(user_meta.base_meta.meta_type) != 'NvDsMetaType.NVDS_OPTICAL_FLOW_META':
                dsdir = pyds_tracker_meta.DsDirectionObject2_cast(user_meta.user_meta_data)
                print('dsdir: ', dsdir)
        except Exception as e:
            print(e)

        user_meta_list = user_meta_list.next

Then the output is:

user_meta:  NvDsMetaType.NVDS_OPTICAL_FLOW_META
user_meta:  NvDsMetaType.???
Unable to convert function return value to a Python type! The signature was
	(obj: capsule) -> DsDirectionObject2

I hope you can me with this
All the best,

8

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Oh, seems you also need to add binding function for your user meta type, it’s NVDS_DIRECTION_USER_META you defined, right?