gst_buffer_iterate_meta does not iterate all metadata from a gst-plugin

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hey,

I have a gst plugin that integrates with deepstream v3, it works perfectly when using only one stream and batch=1, but when we run multiple streams using the example in test3 app, my plugin only reads the metadata for one of the streams.

this is how I implemented it (which is pretty much a copy paste from the test3 app):

static GstFlowReturn gst_myplugin_transform_ip(
		GstBaseTransform * btrans, GstBuffer * inbuf) {
       
        // GstMeta is the struct that contains metadata
	GstMeta *gst_meta;
	GstNvStreamMeta *streammeta = NULL;
	int size = 0;
	// NvDsMeta is how the pgie nvInfer saves the metadata
	NvDsMeta *nv_meta;

	NvDsFrameMeta *frame_metadata = NULL;
	// NOTE: Initializing state to NULL is essential
	gpointer state = NULL;
        static GQuark _nvdsmeta_quark = 0;
	if (!_nvdsmeta_quark) {
		_nvdsmeta_quark = g_quark_from_static_string(NVDS_META_STRING);
	}

	// Standard way of iterating through buffer metadata
	// state = NULL, the fist metadata is returned
	// otherwise retrieve the next GstMeta after current
	streammeta = gst_buffer_get_nvstream_meta (inbuf);
	g_print(" starting buffer %d\n", streammeta->num_filled);
	while ((gst_meta = gst_buffer_iterate_meta (inbuf, &state))) {
  		// Check if this metadata is of NvDsMeta type
		if (gst_meta_api_type_has_tag(gst_meta->info->api, _nvdsmeta_quark)) {
			
			nv_meta = (NvDsMeta *) gst_meta;
			// Check if the metadata of NvDsMeta contains object bounding boxes
			if (nv_meta->meta_type == NVDS_META_FRAME_INFO) {
				frame_metadata = (NvDsFrameMeta *) nv_meta->meta_data;
				if (frame_metadata == NULL) {
					g_print ("WARN NvDS Meta contained NULL meta \n");
					return GST_PAD_PROBE_OK;
				}
				frame_metadata->num_strings = 0;
				size += frame_metadata->num_rects;
				// Check if these parameters have been set by the primary detector/tracker
				
				FSBoundingBox *bboxes = (FSBoundingBox *) g_malloc0(
						sizeof(FSBoundingBox) * frame_metadata->num_rects);

				// Iterate through all the objects
				for (guint i = 0; i < frame_metadata->num_rects; i++) {
					NvDsObjectParams *obj_param = &frame_metadata->obj_params[i];
                                ///my code
...
..
.

I can see how the same code works and iterates through all of the streams when is executed from a probe function attached to the inference pad within the same app, but not when called from transform_ip function in a gst-plugin.

What can I do to make it work from the gst-plugin?

Many thanks for the help!

jl

nvgst-util.c (3.87 KB)

2

but not when called from transform_ip function in a gst-plugin
What’s your pipeline?
For multiple streams, do you iterate through all of streams but no NvDsFrameMeta, or iterate only one stream with only one NvDsFrameMeta?

3

The pipeline is pretty much the same as the test app 3 from deepstream,

if (!gst_element_link_many (streammux, pgie, tiler, nvvidconv, myplugin, nvosd, 
      nvvidconv2, filter3, videoconvert1, filter4, x264enc, mpegtsmux, filesink, 
      NULL)) {

I’m sure the pipeline goes through all of the streams because I can see the multiplexed output with all the boxes in each video.

4

The deepstream release has example plugin “sources/gst-plugins/gst-dsexample”
Can you add this plugin in your pipeline and check the NvDsFrameMeta in gst_dsexample_transform_ip() ?

5

the tiler plugin will change the meta data in the pipeline because it will merge all frames into one, not sure…

you can move your custom plugin between pgie and tiler to see the result.

6

Yes. Zhouzhi

Hi Juan
What’s your “myplugin” functions? Why do you add it after tiler?

7

Thanks for your responses,

I don’t need the specific output from the tiler, all I care about is the bounding box and the stream id (and in the future the tracking id).

I changed the pipeline to

if (!gst_element_link_many (streammux, pgie, myplugin, tiler, nvvidconv, nvosd,
nvvidconv2, filter3, videoconvert1, filter4, x264enc, mpegtsmux, filesink,
NULL)) {

and I get an error:

Running...
ERROR from element h264parse0: Internal data stream error.
Error details: gstbaseparse.c(3611): gst_base_parse_loop (): /GstPipeline:dstest5-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin/GstDecodeBin:decodebin0/GstH264Parse:h264parse0:
streaming stopped, reason not-negotiated (-4)
Returned, stopping playback

This error may be something basic about GStreamer; I’m still new to it, so I have not figured it out yet.

Thanks for your help.

Juan Luis

8

Hi and good time
Hope you are feeling well
I wana use nvivafilter and nvsample for a processing application
In the sample you have wrote a funcation named
static void add_metadata(void ** usrptr)

And in the iva_metadata.h file is defined some metadata varible

typedef struct _location_t Location;

struct _location_t
{
int x1, y1;
int x2, y2;
};

typedef struct _bbox_t BBOX;

struct _bbox_t {
unsigned int framecnt;
unsigned int objectcnt;
Location location_list[NUM_LOCATIONS];
};

Right now,i wanna know how i can use the metadata in the gstreamer Main looper
For example ,how can i get the metadata in main gsteramer loop launcher??
May you help me??
Thank you for your time
Appreciate that
Best regards.

9

Hello @hhami.2040 ,
I’m going to close this topic as it has lasted too long and people initially involved may not be able to come back to this thread. There were several DeepStream new versions released since this topic opened, suggest retesting on the latest DeepStream version, please open a new topic to discuss in case this is still an issue, thanks.