Cannot get NvDsBatchMeta on sink

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I am trying to get the nvinfer meta data but gst_buffer_get_nvds_batch_meta always returns null;
Here is my pipeline uridecodebin → nvinfer → nvvideoconvert → appsink

What I tried.

NvDsBatchMeta *batch_meta = gst_buffer_get_nvds_batch_meta (buf);
if (!batch_meta) {
    // this is always true
}

But this always returns null.

Also tried

for (l_frame = batch_meta->frame_meta_list; l_frame != NULL;
      l_frame = l_frame->next) {
 // this gives segfault, because batch_meta is null
}

I just want to get the metadata that nvinfer added i.e bounding boxes, classes etc.

Background.

I am trying to separate the pipeline (reason being the user will choose which plugins to run) . So I will have something like:

decode stream
uridecodebin → appsink // then pass frame data to next pipeline

run inference
appsrc-> nvinfer → appsink then pass frame data to another application

Below is the config file used:
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-file=…/…/…/…/samples/models/Primary_Detector/resnet10.caffemodel
proto-file=…/…/…/…/samples/models/Primary_Detector/resnet10.prototxt
labelfile-path=…/…/…/…/samples/models/Primary_Detector/labels.txt
int8-calib-file=…/…/…/…/samples/models/Primary_Detector/cal_trt.bin
batch-size=1
network-mode=1
num-detected-classes=4
interval=0
gie-unique-id=1
output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid

[class-attrs-all]
threshold=0
eps=0.2
group-threshold=1

2

I hit a similar problem. In my case, I was not using appsink, but rather a custom element downsteam of nvvideoconvert, and there was still some meta, but seems the nvvideoconvert was stripping the tensors that I needed.

Anyway, try a probe function that’s upstream of nvvideoconvert.

3

I removed nvvideoconvert added a queue but the problem still persists. uridecodebin → nvinfer → queue-> appsink

4

Try adding nvstreammux, before nvinfer.

5

Are you sure that nvvideoconvert removes the meta data. I dont understand what I am doing wrong.

6

In my case, it was removing the tensors, because those were in GPU memory and my element (at the time) didn’t have the (memory:NVMM) feature on its upstream caps. However, there was still NvDsBatchMeta. So, it’s not exactly your case.

Anyway, did you try adding nvstreammux? I had that, and the docs say that what initially creates the NvDsBatchMeta.

7

See: [url]NVIDIA Metropolis Documentation

It says: “The basic metadata structure NvDsBatchMeta starts with batch level metadata, created inside the Gst-nvstreammux plugin. Subsidiary metadata structures hold frame, object, classifier, and label data.”

More on nvstreammux, here: [url]NVIDIA Metropolis Documentation

8

BTW, in case you didn’t know, you’ll have to get a request pad on nvstreammux, with a name like “sink_0”.

And the trailing number is how you’ll identify that stream in the resulting meta.

9

It works with nvstreammux.
nvinfer does indeed require nvstreammux.