Parent lost after nvvidconv element

Hi there, I’m running a deepstream pipeline in python and I have 2 gie, both are classifiers. Sgie runs only on one class of the first classifier. The important part of my pipeline looks like this. Input is a RTSP stream.

streammux.link(queue1)
queue1.link(pgie)
pgie.link(queue2)
queue2.link(tracker)
tracker.link(queue3)
queue3.link(sgie)
sgie.link(queue4)
queue4.link(tiler)
tiler.link(queue5)
queue5.link(nvvidconv)
nvvidconv.link(queue6)
queue6.link(nvosd)
nvosd.link(queue7)
queue7.link(nvvidconv_postosd)
nvvidconv_postosd.link(caps)

I have 2 probes, first in the src pad of the sgie and second on the src pad of the nvvidconv.

In the first probe I extract meta data from the sgie and look for its parent field on the object_meta to extract meta data from pgie.

The problem I have is that I can’t extract the frame array in that probe because at that moment the format is not supported by the function pyds.get_nvds_buf_surface().

I can extract the frame array from the second probe after nvvidconv, but at that moment the metadata from the sgie lost the parent.

Why the parent is lost after the nvvidconv? Is there any option to convert before the pgie? If I put a nvvidconverter before pgie do I need a caps filter too?

Thank you

nvmultistreamtiler will convert the batched videos into one single video, the old batch information is of no use for the new video.
Your may convert the format before nvmultistreamtiler.

I try to put the nvvidconverter before the tiler, like this:

queue1.link(pgie)
pgie.link(queue2)
queue2.link(tracker)
tracker.link(queue3)
queue3.link(sgie)
sgie.link(queue4)
queue4.link(nvvidconv)
nvvidconv.link(queue5)
queue5.link(tiler)
tiler.link(queue6)

And then I put my probe on the sink of the tiler:

tiler_sink_pad = tiler.get_static_pad("sink")
tiler_sink_pad.add_probe(Gst.PadProbeType.BUFFER, probe_callback, 0)

but the parent data is still missing.

@ACazaliG Can you tell us why do you need to query “parent” for the object meta? How did your two classifiers work together as PGIE + SGIE? The PGIE should be a detector to provide object meta.

PGIE and SGIE are both object detectors. SGIE works on a specific class from the SGIE so I need to relate the object from the SGIE to the object that tracker gives.

The nvvideoconvert does not support parent too. The requirement is added to our future roadmap.