Demuxer is corrupting object meta of secondary detector

• DeepStream Version 6.0

I am following this repo deepstream_reference_apps/back-to-back-detectors at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub and running secondary gie as secondary detector mode.

I have pipeline like this
source → h264parser → decoder → streammux → primary_detector → tracker → secondary_detector ->nvvidconv → nvosd → sink

there is a callback attached at the sink of nvvidconv.

I can able to process the object meta of a primary as well as a secondary detector. I am able to get parent meta from secondary object meta.

But when I change my pipeline to something like this (adding demuxer between nvvidconv and secondary detector). I know there is no need for demuxer here but in other application I have demuxer in between.

source → h264parser → decoder → streammux → primary_detector → tracker → secondary_detector → demux → nvvidconv → nvosd → sink

there is a callback attached at the sink of nvvidconv.

I can able to process the object meta of a primary as well as a secondary detector. But now I can’t get parent meta from secondary object meta.

For debugging, I attached callbacks on sink and src of demuxer. On sink callback, I can get parent meta from secondary object meta as expected but on src callback, I can’t get parent meta from secondary object meta.

What could be the problem here? Am I missing something here?

The demux will destroy the batch from streammux and create new batch for single frame. The parent of SGIE is not copied to the new batch meta in nvstreamdemux.

@Fiona.Chen this used to work in deepstream 5.0. How can I use demux and still have the parent of SGIE in the new batch meta in nvstreamdemux?

@Fiona.Chen Any update?

@dmitry.skorokhod Any suggestions?

sorry, I don’t know.
by the way, what do you call

?

@dmitry.skorokhod Object meta which has unique_component_id ==2.
I call below code to get parent id.
obj_meta->parent->object_id

To be honest, a lot of DS things have no explanation in official documentation and I don’t remember I saw anything about that parent thing i.e what it is, who builds it and how to use it (and I read a lot of DS docs and source code).
Combined with the fact DS isn’t open source and support is limited by Nvidia only, I won’t be surprised if the only answer you might get will be like this one.

@dmitry.skorokhod Thanks for your response. Maybe I have to go to other forums. So, how do you relate primary object meta and secondary object meta?

Do you know any? I don’t because it’s a closed system, that’s the problem.

Can you tell me what’s your goal in general?
Normally PGIE produces some object meta with its unique_id and SGIE processes only those objects and generates some additional meta if PGIE’s unique_id matches, that’s how I see it (again, I know nothing about that parent thing, that might be the key to employ here but one cannot do that without having some knowledge).

I am working on Automatic number plate recognition in which first I have to detect number plates as the primary detector and then detect and recognize characters in the plate as the secondary detector.

There could be multiple plates in the frame therefore characters can belong to multiple number plates.
So while processing object meta I should have information about which character belongs to which numbers plate.

I was talking about StackOverflow so that I could reach more people like you.

Ok. I’m not very familiar with ALPR but understand what you’re doing (and it’s a very popular topic with a lot of examples if you ask me).

Are you saying that the approach that worked with DS 5.0 no longer works with DS 6.0? Is the main issue that all meta is there but paren’t information is lost after demux?

I wasn’t successful in finding anything up there, the only place to get help is Nvidia forums and you see how it works.

@dmitry.skorokhod Yes, it used to work in deepstream 5.0 but not now. The problem arises only when I use demuxer in between my pipeline otherwise it is working properly.

you didn’t answer the second part of my question though…

@dmitry.skorokhod Yes, parent information is lost after demux.

@Fiona.Chen @bcao Any suggestions?

well, it the only change between DS 5.0 and 6.0 is that loss of parent info then I’d say it’s either a bug or an unannounced breaking change, both should be dealt with by Nvidia.

Parent is available before nvstreamdemux. Is there any special reason to get the parent information after nvstreamdemux?