How to read deepstream metadata after nvstreamdemux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

•Hardware Platform (Jetson / GPU): GTX2080
•DeepStream Version 5 (deepstream:5.0.1-20.09-samples)
•NVIDIA GPU Driver Version 460.56
•Issue Type( questions, new requirements, bugs) question

Hi. I’m running a pipeline inside the deepstream:5.0.1-20.09-samples docker container. The pipeline roughly does the following (omitting converts, encoders/decoders, and other irrelevant stuff)

src1 ->  nvstreammuxmux ->(nvinfer, others) -> nvstreamdemux -> [block1] -> ...
src1 -/                                                       \-[block2] -> ...

I can access deepstream metadata before the nvstreamdemux using the pyds.gst_buffer_get_nvds_batch_meta, but obviously that works in a probe before the demuxer.

Q: Is it possible to access deepstream metadata after the nvstreamdemux element?

Notes:

  • the demux element mentions here:

    Output

    • Gst Buffer (non-batched, single source)
    • Meta related to each Gst Buffer source

    What is / how can I access “Meta related to each Gst Buffer source” (ideally from python)? In the past, I’ve been able to workaround missing bindings (eg here and here ), but I do not know whether this is the case or not.

  • according to this diagram, I think I’m looking for NvDsFrameMeta

  • here I can see a gst_buffer_get_nvds_meta, but cannot find it in the pyds bindings…

3

Hi @pwoolvett ,
Could refer to deepstream_python_apps/deepstream_imagedata-multistream.py at 46bfd6b89c618f4c41d01d2d3e8b1d8d9628ef0b · NVIDIA-AI-IOT/deepstream_python_apps · GitHub .
Add a probe on nvstreammux and get the meta in the probe function

Thanks!

4

Hi @mchi,

Thank yout for your response.

I’ve been able to read metadata in the probe from the muxer, I’ve no issues with that.

The gist of the problem is I’m doing some metadata postprocesing offline, separately on each branch of the demux.

What I’m looking for is deepstream’s frame_num after the demux element.

I could add a counter in another buffer probe, but frame drops would make sync a bigger problem…

5

My mistake, will check and get back.

Thanks!

6

Found the problem.

The metadata reading works fine soon after the demux. Regardless of its “de-batch”, the same pyds functions can be used and work fine.

The probe callback I had previously was attached to a vp8enc, which lost deepstream metadata and produced the issue. Found this out by creating two buffer probes, one at a sinkpad and one at a sourcepad, and moving the buffer probes downstream until the vp8enc, where the sinkpad worked and the sourcepad failed.

1 Like