Questions re: differences between RTSP Source in deepstream_source_bin.c & deepstream_test_sr_app.c

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jestson
• DeepStream Version
6.0.1
• JetPack Version (valid for Jetson only)
4.6.3
• TensorRT Version
8.2.1-1+cuda10.2
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
Questions

Looking at the RTSP Source bin implementation in deepstream_source_bin.c I see the following.
First, in the create_rtsp_src_bin function we have pre-decode tee linked to the decode queue which is linked to the decoder. The tee is also optionally linked to the recordbin

  link_element_to_tee_src_pad(bin->tee_rtsp_pre_decode, bin->dec_que);
  NVGSTDS_LINK_ELEMENT (bin->dec_que, bin->decodebin);
  if (ctx)
    link_element_to_tee_src_pad(bin->tee_rtsp_pre_decode, ctx->recordbin);

In the select-stream callback, the depay is linked to the parser which is linked to the pre-decode tee

    NVGSTDS_LINK_ELEMENT (bin->depay, bin->parser);
    NVGSTDS_LINK_ELEMENT (bin->parser, bin->tee_rtsp_pre_decode);

And in the pad-added callback the rtspsrc element is linked to the depay

    GstPad *sinkpad = gst_element_get_static_pad (bin->depay, "sink");
    if (gst_pad_link (pad, sinkpad) != GST_PAD_LINK_OK) {

This gives us

                             |->queue->decoder->
  rtcpsrc->depay->parser->tee
                             |->recordbin

This is exactly what is show in the diagram here

Looking at the Smart Recording example, in the main function we have the depay linked to the pre-decode tee which is linked to the queue which is linked to the decoder

Surprisingly, there is no parser between the depay and the decoder!


  /* Link the elements together till decoder */
  if (!gst_element_link_many (depay_pre_decode, tee_pre_decode,
          queue_pre_decode, decoder, NULL)) {

Then in the pad-added callback, the rtspsrc is linked to the depay and then the pre-decode tee is linked to the parser which is linked to the recordbin

    GstPad *sinkpad = gst_element_get_static_pad (depay_elem, "sink");
    if (gst_pad_link (element_src_pad, sinkpad) != GST_PAD_LINK_OK) {
   ...
    GstElement *parser_pre_recordbin =
        gst_element_factory_make ("h264parse", "parser-pre-recordbin");

      if (!gst_element_link_many (tee_pre_decode, parser_pre_recordbin,
              nvdssrCtx->recordbin, NULL)) {

This gives us

                     |->queue->decoder->
  rtcpsrc->depay->tee
                     |->parser->recordbin

The approach used in the common RTSP source bin in deepstream_source_bin.c seems correct to me, and used to work… however, I’m only able to get the recordbin to link if I follow the approach in the Smart-Recording example. This seems to have change when audio support was added??

Can anyone explain why?

Thanks,
Robert.

3

We can decode the h264 data after the depay plugin. When add audio support, we cannot add parse plugin before tee.
What exactly do you mean by the following? Have you encountered any practical problems?

4

@yuweiw thanks for the quick response.

re: We can decode the h264 data after the depay plugin
I assume this is true when using the rtph265depay and rtpjpegdepay plugins as well?

re: Have you encountered any practical problems?
Yes, my implementation (which is clearly out of date) was based on the RTSP source bin implementation in the common apps file deepstream_source_bin.c where the parse plugin is before the tee.

re: When add audio support, we cannot add parse plugin before tee.
NVIDIA might want to consider updating deepstream_source_bin.c as I’m assuming it will fail if someone tries to use it for smart-recording? (this is DS 6.0.1)

Also, the smart record diagram under the DS 6.2 documentation still shows the parse plugin before the tee
Something else to consider updating

image
image1893×673 15.9 KB

Thanks again,
Robert

5

For deepstream-app, we cannot support audio record at present. You can refer to the cb_rtspsrc_select_stream. The documentation shows the deepstream-app record diagram, not the deepstream-testsr app. The deepstream-testsr app is just a reference for customers to develop their own code. They can make any expansion they want on this basis.

6

@yuweiw I guess I’m still a little confused. For my setup, my cameras do NOT support audio… yet, trying to link the parse plugin before the Tee as show in the diagram will fail. I’m assuming the deepstream-app will fail in the same way.

It sounds like you are saying that the diagram and deepstream-app should work for video only?

7

Yes. At present, we have not implemented the audio function in deepstream app

Could you attach the fail log with GST_DEBUG=3?

8

@yuweiw My apologies for the tardy response… but I really wanted to make sure I understood what I was doing and it looks like I misspoke re: issue.

If I add the recordbin to my source bin prior to setting the Pipeline state to PLAYING, then I can link the parser pre-decode-tee as follows. This is how the DeepStream App common source-bin works.

                             |->queue->decoder->
  rtcpsrc->depay->parser->tee
                             |->recordbin

But if I create the recordbin in the select-stream callback after setting the Pipeline state to Playing then it fails to link as above.

In this case, the parser must be linked after the tee directly to the recordbin as follows

                     |->queue->decoder->
  rtcpsrc->depay->tee
                     |->parser->recordbin

This is how it is done in the smart-record example.

So both the DeepStream App and Smart-record samples work.

The take away here is… if you’re going to create and link the recordbin in the “select-stream” or “pad-added” callbacks then you need follow the smart-record example with the parser after the tee connected to the recordbin only.

Thanks,
Robert.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.