Add data to the pipelined

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

In the DeepStream 6.3 package, there’s a file named deepstream-test3 that, according to the official documentation, is designed for multi-stream processing. Inside this folder, there’s a file named deepstream_test3_app.c, which contains a function called tiler_src_pad_buffer_probe. This function includes code for displaying text output, but this part of the code is commented out. After uncommenting it and recompiling, the text still does not display.

In the deepstream-test1 folder, there is a file named deepstream_test1_app.c, which contains a function called osd_sink_pad_buffer_probe. In this case, the text successfully displays in the final video output.

I also experimented with adding data in the tile_sink_pad_buffer_probe function in other programs, and the text displayed correctly in the final video.

I would like to know if data can only be transmitted through the pipeline if it is added at the sink, and if adding data at the src does not allow it to be transmitted through the pipeline. If this is the case, could you please explain why?

3 
            ┌──────────────────┐                            
            │                  │                            
            │                  │                            
    sink    │                  │   src                      
 ──────────►│     osd plugin   ├────────────►               
            │                  │                            
            │                  │                            
            │                  │                            
            └──────────────────┘

This is a feature of gstreamer, Just like the figure above. The plugin can only process data input from the sink pad

4

In test3, the tiler_src_pad_buffer_probe is associated with the tiler’s output pad. Why is the code for displaying text commented out in this example? Since there is also an OSD element after the tiler, it should be possible to display text at the OSD’s sink pad. However, when I uncomment the text display code in test3, the text still does not appear in the final output. Could you please explain why?

5

Not so.

  1. The tiler_src_pad_buffer_probe function belongs to pgie’s src pad. When the tiler merges multiple streams, some metadata needs to be dropped.

  2. We usually recommend operating the display metadata on the osd sink pad.

6

I want to add some data at the beginning of the pipeline and then extract it after it has passed through some elements in the pipeline. In this case, is it better to add the data at the sink pad of an element, or can it be added at any pad?

7

This approach is better.

8

If possible, could you please explain the reason?

9

As I said above, some elements may need to drop certain members when copying or processing metadata.

In other words, in the pipeline, metadata is not immutable.

So only add data on the sink pad of the element that needs to process metadata is recommended.

10

My pipeline structure has a tiler element followed by two queue elements and an nvvidconv element before reaching the OSD element. When I add data at the output of the tiler and OSD, the data is successfully displayed in the final output. However, when I add data at the input of the tiler, it seems to be filtered out. But when I tried adding data at the input of the OSD, the data still didn’t display in the final output. Why is this happening? Since adding data at the output of the tiler results in it being displayed, it means that at least the OSD is not filtering out the data. But why is the data not being displayed when I add it at the input of the OSD?
I look forward to your response.

11

Can you share sample code that can reproduce the problem? I can’t find the problem from the description above.

12

In the deepstream_faciallandmark_app file, lines 334 to 501 contain the input data for the OSD. In this function, I added data between lines 359 and 374, and the data was successfully displayed as ‘9’. Lines 654 to 940 contain the OSD’s sink pad, and in this function, I modified the code at line 917 to display ‘hello’ in the final output. Lines 945 to 1103 contain the tiler’s sink pad, and I added some code between lines 1084 and 1098 that also allows ‘hello’ to be displayed in the final output. The final display shows two ‘hello’ messages as expected, but the number ‘9’ is not displayed.

The pipeline connection sequence is located on line 1816. Could you please explain why adding data at the output of the nvtile element results in it being displayed, while adding data at the input of the nvosd element does not?
deepstream_faciallandmark_app.txt (85.1 KB)

13

This is normal, this is your pipeline.

  if (!gst_element_link_many (streammux, queue1, primary_detector_1, primary_detector, queue2, 
        second_detector, queue3, nvtile, queue4, nvvidconv, queue5,                   
        nvosd, NULL)) {
    g_printerr ("Inferring and tracking elements link failure.11111\n");
    return -1;
  }

...... --- nvtiler ---> queue5 --> nvosd -->
        |                       |         |
        |                       |         |
     tiler sink              osd sink   osd src

OSD can only process data added before the sink pad

15

deepstream_test3_app.txt (20.5 KB)
In the deepstream_test3_app file, line 85 corresponds to the tiler’s src pad. Why is it that adding data at this point does not propagate further down the pipeline? When adding data at the tiler’s sink pad, the data is displayed in the final output. Could you please explain this? The original code from the official file had the display code here commented out. Was it commented out because it was found that the data could not be displayed?

16

tiler_src_pad_buffer_probe is attached to the source pad of pgie, not the tiler.

  tiler_src_pad = gst_element_get_static_pad (pgie, "src");
  if (!tiler_src_pad)
    g_print ("Unable to get src pad\n");
  else
    gst_pad_add_probe (tiler_src_pad, GST_PAD_PROBE_TYPE_BUFFER,
        tiler_src_pad_buffer_probe, NULL, NULL);
17

I want to add metadata to a certain element in the pipeline and use it in subsequent elements. Can you tell me a useful method? If you could provide some simple examples, that would be even better. Thank you very much for your answer.

18

If you mean user-defined meta, please refer to /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-gst-metadata-test

19

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

20

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.