Pipeline stops with tee and queue

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

• Hardware Platform (Jetson / GPU) - Jetson
• DeepStream Version - 6.0
• JetPack Version (valid for Jetson only) - 4.6
• TensorRT Version - 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) - questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Deepstream version - 6.0

Deepstream pipeline worked before adding the tee and queue plugins. After creating two branches with tee and queue, pipeline stops running. Attached the code and debug for reference. It says queue is empty in the debug, however there is no error though.

DEBUG:

0:01:00.669725446 17590 0x7eb00028f0 DEBUG GST_BUFFER gstbuffer.c:2384:gst_buffer_foreach_meta: remove metadata 0xe261b68 (NvDsMeta)
0:01:00.669856282 17590 0x7eb00028f0 DEBUG GST_BUFFER gstbuffer.c:1375:gst_buffer_is_memory_range_writable: idx 0, length -1
0:01:00.669933263 17590 0x7eb00028f0 DEBUG nveglglessink gsteglglessink.c:740:render_thread_func: Successfully handled object
0:01:00.670090610 17590 0x7eb0002b20 DEBUG nveglglessink gsteglglessink.c:1265:gst_eglglessink_queue_object: Object handled: ok
0:01:00.670136184 17590 0x7eb0002b20 DEBUG basesink gstbasesink.c:3577:gst_base_sink_chain_unlocked: object unref after render 0x7f100061d0
0:01:00.670181080 17590 0x7eb0002b20 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:nvvideo-renderer:sink called chainfunction &gst_base_sink_chain with buffer 0x7f100061d0, returned ok
0:01:00.670227123 17590 0x7eb0002b20 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:nvegl-transform:sink called chainfunction &gst_base_transform_chain with buffer 0xdfbea10, returned ok
0:01:00.670270822 17590 0x7eb0002b20 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:onscreendisplay:sink called chainfunction &gst_base_transform_chain with buffer 0xdfbea10, returned ok
0:01:00.670333323 17590 0x7eb0002b20 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:convertor:sink called chainfunction &gst_base_transform_chain with buffer 0xdfbe5d0, returned ok
0:01:00.670378480 17590 0x7eb0002b20 DEBUG queue_dataflow gstqueue.c:1516:gst_queue_loop: queue is empty
0:01:01.554548926 17590 0x7eb00029e0 DEBUG GST_PADS gstpad.c:3597:probe_hook_marshal:nvapp-sink:sink probe returned OK
0:01:01.554650386 17590 0x7eb00029e0 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:nvapp-sink:sink calling chainfunction &gst_base_sink_chain with buffer buffer: 0xdfbe4c0, pts 0:00:00.033333333, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:01:01.554730128 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3421:gst_base_sink_chain_unlocked: got times start: 0:00:00.033333333, end: 99:99:99.999999999
0:01:01.554783983 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2006:gst_base_sink_get_sync_times: got times start: 0:00:00.033333333, stop: 99:99:99.999999999, do_sync 1
0:01:01.554812473 17590 0x7eb00029e0 DEBUG default gstsegment.c:737:gst_segment_to_running_time_full: invalid position (-1)
0:01:01.554844297 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2591:gst_base_sink_do_sync: reset rc_time to time 0:00:00.033333333
0:01:01.554874714 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2603:gst_base_sink_do_sync: possibly waiting for clock to reach 0:00:00.033333333, adjusted 0:00:00.033333333
0:01:01.554898829 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2223:gst_base_sink_wait_clock: sync disabled
0:01:01.554925965 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2610:gst_base_sink_do_sync: clock returned 4, jitter 0:00:00.000000000
0:01:01.554945861 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3538:gst_base_sink_chain_unlocked: rendering object 0xdfbe4c0
0:01:01.554965913 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:994:gst_base_sink_set_last_buffer_unlocked: setting last buffer to 0xdfbe4c0
0:01:01.554994195 17590 0x7eb00029e0 DEBUG appsink gstappsink.c:876:gst_app_sink_render_common: pushing render buffer/list 0xdfbe4c0 on queue (1)
0:01:01.555036331 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3577:gst_base_sink_chain_unlocked: object unref after render 0xdfbe4c0
0:01:01.555063676 17590 0x7eb00029e0 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:nvapp-sink:sink called chainfunction &gst_base_sink_chain with buffer 0xdfbe4c0, returned ok
0:01:01.555112479 17590 0x7eb00029e0 DEBUG GST_PADS gstpad.c:3810:do_probe_callbacks:nvapp-sink:sink data is passed
FPS*******************
Fps of stream 0 is 1.0
0:01:02.567993600 17590 0x7eb00029e0 DEBUG GST_PADS gstpad.c:3597:probe_hook_marshal:nvapp-sink:sink probe returned OK
0:01:02.568085217 17590 0x7eb00029e0 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:nvapp-sink:sink calling chainfunction &gst_base_sink_chain with buffer buffer: 0xdfbe6e0, pts 0:00:00.066666666, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:01:02.568136364 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3421:gst_base_sink_chain_unlocked: got times start: 0:00:00.066666666, end: 99:99:99.999999999
0:01:02.568172927 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2006:gst_base_sink_get_sync_times: got times start: 0:00:00.066666666, stop: 99:99:99.999999999, do_sync 1
0:01:02.568201209 17590 0x7eb00029e0 DEBUG default gstsegment.c:737:gst_segment_to_running_time_full: invalid position (-1)
0:01:02.568230376 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2591:gst_base_sink_do_sync: reset rc_time to time 0:00:00.066666666
0:01:02.568259387 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2603:gst_base_sink_do_sync: possibly waiting for clock to reach 0:00:00.066666666, adjusted 0:00:00.066666666
0:01:02.568278606 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2223:gst_base_sink_wait_clock: sync disabled
0:01:02.568305482 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2610:gst_base_sink_do_sync: clock returned 4, jitter 0:00:00.000000000
0:01:02.568325742 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3538:gst_base_sink_chain_unlocked: rendering object 0xdfbe6e0
0:01:02.568346420 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:994:gst_base_sink_set_last_buffer_unlocked: setting last buffer to 0xdfbe6e0
0:01:02.568372202 17590 0x7eb00029e0 DEBUG appsink gstappsink.c:876:gst_app_sink_render_common: pushing render buffer/list 0xdfbe6e0 on queue (2)
0:01:02.568410744 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3577:gst_base_sink_chain_unlocked: object unref after render 0xdfbe6e0
0:01:02.568437724 17590 0x7eb00029e0 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:nvapp-sink:sink called chainfunction &gst_base_sink_chain with buffer 0xdfbe6e0, returned ok
0:01:02.568492829 17590 0x7eb00029e0 DEBUG GST_PADS gstpad.c:3810:do_probe_callbacks:nvapp-sink:sink data is passed
0:01:03.584665108 17590 0x7eb00029e0 DEBUG GST_PADS gstpad.c:3597:probe_hook_marshal:nvapp-sink:sink probe returned OK
0:01:03.584840424 17590 0x7eb00029e0 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:nvapp-sink:sink calling chainfunction &gst_base_sink_chain with buffer buffer: 0xdfbe5d0, pts 0:00:00.099999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 64, offset none, offset_end none, flags 0x0
0:01:03.584915946 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3421:gst_base_sink_chain_unlocked: got times start: 0:00:00.099999999, end: 99:99:99.999999999
0:01:03.584979437 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2006:gst_base_sink_get_sync_times: got times start: 0:00:00.099999999, stop: 99:99:99.999999999, do_sync 1
0:01:03.585045323 17590 0x7eb00029e0 DEBUG default gstsegment.c:737:gst_segment_to_running_time_full: invalid position (-1)
0:01:03.585107981 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2591:gst_base_sink_do_sync: reset rc_time to time 0:00:00.099999999
0:01:03.585163972 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2603:gst_base_sink_do_sync: possibly waiting for clock to reach 0:00:00.099999999, adjusted 0:00:00.099999999
0:01:03.585205691 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2223:gst_base_sink_wait_clock: sync disabled
0:01:03.585257255 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:2610:gst_base_sink_do_sync: clock returned 4, jitter 0:00:00.000000000
0:01:03.585298714 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3538:gst_base_sink_chain_unlocked: rendering object 0xdfbe5d0
0:01:03.585348142 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:994:gst_base_sink_set_last_buffer_unlocked: setting last buffer to 0xdfbe5d0
0:01:03.585443352 17590 0x7eb00029e0 DEBUG appsink gstappsink.c:876:gst_app_sink_render_common: pushing render buffer/list 0xdfbe5d0 on queue (3)
0:01:03.585531427 17590 0x7eb00029e0 DEBUG basesink gstbasesink.c:3577:gst_base_sink_chain_unlocked: object unref after render 0xdfbe5d0
0:01:03.585592990 17590 0x7eb00029e0 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:nvapp-sink:sink called chainfunction &gst_base_sink_chain with buffer 0xdfbe5d0, returned o

Attached pipeline code as well. Kindly help fix this.
pipeline.py (9.5 KB)

3

Hi Team,

Any lead on this? Thanks.

4

which deepstream sample is your code based on? what is your media pipeline? please narrow down this issue, which code will cause stop after adding queue and tee?

5

It’s based on deepstream-test2 app. The source is nvarguscamera. I’ve already attached the pipeline code that has issue. I just want to check if there is any issue with the pipeline. Please have a look at the pipeline and let me know if you need further details. Thanks.

6

branch1 is used to display, and branch2 is used to appsink, please simplify the code to check, check if no tee + branch 1 is ok, check if tee + branch1 is ok, check if tee + branch2 is ok.

7

Thanks, I will try this and let you know.
However I have a question. The reason I use appsink in the second branch is to do some compute intensive operations in the appsink callback with real time frame rendering on the first display branch. Will this approach work?

8

@fanzh , Can you pls help me understand if delegating compute intensive operations on appsink in a different branch help achieve real time display of frames on another branch?

9

Team, Any clarification on the query is appreciated.

10
  1. about " help achieve real time display of frames on another branch?", what does that mean?
  2. please check the costing time in appsink if do some compute intensive operations in the appsink.
  3. to narrow down this issue, you can do noting in aapsink to check if it will stop.
11
  1. about " help achieve real time display of frames on another branch?", what does that mean? -

My question is, will the two branches run independent on one another?

12

no, by the tee, two branches will process one buffer by turn, if one blocks, the other will wait. please refer to tee: GStreamer Core Plugins 1.0 Plugins Reference Manual

1 Like
13

Thanks for the clarification.

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.