When remove infer engine dynamically , ds-app can`t push rtmp stream correctly

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.1
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.2.5-1+cuda11.4
• NVIDIA GPU Driver Version (valid for GPU only) 510.68.02
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
When pause the pipeline, the origin pipeline like:


i want infer engine be removed from the pipeline,

so it can push origin stream to the rtmp server. I refer to gstreamer pipeline manipulation doc to delete the infer engine dynamically . Link streammux and streamdemux, and unlink the element they link before. But the rtmp server can`t receive data from the pipeline. (I use tcpdump to catch the datapack) It looks the data be blocked by some element. But it shows normally through the pipeline pic. All the elements are in playing state in the pipeline. There are no block pads also.

Here is the changed pipeline pic:

I also save the debug log:
1.tar.gz (38.8 MB)

Here are some key code:
static GstPadProbeReturn
event_probe_cb(GstPad * pad, GstPadProbeInfo * info, gpointer user_data)
AppCtx appCtx = (AppCtx)user_data;
GstPad *last_ele_src_pad = NULL;
char pad_name[15];
gboolean pad_status;
NVGSTDS_INFO_MSG_V(" Get not eos event ");

GstState cur;
GstState pending;

gst_pad_remove_probe(pad, GST_PAD_PROBE_INFO_ID(info));
gchar* name = gst_element_get_name(appCtx->first_elem);
NVGSTDS_INFO_MSG_V(" first_elem name %s ",name);
gst_element_unlink(appCtx->pipeline.multi_src_bin.bin, appCtx->first_elem);	
NVGSTDS_INFO_MSG_V("src_bin first_elem unlink ");
gst_element_unlink(appCtx->last_elem, appCtx->pipeline.demuxer);
NVGSTDS_INFO_MSG_V("last_elem demux unlink ");
gchar* last_name = gst_element_get_name(appCtx->last_elem);
NVGSTDS_INFO_MSG_V(" last_elem name %s ", last_name);
/* remove unlinks automatically */
//gst_bin_remove(GST_BIN(pipeline), cur_effect);

guint tee_pads_num = appCtx->last_elem->numpads;
NVGSTDS_INFO_MSG_V("tee pad num: '%u'", tee_pads_num);
g_snprintf(pad_name, 15, "src_%u", tee_pads_num - 2);
last_ele_src_pad = gst_element_get_static_pad(appCtx->last_elem, pad_name);
if (last_ele_src_pad) {
	//NVGSTDS_INFO_MSG_V("Find last_ele_src_pad");
	gst_element_release_request_pad(appCtx->last_elem, last_ele_src_pad);
if (!gst_element_link(appCtx->pipeline.multi_src_bin.bin, appCtx->pipeline.demuxer)) {
		GstCaps *src_caps, *sink_caps; 
		src_caps = gst_pad_query_caps((GstPad *)(appCtx->pipeline.multi_src_bin.bin)->srcpads->data, NULL); 
		sink_caps = gst_pad_query_caps((GstPad *)(appCtx->pipeline.demuxer)->sinkpads->data, NULL); 
		NVGSTDS_ERR_MSG_V("Failed to link '%s' (%s) and '%s' (%s)", \
			GST_ELEMENT_NAME(appCtx->pipeline.multi_src_bin.bin),  \
			gst_caps_to_string(src_caps), \
			GST_ELEMENT_NAME(appCtx->pipeline.demuxer), \
GstPad *demux_sink_pad = NULL;
demux_sink_pad = gst_element_get_static_pad(appCtx->pipeline.demuxer, "sink");
//gst_pad_send_event(demux_sink_pad, gst_event_new_flush_stop(TRUE));
//NVGSTDS_INFO_MSG_V(" Send flush stop event ");
gst_element_get_state(appCtx->pipeline.pipeline, &cur, &pending,
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(appCtx->pipeline.pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pause-pipeline");
NVGSTDS_INFO_MSG_V("After stop ai analysis");

pad_status = gst_pad_is_blocking(demux_sink_pad);
NVGSTDS_INFO_MSG_V(" Demux sink pad status '%d' ",pad_status);

GstPad *streammux_sink_pad = NULL;
streammux_sink_pad = gst_element_get_static_pad(appCtx->pipeline.multi_src_bin.bin, "src");
pad_status = gst_pad_is_blocking(streammux_sink_pad);
NVGSTDS_INFO_MSG_V(" Streammux sink pad status '%d' ", pad_status);

static GstPadProbeReturn
pad_probe_cb(GstPad * pad, GstPadProbeInfo * info, gpointer user_data)
GstPad srcpad, sinkpad;
gchar last_elem_name[15];
pad_name = gst_pad_get_name(pad);
NVGSTDS_INFO_MSG_V(" pad name %s ", pad_name);
NVGSTDS_INFO_MSG_V(“pad is blocked now”);
appCtx = (AppCtx*)user_data;
/* remove the probe first */
gst_pad_remove_probe(pad, GST_PAD_PROBE_INFO_ID(info));

guint tee_pads_num = appCtx->last_elem->numpads;
NVGSTDS_INFO_MSG_V("tee pad num: '%u'", tee_pads_num);
g_snprintf(last_elem_name, 15, "src_%u", tee_pads_num - 2);
srcpad = gst_element_get_static_pad(appCtx->last_elem, last_elem_name);

/* install new probe for EOS */
//srcpad = gst_element_get_static_pad(appCtx->first_elem, "src");
gulong probe_id = gst_pad_add_probe(srcpad, (GstPadProbeType)(GST_PAD_PROBE_TYPE_BLOCK |
	GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM), event_probe_cb, user_data, NULL);
NVGSTDS_INFO_MSG_V("first elem probe id: %lu",probe_id);

/* push EOS into the element, the probe will be fired when the
 * EOS leaves the effect and it has thus drained all of its data */
sinkpad = gst_element_get_static_pad(appCtx->first_elem, "sink");
gst_pad_send_event(sinkpad, gst_event_new_eos());

NVGSTDS_INFO_MSG_V("pad_probe_cb return");


The appCtx->first_elem repres the first elem after nvstreammux , last_elem repres the elem before nvdemux. When the grpc server receive a pause request, I would invoke:
GstPad *streammux_src_pad = NULL;
streammux_src_pad = gst_element_get_static_pad(appCtx->pipeline.multi_src_bin.bin, “src”);
gst_pad_add_probe(streammux_src_pad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM,
pad_probe_cb, appCtx, NULL)
Thus, the callback can be executed.
I can`t find a solution even through the debug log . Could u please give me some advices?

I refer to
https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/blob/master/runtime_source_add_delete/deepstream_test_rt_src_add_del.c, send a flush_stop event after the eos event. But It comes a new question without a sleep between the eos and flush_stop event .

Without sleep:

With sleep 100ms,the pipeline can pause ai-analysis normally, play the original stream.

I look up some gstreamer docs. It says the flush_event would sync the clock_time in the pipeline. But i send a flush_event to a sink_pad that already unlink from the pipeline(the preprocess sink_pad). Without the flush_event, why can`t play the stream normally? What reason block the stream?

Why do you need to do this? Even with nvinfer and nvtracker in the pipeline, the stream is the original one.

Please upgrade DeepStream to the latest version. Please enable the encoder log to check whether there are frames to output.

export GST_DEBUG=v4l2videoenc:5

This is a project demand. We use deepsteam for the drone ai-analysis. When the drone reach the set location, it start to ai-analysis. Or it will paly the original stream without alarm message . If not reach the location, it will produce wrong alarm or show wrong bbox in the video due to the ai-model precision issue with ai-anlysis. I will try export v4l2 debug log.

Anothe quesiton, why after remove the infer_element , the stream in the pipeline would be stucked. And after send a flush_stop event can resume flow? The eos event cause this problem? I make many tries send flush_stop event through element sink_pad, only the preprocess_bin sink_pad can work (the first element unlink from the pipeline). But it already unlink with the streammux and demux when send flush_stop event after sleep 100ms , how the event travels to streammux and demux ? Or the streammux and demux don`t receive a flush_stop event? What reason make the pipeline flow normally? The confused me for a week.

If you don’t want the pipeline output the inference result, just remove the corresponding NvDsMeta data is enough. You don’t need to remove the gst-nvinfer from the pipeline.

To remove the gst-nvinfer correctly, you need to change the state of the pipeline to NULL before removing and connecting nvstreammux to nvstreamdemux. And then you can start the pipeline again with changing the state to PLAYING.

This is a good solution . I will try .After i figure this question.

Can`t be removed dynamically like other gstreamer element when pipeline in playing state?

No. Even the others element is not dynamically removed. The source element can be removed is because there are other sources still working, If all sources are removed, the whole pipeline stopped.

Here is the log, it seems the v4l2 element encode data normally.
v4l2debug.log (66.0 KB)

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

So the flv stream is live. You need to check your RTMP server.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.