Information
• Hardware Platform (Jetson / GPU) : Jetson TX2 - 4GB Ram
• DeepStream Version : DS-4.01 - 4.03
• JetPack Version (valid for Jetson only) : JetPack 4.1 - 4.3
We have below pipeline to which dynamically add/remove one or more input arm and correspondent one or more output arm
interpipeSrc-1 -> queue -> NvVideoConvert -> \ / AppSinkBin-1
interpipeSrc-2 -> queue -> NvVideoConvert -> / -> NvStreamMux -> NvInfer -> NvTracker -> NvStreamDemux -> \ AppSinkBin-2
. . . . . . . . . . . . . . . . . . . / \ . . . .
interpipeSrc-N -> queue -> NvVideoConvert -> / \ AppSinkBin-N
AppSinkBin contains these simple components
AppSinkBin-1 = queue → AppSink-1
It happens that
-) when removing one o more channels all work correctly, so for example when removing channel 2 it is removed correctly and the other channels keep on working correctly and producing correct tracking metadata.
-) when adding channels, the new channel work if no other channels are already active
but if adding a new channel having 1 or more other channel active then “after adding” new channel we get below crash
ATTENTION: it seems that crash happen not during new component creation so but when new channel frames are starting arriving.
(gdb)
(gdb) bt
#0 0x0000007f5b14f718 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so.1.0.0
#1 0x0000007f5b153ba4 in NvBufSurfTransform () from /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so.1.0.0
#2 0x0000007f783539d0 in ?? () from /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libnvdsgst_multistream.so
#3 0x0000007f7e5c7b44 in ?? () from ./libgstreamer-1.0.so.0
#4 0x000000000992faa0 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) quit
A debugging session is active.
ATTENTION: if for output component instead of AppSinkBin-X we use only one single TiledBin → FileSink … it seems that addition works without crash.
So it could be that problem could be in NvStreamDemux after dynamic addition/linking of a component to pad of NvDeMuxer, because in crash there is no presence of AppSink components.
Or could it be there is some problem between NvStreamDemux and AppSinkBin ?
Than it is not clear why it involved also “nvbufsurftransform”
I saw that there are other guys having very similar problems like the one described here
How to add source and sink with nvstreammux and nvstreamdemux dynamically - #10 by pineappleProcessor
and so I followed Nvidia support suggestion for nvDemuxer pad pre-allocation, because it seems Demuxer doesn’t support dynamic allocation, this helped a lot during channel deletion … but during channel addition we always get a crash.
Do you have some suggestions ? So what is the correct way to dynamically allocate pad in NvDemuxer and inject new frames ?
Here below there some parts of code showing how we add new dynamic input and output components.
There is not code for the static central part for NvMuxer + NvInfer + NvTracker + NvDemux … becuase it is very similar to what you use in your deepstream-app.
Alternatively another proposal is: if you can send me a debug version of library components (so library compiled with symbols ) of following 2 objects
libnvdsgst_multistream.so
libnvbufsurftransform.so.1.0.0
I can add them to our environment and we could have exact methods names producing crash
Now we are using both JetPack/deepsream 4.1 and 4.3, crash is always the same on both releases
thanks for support
Maurizio
// During creation of static NvStreamMux + NvInfer + NvTracker + NvStreamDemux there is the suggested preallocation of Demux pad
// ----------------------------------------------------------------------------------------------------------------------
for ( gint i= 0; i< MAX_ANALYTICS_CH; ++i ) // i++ )
{
GstPad *demux_src_pad;
gchar pad_name[16];
g_snprintf(pad_name, 16, "src_%02d", i);
{
demux_src_pad = gst_element_get_request_pad(_coreSet->demuxer, pad_name);
if (demux_src_pad == NULL)
g_printerr ("\n ERROR: Failed to get the request pad [%s]\n", pad_name);
else
g_print ("\nOK: Succes to get the request pad [%s]", pad_name);
gst_object_unref (demux_src_pad);
}
}
// During creation of dynamic channel component there is linking of above created output DeMux pads with AppSinkBin
// ----------------------------------------------------------------------------------------------------------------------
GstElement *
createInOutComp_forChan ( CorePipeElements *_coreSet, gint pipeNum, gchar *interPipeName )
{
// ===============================================================================
// Input components Adding Interpipe src and nvvideoconvert before NvMux + NvInfer + NvTracker + NvDeMux
// ===============================================================================
g_snprintf (elem_name, sizeof (elem_name), “interPipeSrcIn-%d”, pipeNum);
_coreSet->interPipeSrcIn[pipeNum] = gst_element_factory_make(“interpipesrc”,elem_name);
g_assert(_coreSet->interPipeSrcIn[pipeNum]);
// --------------------------------------------------------------
g_snprintf (elem_name, sizeof (elem_name), “%s”, (const char *)_coreSet->chName[pipeNum]->str );
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “listen-to”, elem_name, NULL ) ;
// ---------------------------------------------------------------
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “format”, 3, NULL);
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “enable-sync”, 1, NULL);
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “do-timestamp”, 1, NULL);
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “is-live”, 1, NULL);
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “accept-events”, TRUE, NULL);
g_object_set ( _coreSet->interPipeSrcIn[pipeNum], “allow-renegotiation”, TRUE, NULL);
// ----------------------------------------------------------------------------
g_snprintf (elem_name, sizeof (elem_name), “interQueueCv-%d”, pipeNum);
_coreSet->interQueue[pipeNum] = gst_element_factory_make(“queue”,elem_name);
g_assert(_coreSet->interQueue[pipeNum]);
//// ----------------------------------------------------------------------------
g_snprintf (elem_name, sizeof (elem_name), “nvVideoConvert-My-%d”, pipeNum);
_coreSet->nvVideoConvertMux[pipeNum] = gst_element_factory_make(“nvvideoconvert”,elem_name);
g_assert(_coreSet->nvVideoConvertMux[pipeNum]);
// =================================================================================
gst_bin_add_many(GST_BIN(_coreSet->CoreInferencePipe), _coreSet->interPipeSrcIn[pipeNum],
_coreSet->interQueue[pipeNum], _coreSet->nvVideoConvertMux[pipeNum], NULL);
if (!gst_element_link_many(_coreSet->interPipeSrcIn[pipeNum], _coreSet->interQueue[pipeNum], _coreSet->nvVideoConvertMux[pipeNum], NULL))
{
g_print("\n ERROR: cannot link _coreSet->interPipeSrcIn[%d] ... interQueue[%d] to _coreSet->nvvidconv[%d] ....\n", pipeNum, pipeNum, pipeNum);
_coreSet->CoreInferencePipe = NULL;
goto done;
}
// this function link interPipeSrcIn to input mux pad number 0, 1, 2, 3, ...
// --------------------------------------------------------------------------------
if (!link_element_to_streammux_sink_pad(_coreSet->nvstreammux, _coreSet->nvVideoConvertMux[pipeNum], pipeNum ))
{
g_print("\n ERROR: cannot link _coreSet->nvVideoConvertMux[pipeNum] to nvstreammux ....\n");
gst_object_unref(_coreSet->CoreInferencePipe);
_coreSet->CoreInferencePipe = NULL;
goto done;
}
g_print("\n OK ... linked _coreSet->nvVideoConvertMux[%d] to nvstreammux ....", pipeNum);
// ==============================================================================
// Output components after NvMux + NvInfer + NvTracker + NvDeMux
// ==============================================================================
{
gchar pad_name[16];
GstPad *demux_src_pad;
if (!create_app_sink_bin_CCM(&_coreSet->appSinkBins[pipeNum], pipeNum))
{
NVGSTDS_ERR_MSG_V("create_app_sink_bin %d failed\n", pipeNum);
goto done;
}
gst_bin_add(GST_BIN(_coreSet->CoreInferencePipe), _coreSet->appSinkBins[pipeNum].bin);
// -----------------------------------------------------------------------------------------------
GstAppSinkCallbacks callbacks;
callbacks.new_preroll = new_preroll_comp;
callbacks.new_sample = new_sample_comp;
callbacks.eos = eos_comp;
gst_app_sink_set_callbacks(GST_APP_SINK(_coreSet->appSinkBins[pipeNum].appsink), &callbacks, GUINT_TO_POINTER(pipeNum), NULL);
// -----------------------------------------------------------------------------------------------
g_snprintf(pad_name, 16, "src_%02d", pipeNum);
g_print ("\n linking demuxer with pad_name=[%s] ", pad_name );
demux_src_pad = gst_element_get_static_pad (_coreSet->demuxer, pad_name);
// linking demux_src_pad with appSinkBin sink_pad
---------------------------------------------------
GstPad *sink_pad = gst_element_get_static_pad (_coreSet->appSinkBins[pipeNum].bin, "sink");
// link the pads
if (gst_pad_link (demux_src_pad, sink_pad) != GST_PAD_LINK_OK) {
g_printerr ("Failed to link sink_element to demux!\n");
// handle errors in any way you see fit (maybe exit)
}
else
g_print ("\n linked demuxer with appSink=[%s] <<<<<<<<<<<<< ", pad_name );
gst_object_unref(demux_src_pad);
}