I am trying to make a deepstream application with which I can use the NEW NVSTREAMMUX to add and remove RTSP video streams dynamically. I have detected the following scenarios with which errors occur. A minimal example to replicate the two problems detected is attached at the end of this thread:
ERROR 1. When an RTSP source connection is lost and an attempt is made to reconnect, it does not feed data to the DS stream and the transmission does not start.
Details: If EOS or RTSP error arrives, the source_bin is removed from the pipeline. This seems to be done correctly. In case you then retry to add that source, which is transmitting again, it seems to add and work correctly.
PROBLEM: An attempt is made to add a new stream that is not available (RTSP error). When this happens, after checking source unavailability, the pipeline components are removed. This causes that the subsequent source_bins that you try to include do not load correctly and do not decode the video.
How to replicate it?
After the pipeline.set_state(Gst.State.PLAYING) line, include the following. This adds a new source every 15 seconds indefinitely (the uri of the first stream offered as argument is added):
GLib.timeout_add(20000, add_streams)
Start execution. After a few seconds stream 0 (first stream as argument) must be terminated so that an EOS signal is sent to DS.
Two scenarios:
Re-enable RTSP stream in a very short period of time (before DS tries to rearm it). It will work correctly.
Wait for DS to try to rearm the RTSP video stream. Once it shows a failure that there is an RTSP error → Re-enable Stream RTSP. When it tries to reassemble it again, it will not expose any error. However, the source_bin will not be fully constructed and will not feed data to DS. The network generated at the time of the error is exposed:
ERROR 2. Depending on the activation of adaptive batching and batch-size of the NEW NVSTREAMMUX different errors are detected.
Details: The following errors are detected when adding new sources:
With Adaptive-Batching enabled in NEW NVSTREAMMUX:
PROBLEM: If we have deleted any source we can add new ones. As long as No. Sources <= initial No. Sources. A failure is displayed by the tracker batching:
[ERROR push 333] push failed [-5] gstnvtracker: NvBufSurfTransform failed with error -3 while converting buffergstnvtracker:
Tested on DS 6.3 and same error.
With Adaptive-Batching disabled in NEW NVSTREAMMUX:
It is possible to add streams larger than initial batch size. But that makes the DS stream computationally inefficient. In the given example code:
If batch-size equal or smaller than initial No. Streams, it allows to add new streams indefinitely.
PROBLEM: If batch-size greater than initial No. Streams, it gives the above tracker error when trying to add new source.
How to replicate it?
Adaptive-Batching enabled:
Enable adaptive-batching in dstest_streammux_config.txt.
After the “pipeline.set_state(Gst.State.PLAYING)” line, include the following. This removes two video sources and then every 30 seconds adds a new one indefinitely (the uri of the first stream offered as argument is added):
Similarly, if DS starts with 3 Streams and a fourth one is added. The source_bin loads fine and connects to Streammux. Streammux updates batch-size to 4. However, the tracker error indicated above appears, which interrupts all the sources:
[ERROR push 333] push failed [-5] gstnvtracker: NvBufSurfTransform failed with error -3 while converting buffergstnvtracker:
I attach the DS network that is generated when the 4 source is added and gives the commented error:
Note: When no NVTRACKER is added, this error disappears, so it seems to be an implementation error of that module.
Adaptive-Batching disabled
Disable adaptive-batching in dstest_streammux_config.txt.
After the “pipeline.set_state(Gst.State.PLAYING)” line, include the following. This adds a new source every 15 seconds indefinitely (the uri of the first stream offered as argument is added):
I need my code to work with NEW NVSTREAMMUX with adaptive batching, allowing to add new sources correctly. I understand that this is not a bug in my application, as it may or may not work depending on the activation of that property and whether or not a Tracker is included.
could you highlight the issue description? we suggest focusing on one problem in one topic.
is the code based on deepstream_rt_src_add_del.py? nocticing there are big code changes in deepstream-app.py, can you simplifiy the deepstream_rt_src_add_del.py to reprodcue the error?
Summarizing, the main problem is that I can’t add new sources dynamically using new nvstreammux because I get the following error, which seems to come from the nvtracker:
[ERROR push 333] push failed [-5] gstnvtracker: NvBufSurfTransform failed with error -3 while converting buffergstnvtracker:
I think the code is sufficiently self-descriptive and replicates my application very well, but I will try to make it more similar to the original example to make it easier for you to debug the problem. I will provide you with such code tomorrow morning, as I am out of the office.
Indeed, the sources are the same. I have tried with the same and different sources, with the same or different resolutions. I have also added nvvideoconvert to each source before the nvstreammux, all with the same resolution, but I get the same result.
I see that you propose to add the batch size of the New Nvstreammux to the same value as the PGIE. I understand that then the adaptive batching does not work or does it wrong? Is this a solution that makes sense according to the documentation?
I understand that activating the adaptive batching would not have to be indicating an explicit value. Besides this has a noticeable performance loss.
new nvstreammux’s batch-size means “Maximum number of frames in a batch.”, if batch-size is set to 1, when adaptive batching is enabled and there are two sources, the num_frames_in_batch will be 2, which is inconsistent with max number 1.
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks
please delete batch-size=30 in the configuration file, then modify streammux.set_property(“batch-size”, 1) to streammux.set_property(“batch-size”, 30) in the code.
