• Hardware Platform: GPU
• DeepStream Version: 6.1
• TensorRT Version: 8.2.5.1
• NVIDIA GPU Driver Version: 510.47.03
• Issue Type: Bug
• How to reproduce the issue? Please see description below!
We are using an inference pipeline with the new nvstreammux where new sources are added on request. A source is terminated and disconnected from nvstreammux when the source reaches EOS. As long as there is at least one active source this works very well. The EOS from the source results in an GST_NVEVENT_STREAM_EOS event from the nvstreammux which we handle downstream in the pipeline followed by removing the stream as well as the nvstreamux request pad in a way which is very similar to the NVIDIA reference app runtime_source_add_delete. When new streams are added the lowest nvstreammux source id, which is not active, is used for the new stream.
The problem is the case when the last stream reaches EOS. Initially this works well with the nvstreammux which generates GST_NVEVENT_STREAM_EOS followed by the pipeline EOS event. The result of this is that the pipeline goes to the PAUSED state. After that it should be possible to go back to PLAYING again when a new stream is added as a FLUSH_STOP is sent to take the elements out of EOS state (as described here and in the same way as in the NVIDIA reference app for runtime_source_add_delete). The pipeline actually reaches PLAYING state when the new source is added, but immediately after that there is an error from nvinfer due to a missed GST_NVEVENT_PAD_ADDED from nvstreammux. We have tried different work-arounds with listening to the pipeline EOS event on the bus followed by changing state to NULL/READY/PAUSED and then back to PLAYING, but we have not found a way for nvstreammux to get back to the PLAYING state without any errors.
This error can be reproduced with the NVIDIA reference app runtime_source_add_delete in the following way:
Use the deepstream:6.1-devel image. E.g.
docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/deepstream:6.1-devel
Get the reference apps
cd /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps
git clone https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps.git
cd deepstream_reference_apps/runtime_source_add_delete
Comment out the g_main_loop_quit call after pipeline EOS in deepstream_test_rt_src_add_del.c
(At this line)
vi deepstream_test_rt_src_add_del.c
...
Build the program
CUDA_VER=11.6 make
Run the program with the new nvstreammux and a file shorter than 10 seconds
USE_NEW_NVSTREAMMUX=yes ./deepstream-test-rt-src-add-del file:///opt/nvidia/deepstream/deepstream/samples/streams/yoga.mp4
The second stream goes to playing, but then there is a warning from nvinfer
WARN nvinfer gstnvinfer.cpp:1654:gst_nvinfer_process_objects:
Source info not found for source 3.
Maybe the GST_NVEVENT_PAD_ADDED event was never generated for the source.
This warning is written a number of times, finally followed by GST_FLOW_ERROR from nvinfer.
We currently have a work-around with a videotestsrc which is always running, but this is not ideal as the application keeps using both CPU and GPU resources when it should be idle, so it would be very good for us to get a solution to this problem.