Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) GPU (T4)
• DeepStream Version 6.0, 6.0.1, 6.1
• JetPack Version (valid for Jetson only)
• TensorRT Version DeepStream container default
• NVIDIA GPU Driver Version (valid for GPU only) 450 / 510
• Issue Type( questions, new requirements, bugs) bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
We have reliably working DeepStream 5.1 applications using multiple input sources which are connected to an nvstreammux element. We run them at scale in production without any issues. Since upgrading these applications to DeepStream 6 we can’t use multiple input sources anymore. I’ve tried out the three available versions of DS6:
DS6.0:
- The pipelines with more than 1 input source randomly fails in ~5% of cases.
DS6.0.1 and DS6.1:
- 1 input source works reliably.
- 2 input sources fail randomly in ~5% of cases like with DS6.0.
- With more than 2 input sources the pipeline immediately hangs and never shows any GPU utilisation.
All that has changed is the container we are using, the Python DeepStream bindings and the TensorRT engine which we generated for the required TensorRT version.
Furthermore, I tried out the new nvstreammux element in 6.1 but this showed the same issue as described above.
I’ve analysed the Gstreamer debug logs and double checked any message that is getting put on the message bus to the best of my knowlegde but couldn’t find anything helpful.
I came across other forum entries which sound similar to our problem, however, they were never resolved but just closed like this.
The release notes of DS 6.0.1 mention:
Minor bug fixes in Gst-nvvideo4linux2 encoder/decoder, Gst-nvstreammux, and Gst-
nvstreamdemux plugins
I suspect this to be a bug due to a change in DeepStream 6 and potentially related to the bug 6.0.1 tried to fix.
Please see the Gstreamer diagram below for further reference. It shows a case where a pipeline with 2 input sources was working. The error cases are pipelines with the same structure just more input bins.
Kind regards,
Jens