Adding and removing streams during runtime

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Is there a way to add or remove a stream to the pipeline during runtime?

Example: Running deepstream with inferencing on four unique inputs and then adding a fifth input several minutes after the program has started on the original four. Then a few minutes later, removing two of the original four streams so that only three are left.

I would imagine that this should be possible as it is mentioned in the Deepstream SDK pdf, but I haven’t seen any examples of it and have not found a way to do it either.

Does anyone have any examples? Preferably one that can work with the deepstream-app sample as I’m currently most familiar with that workflow.

3 Likes
2

Hi,
This is not supported. Please share where you see the description in ‘the Deepstream SDK pdf’. Seems to have ambiguous content and makes confusion.

3

According to the Deepstream SDK FAQ(https://developer.nvidia.com/deepstream-faq) page:

Is there an example on Dynamic stream management?
Anomaly Detection Reference App (<a target='_blank' rel='noopener noreferrer' href='https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/anomaly'>https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/anomaly</a>)
Dynamically add or delete channels when the pipeline is running.

Also, the post here https://devtalk.nvidia.com/default/topic/1046629/deepstream-for-tesla/dynamic-stream-management-where-how-why-/ suggests what I’m asking is possible.

I can’t imagine this is a limitation. We know a T4 can handle multiple streams with inferencing concurrently. If this isn’t possible then whenever a camera goes down and gets fixed, then the entire pipeline needs to be restarted just to include the fixed camera into the mix. This would cause an interruption in all the other cameras also and would never be considered acceptable by commercial/enterprise clients.

I’m certain that we’re just misunderstanding each other. Please have a look at the links I posted.

4

Hi,
We have samples in DeepStream SDK which are verified in each release. There is no sample in DS4.0.1 demonstrating this usecase. We will check if we can include it in future releases.

5

I know I read something about this. It turns out it was in the “4.0 Early Access Release Notes”:

1.4 DIFFERENCES WITH DEEPSTREAM 3.0
Certain features included with DeepStream 3.0 release are not supported in this release. These include:

  • Dynamic Stream Management (runtime addition and removal of streams)
    …

However, I also checked the 4.0 and 4.0.1 release notes - neither of them list that restriction. So, it seems like it was a known test case that was broken and later believed to be fixed.

I agree with @cbstryker that this is a core use case for many users, myself included. Unfortunately, another current thread makes it sound like there are (still) issues with this functionality:

6

Hi,
We are working on a sample of demonstrating runtim deleting/adding source. Will publish it once it is ready. Please stay tuned. Thanks.

7

+1

8

+1

9

+1

10

+1

11

It seems there are a lot of groups interested in this functionality. Do you have any rough timeline of when the sample would be ready - days, weeks, or months?

12

We have Dynamic stream addition/removal app added on GitHub. Here is the link for the app:
https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/runtime_source_add_delete

2 Likes
13

Hooray! Thanks for this. It looks like the main thing that I was missing is sending gst_event_new_flush_stop (FALSE) to the request pad before releasing it.

14

[/url]

Hi,
we met a problem in out application with dynamic stream addition/removal, here is our pipiline structure:

{…,live_stream_i,…}—>nvstreammux—>pgie–>nvtracker–>nvstreamdemux–> {…,output_stream_i,…}–> etc_i

different from the exmaple https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/runtime_source_add_delete, besides uridecodebin, there are also dynamic del/add of GSTelements after nvstreamdemux of pipeline in each stream. our problem is:
removal of stream was ok, but after addition of stream, the stream just played for a while(several seconds), then stoped. once addition of next stream (played for a while then stoped ), the stoped stream recovered. and the problem only occured in rtmp live source, while rtsp is ok.
could you give me some advices? thanks!

15

Hi 489536658,

Please open a new forum topic for your issue amd we will pick it up there.

16

we have opened a new forum topic

thanks

17

I’ve noticed in the provided dynamic example on deepstream_reference_apps/runtime_source_add_delete at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub that applications need to set the batch-size for the stream-muxer and gies to to MAX_NUM_SOURCES at a minimum.

Is there any concern or potential issue if the number of sources never reaches the batch-size? For the case were the user, or other runtime conditions, controls the add/delete.

For example, setting the MAX to 20, but only ever reaching 2?

thanks

18

Any chance of getting a python example for this please?

Our use case is that we k8s pod’s with one GPU assigned to each, when a user streams video for object detection from a smart device the instance connects to the stream and communicates classification via websockets.

I suppose we could just launch one pod per user, with the stream ID as a parameter but it’s a bit inefficient. :-)

2 Likes
19

The behavior i’ve noticed is that there can be problems if the number of connected sources doesn’t match the batch-size on the stream-muxer, but the nvinfer element just uses it as a maximum for the batch-size.

Frankly, I am still very confused but that last above linked post cleared up a lot. I am hoping for DeepStream 5, the handling of this parameter, as well as the engine file name which depends on batch-size, is automatically handled by all nvidia elements.