• Hardware Platform (Jetson / GPU) Laptop with NVIDIA GPU RTX 4060
• DeepStream Version 7.0
• TensorRT Version 8.6.1.6-1+cuda12.0
• NVIDIA GPU Driver Version Driver Version: 550.107.02 CUDA Version: 12.4
• Issue Type Questions
Greetings.
There is a pipeline that looks as follows:
multiurisrcbin --> queue --> nvinfer1 --> nvinfer2 --> queue --> tracker --> tee --> tiler --> queue --> videoconvert --> capsfilter --> queue --> osd --> eglglessink
There is some logic that is executed inside the tracker’s probe SRC pad that generates output for each of the sources, and also generates some output for the osd element.
There is a need to start recording one of the active batched streams to a file at a certain moment and then stop it at a certain moment. At the same time, the recording to file should include graph overlays by osd element, but of course without including tiler in the output file (since we record only one stream). Also, there is a need to take periodic screenshots with the same conditions.
To solve this task, I went the following way: when I receive a message on the bus with the string “stream-add”/“stream-remove” in the structure name, I dynamically add/remove an additional branch (of course, doing it from the main thread), which I link to the tee element of the main pipeline. Here is what the dynamic branch looks like:
tee |-> nvstreamdemux --> queue --> videoconvert --> queue --> osd --> tee --> fakesink
Also, there can be many such branches, depending on the number of active sources limited by batch-size.
In my case, a separate nvstreamdemux is used for each new branch, because when using a single nvstreamdemux for all branches, I encountered a problem where new connection to the SRC pad of the nvstreamdemux can only be made when this element is in NULL state. In my case, for example, if I already have an active stream running through nvstreamdemux, then when linking a new branch I will have to set nvstreamdemux to NULL, which will affect existing streams.
On the osd element of the dynamic branch, I have a probe on the SRC pad, from which I perform a screenshot save to a file using the nvds_obj_encode API if necessary.
Then, when needed to start recording, I dynamically create a new branch that looks like this and links to the output SRC pad of the tee element of the dynamic branch created for a particular source:
tee |-> queue --> videoconvert --> v4l2h264enc --> h264parse --> qtmux --> filesink
There are two problems I’ve encountered:
- The above pipeline logic works (adds and removes dynamic branches correctly) as long as I have one source. When I add a second or more sources, I stop getting data on the SRC pad of nvstreamdemux for each new branch. That is, when I set up probes on the SINK and SRC pads of nvstreamdemux for each item, I see output in the console like this:
[STREAM:0] Got on demux SINK pad! Frame source_id: 0
[STREAM:0] Got on demux SINK pad! Frame source_id: 1
[STREAM:0] Got on demux SRC pad! Frame source_id: 0
[STREAM:1] Got on demux SINK pad! Frame source_id: 0
[STREAM:1] Got on demux SINK pad! Frame source_id: 1
That is, here you can see how frames from both sources are received by the SINK pads of both elements (nvstreamdemux), but at the same time you can see that the SRC pads only generate data for the first source (number 0). Of course, this behavior makes the whole branch inoperable.
Does anyone have positive experience in the context of trying nvstreamdemux to solve this kind of tasks?
What could be the problem in my case?
I will be glad to get any ideas/advice.
Unfortunately, I can’t share the entire source code, however I can do so for specific sections of code if needed.
For a more detailed understanding, I attach the pipeline graph during two active sources and active recording on one of them.
0.00.17.679427233-deepstream-pipeline-graph-new.dot.txt (97.3 KB)
- Also, there is a strange problem of osd elements conflicting during recording/capturing screenshots, and I can’t understand the reason for this behavior.
As you can see from the video, the frame shows those overlaps that are formed after tiler and osd elements of the main branch of the batched stream. At the same time, no overlaps occur on the osd that is after the tiler (in the main branch).
If anyone has any ideas about this?
