How to set the order of tee and nvstreammux and link them to duplicate a video file, before nvinfer in deepstream pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi everyone,

I constructed a pipeline in deepstream-5.0 on GPU.
in this pipeline I want to copy my source video file into multiple queues, so to this end, I used tee to create some branches and set a queue for each branch, which all queues have the same frame content.
then I want to batch these queues using nvstreammux and finally the output of the nvstreammux feed to the nvinfer. the image below shows my pipeline.

pipeline2
pipeline21457×393 45.7 KB

but when I run this pipeline it can not link the queues to the tee and gives me some errors such as unexpected link.

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! videorate ! “video/x-raw(memory:NVMM),framerate=5/1” ! tee name = t t. ! queue t. ! m.sink_0 nvstreammux name=m batch-size=2 width=1080 height=720 ! queue ! m.sink_1
! nvvideoconvert ! nvinfer gpu-id=0 config-file-path=/opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/config_infer_primary_detectnet_v2.txt ! nvvideoconvert ! nvdsosd! nvvideoconvert ! x264enc ! avimux ! filesink location=videot.avi

Where did I do wrong in my pipeline?

3

Please refer to:
gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! videorate ! ‘video/x-raw(memory:NVMM),framerate=5/1’ ! tee name=t t.src_0 ! queue ! m.sink_0 nvstreammux name=m batch-size=2 width=1080 height=720 ! nvinfer gpu-id=0 config-file-path=/opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/config_infer_primary_trafficcamnet.txt ! nvvideoconvert ! nvmultistreamtiler ! nvdsosd ! nvvideoconvert ! x264enc ! avmux.video_0 avimux name=avmux ! filesink location=videot.avi t.src_1 ! queue ! m.sink_1

1 Like
4

Thanks a lot @Fiona.Chen for the reply.
It works for 2 queues.

If we want to add 4 queues instead of 2, when I add them as below, it gives me an error:
Error: could not link t to queue2
Error: could not link t to queue3

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! videorate ! “video/x-raw(memory:NVMM),framerate=5/1” ! tee name=t t.src_0 ! queue ! m.sink_0 nvstreammux name=m batch-size=4 width=1080 height=720 ! nvinfer gpu-id=0 config-file-path=/opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/config_infer_primary_detectnet_v2.txt ! nvvideoconvert ! nvmultistreamtiler ! nvdsosd ! nvvideoconvert ! x264enc ! avmux.video_0 avimux name=avmux ! filesink location=videot.avi t.src_1 ! queue ! m.sink_1 ! t.src2 ! queue ! m.sink_2 ! t.src3 ! queue ! m.sink_3

I have just some more questions to understand your solution better:

  • How does the nvinfer performs inference on the second queue, since in this pipeline we feed the first queue to the nvinfer and add the second queue after the nvinfer?
  • why is it necessary to use nvmultistreamtiler in this pipeline?
  • why do we feed the video_0 to the avmux?

Thanks.

5

Do you have experience with gstreamer? Before you start with DeepStream, you need to learn the usage of GStreamer first. https://gstreamer.freedesktop.org/

If you use “gst-inspect-1.0 tee” to check the pad template, you may know tee’s source pad name should be ‘src_%u’. “!” means link pad in gst-launch tool.

This is DeepStream forum. Let’s focus on DeepStream topics.

gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvideoconvert ! videorate ! ‘video/x-raw(memory:NVMM),framerate=5/1’ ! tee name=t t.src_0 ! queue ! m.sink_0 nvstreammux name=m batch-size=4 width=1080 height=720 ! nvinfer gpu-id=0 config-file-path=/opt/nvidia/deepstream/deepstream-5.0/samples/configs/tlt_pretrained_models/config_infer_primary_detectnet_v2.txt ! nvvideoconvert ! nvmultistreamtiler ! nvdsosd ! nvvideoconvert ! x264enc ! avmux.video_0 avimux name=avmux ! filesink location=videot.avi t.src_1 ! queue ! m.sink_1 t.src_2 ! queue ! m.sink_2 t.src_3 ! queue ! m.sink_3

For your additional questions:

1 Like
6

More gstreamer related questions, you may refer to GStreamer community mail list. GStreamer: Mailing Lists

And there are also many other 3rd party gstreamer discussion and references. E.G. Newest Questions - Stack Overflow

1 Like
7

Thanks @Fiona.Chen.

9

@Fiona.Chen
Thanks for the support on the forum.
Some of us are fresh with GStreamer, and I know this is Nvidia / Deepstream topic.
As a kind suggestion, it could save time for everybody a quick guide for Deepstream+GStreamer, because it’s a fundamental part of the SDK.
In the near future I plan to write a guide for this and post it on github, but I’m still very beginner.
As always, thanks for your help.

1 Like