Why nvstreamdemux scales all video sources to the resolution of 1st video source linked to new nvstreammux)

• Hardware Platform (GPU, Probably the same issue on Jetson)
• DeepStream Version 6.0.1
• TensorRT Version 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only) 470
• Issue Type ( bugs)

By feeding multiple video sources with mixed video solutions to the new nvstreammux in adaptive batching mode. It ends with all videos are scaled to the same resolution of first video source at nvstreamdemux unexpectedly. Even worse, occasionally, all videos are being scaled and cropped to very tiny image (we have seen 320x180 at the nvstreamdemux).

nvstreamdemux does not scale videos. What is your pipeline and configurations?

Thanks for replay!

That’s true if new nvstreammux beta (so the old nvstreammux) is set to batch-size mode in which case all video are scaled to the same resolution at nvstreammux.

In new nvstreammux beta adaptive batch mode, we observed that all videos get scaled at nvstreamdemux. I am working on modifying a Nvidia deepstream-test case and will upload the code to show you when it is ready.

Here is my modified code from deepstream-test5 sample app. The tiler and tracker are disabled. I only enabled two video sources, one video is 1280x720, the other is 1920x1080. There are two corresponding sinks linked from nvstreamdemux. I included the graph dump “playing.pdf”. You can see all videos from the demuxer show the same resolutions 1920x1080. Make sure that you export USE_NEW_NVSTREAMMUX=yes before you run the app.
Note you will notice that annotations (drawing box) will be out of place if source resolution is different from the demuxer output resolution. This is the same issue from the link Is nvstreamdemux compatible with the new nvstreammux?
deepstream-test5.zip (54.2 MB)

Here is the graph dump when I switched the video source feed order. All demuxed sinks now show 1280x720. Here is the graph.
playing2.pdf (50.2 KB)

1 Like

@Fiona.Chen I can confirm this happens for us as well. Seems the nvstreamdemux is, in fact, scaling all video feeds to the resolution of the first stream it receives.

@Fiona.Chen have you been able to replicate this problem? I have confirmed that the test case provided by @xiaotianshimail does in fact show that nvstreamdemux is performing scaling. We actually have a bizarre case with a customer where nvstreamdemux is scaling to 320x180. We cannot explain where this scaling value is coming from as all input video streams are 1920x1080. Could it be a default within nvstreamdemux?

Is anyone else at @nividia able to take a look at this?

sorry for the late response, I am debugging.

1 Like

Small update, we found an explanation to the 320x180 case. In that particular setup, there was one camera feed of the seven that was set to 320x180. If it happened to be the first feed added to the pipeline, all the other feeds would also be scaled to 320x180. So disregard my question about it being a default. We do still need to understand why the feeds are getting scaled at all since we are using the new nvstreammux and nvstreamdemux. Thanks for looking into it.

it can reproduce on jetson xavier, deepstream6.0. we are checking.
here is the step:
export USE_NEW_NVSTREAMMUX=yes && gst-launch-1.0 -e nvstreammux name=mux batch-size=2 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.txt batch-size=2 ! nvstreamdemux name=demux filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264 ! h264parse ! nvv4l2decoder ! queue ! mux.sink_0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! queue ! mux.sink_1 demux.src_0 ! queue ! nvvideoconvert ! nvdsosd ! nvegltransform ! nveglglessink demux.src_1 ! queue ! nvvideoconvert ! nvdsosd ! nvegltransform ! nveglglessink

1 Like

Hi @fanzh any updates? If scaling is built in to nvstreamdemux, that’s fine, we just need a way to control what it scales to. We could actually use that feature.

please refer to the official doc: Gst-nvstreammux New — DeepStream 6.1 Release documentation
currently new streammux does not accept different resolutions, the solution is convert different resolutions to the same resolution.

@fanzh the original post is pointing out that the official documentation is wrong. I think you verified that nvstreamdemux is scaling all feeds to the resolution of the first detected source, correct? According to the documentation, it should not scale the feeds at all. That’s the behavior I would like, personally. With the new nvstreammux, we’d like to keep the original resolution of the feed all the way through the nvstreamdemux (no scaling). Is this not possible?

only dose not accept different resolutions currently, It is new streammux 's limitation, we will address this in the future.

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.