Nvbufsurface with new streammux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi @fanzh I am using new streammux, along with nvdsanalytics.
The nvdsanalytics supported the muxed width, and height for the old streammux for all the different inputs. But How do I get the frames independent to their resolutions, if I have multi resolution input using the nvdsbufsurface, and along with new streammux, here we know that the inputs are not muxed to a single resolution.

3

please refer to heterogeneous-batching. You can add nvvideoconvert + capsfiler before each new-nvstreammux sink pad for converting different resolutions to the same resolution.

4

Hi @fanzh I do not want to convert to same resolution.

I want to process with different resolutions using the new streammux, but according to the nvbufsurface, it considers a muxed resolution, which I do not want.

And also the nvdsanalytics element has a fixed width, and height properties..

I want the analytics to process all the frames with their original resolutions, and while drawing the points on the individual frames, the ROI are either stretched, or downsized w.r.t to the width and height of the element.

5

@fanzh would really appreciate your help.

6

Thanks for the sharing! You may try the following command-line. which mixes two different sources with new-streammux. Please note that with new mux, a single resolution for this heterogeneous batched buffer is invalid.

$ cd /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-nvdsanalytics-test
export USE_NEW_NVSTREAMMUX=yes  && gst-launch-1.0 filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 ! qtdemux  ! h264parse ! nvv4l2decoder ! mux.sink_0 nvstreammux name=mux batch-size=1 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary.yml !  nvtracker tracker-width=640 tracker-height=384 ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so ll-config-file=../../../../samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml ! nvdsanalytics config-file=config_nvdsanalytics.txt ! nvmultistreamtiler  ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=RGBA' ! nvdsosd  ! nveglglessink  filesrc location=/opt/nvidia/deepstream/deepstream/samples/streams/sample_qHD.mp4 ! qtdemux ! h264parse ! nvv4l2decoder !  mux.sink_1
7

@fanzh are you telling that we should not give camera url with single resolution?

8
  1. I mean, as written in the doc, New nvstreammux does not transform/scale batched buffers to a single color-format/resolution. A batch can have buffers from different streams of different resolutions and formats. So with new mux, a single resolution for this heterogeneous batched buffer is invalid.. the resolution in new-streammux src caps is invalid because there are multiple resolutions in batch generated by new-streammux.
  2. why do you want the analytics to process all the frames with their original resolution? is there an difference or bug?
9

@fanzh How does analytics behave with different resolutions, and also while taking frames from the nvbufsurface??

I need help with these 2 things.

While trying to use different resolution streams in the pipeline, but nvdsanalytics is affected by the config-width, and height property.
For each stream roi, we multiply with the individual stream resolutions, but the above parameter affects it.

I have 4 different resolutions in the pipeline, but the nvdsbufsurface gives one resolution, and that too random out of the 4 different resolutions.

say first couple of times i try to run it gives 1920x1080 which is 1/4 resolutions in the pipeline..
next couple of times it gives 1280x720 which is 1/4 resolutions in the pipeline..

10

I understand the first point.

Why would I want to process all the different resolutions to a single resolution on analytics.
When my nvinfer takes different resolutions.
There is a difference and I have mentioned it above this reply.

11
  1. Please refer to struct NvBufSurfaceParams, which includes width, height. thought there are mutiple resolutions in the batch, the following plugins still can parse the data with surface parameters.
  2. nvstreammux does not give a random resolution. it gives the reoslution of first soruce linking to new-streammux. You can try the command-line above.
12

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

13

of a single buffer?

NvBufSurfaceParams holds information about a single buffer within a batch,

What about other resolutions?

14

I did not mention nvstreammux, I mentioned nvbufsurface.

15

Can you provide help on nvdsanalytics first?

16

GPU

7.1

10.11

17

So when I am trying to extract the frames using the nvbufsurface, every streams will hold its respective resolutions right?

Although it hold the information of a single buffer within a batch.

18

right. please refer to struct NvBufSurface and NvBufSurfaceParams in nvbufsurface.h. NvBufSurface represents a batch. NvBufSurfaceParams represents each frame in batch. it includes width, height, buffer etc. Btw, new-streammux is opensource.

19

@fanzh need help with analytics first.

20

could you elaborate on this question? what do you mean by “For each stream roi, we multiply with the individual stream resolutions, but the above parameter affects it.”? if there is a bug, please provide a simplified command-line to reproduce the issue. you may modify my command-line above.

21

Then why does it only output a single resolution?