Use nvdewarper plugin together with regular video source in single pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
GPU - RTX2060
• DeepStream Version
5.1
• JetPack Version (valid for Jetson only)
n/a
• TensorRT Version
7.2.3-1+cuda11.1
• NVIDIA GPU Driver Version (valid for GPU only)
455.32.00
• Issue Type( questions, new requirements, bugs)
question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
n/a
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I’m trying to use the nvdewarper plugin together with a regular video source. I modified the config file loader so that a dewarper config can be attached to each source element via NvDsSourceConfig::dewarper_config (deepstream_sources.h:77).

The config for the sources is:

[source0]
enable=1
type=3
uri=file:///home/videos/FISHEYE.mp4
gpu-id=0
nvbuf-memory-type=0

dewarper-enable=1
dewarper-config-file=dewarping_config.ini
dewarper-num-surfaces-per-frame=2
dewarper-source-id=0

[source1]
enable=1
type=3
uri=file:///home/videos/REGULAR.mp4
gpu-id=0
nvbuf-memory-type=0

This works well only one of the sources is enabled. However, if using both sources, the application fails with an error from nvstreammux:

ERROR from src_bin_muxer: Input buffer number of surfaces (1) must be equal to mux->num_surfaces_per_frame (2)
Set nvstreammux property num-surfaces-per-frame appropriately

The main source file used is the modified example app from deepstream-occupancy-analytics
Here, I set the num-surfaces-per-frame to equal the number of dewarp surfaces from the source group.

g_object_set (appCtx[i]->pipeline.multi_src_bin.streammux,
“buffer-pool-size”, STREAMMUX_BUFFER_POOL_SIZE,
“num-surfaces-per-frame”, 2,
NULL);

To sum it up, is it possible to use dewarped video sources together with regular video sources in the same pipeline? If so, how to set up the streammux to be able to handle both type of sources (with one or multiple surfaces per frame).

3

Does regular source mean one frame only includes one surface?

4

Yes, a source that does not use the dewarper plugin has only one surface, according to the error message given by streammux.

5

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Ok, can you share your pipeline what you are trying to do?