[Nano] Deepstream-test3-app not working as expected for multiple video source

With Deepstream SDK 4.0, I have successfully run 8 channel 1080p video at 30 FPS with deepstream-app sample program. On deepstream-app.c file, I see the line

/**
 * Function to add components to pipeline which are dependent on number
 * of streams. These components work on single buffer. If tiling is being
 * used then single instance will be created otherwise < N > such instances
 * will be created for < N > streams
 */
static gboolean
create_processing_instance (AppCtx * appCtx, guint index)

which I though equivalent to deepstream-test3-app where N number of sources could be processed with this snippet

for (i = 0; i < num_sources; i++) {
    GstPad *sinkpad, *srcpad;
    gchar pad_name[16] = { };
    GstElement *source_bin = create_source_bin (i, argv[i + 1]);
.......
.......

because it “Configure the stream-muxer to generate a batch of frames and infer on the batch for better resource utilization”
However, my idea got wrong when I tried to run 4 channel 720p video with deepstream-test3-app which drops frame significantly and lags rigorously. Am I getting the idea wrong or is there anything I am missing?

Hi,
We have optimization in config_infer_primary_nano.txt. Please copy it to overwrite dstest3_pgie_config.txt.

nvidia@nvidia-desktop:~/deepstream_sdk_v4.0_jetson/sources/apps/sample_apps/deepstream-test3$ cp -i ../../../../samples/configs/deepstream-app/config_infer_primary_nano.txt dstest3_pgie_config.txt

And modify paths accordingly:

model-file=../../../../samples/models/Primary_Detector_Nano/resnet10.caffemodel
proto-file=../../../../samples/models/Primary_Detector_Nano/resnet10.prototxt
labelfile-path=../../../../samples/models/Primary_Detector_Nano/labels.txt

Thanks for the update. I have tried with the optimized config file for nano. Unfortunately, even with 720p 30fps video, lots of frames are being dropped when running on 4 channels.

I am receiving the warning:

Frame Number = 127 Number of objects = 24 Vehicle Count = 20 Person Count = 4
WARNING from element nvvideo-renderer: A lot of buffers are being dropped.
Warning: A lot of buffers are being dropped.
WARNING from element nvvideo-renderer: A lot of buffers are being dropped.
Warning: A lot of buffers are being dropped.
Frame Number = 128 Number of objects = 8 Vehicle Count = 6 Person Count = 2

Hi,
Please try nvoverlaysink:

#ifdef PLATFORM_TEGRA
-  transform = gst_element_factory_make ("nvegltransform", "nvegl-transform");
+  transform = gst_element_factory_make ("queue", "queue");
#endif
-  sink = gst_element_factory_make ("nveglglessink", "nvvideo-renderer");
+  sink = gst_element_factory_make ("nvoverlaysink", "nvvideo-renderer");

And set ‘sync=false’ to nvoverlaysink:

g_object_set (G_OBJECT(sink), "sync", FALSE, NULL);

Thanks for the update. The modification works fine with 4 channel 720p video @30fps. But adding 8 channel slows down the videos, something not seen with the deepstream-app demo. Maybe on that demo, some more optimizations are used.

Hi,
For eight channels, you may apply interval=4 and try again.
The same setting is in source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt

With non-zero interval value, you shall need nvtracker plugin to get smooth output.

Thanks again DaneLLL for your suggestion. Setting the interval greater than zero reduces video lag/jitters because it inferences the video frames at specified intervals. I would try with nvtracker plugin and let you know.

Hi where do I make these changes in the python file for test 3 as I can’t find these?

Hi,
The samples are identical in C and python. Please check

I had a similar performance issue on a nano when trying to do real time with an RTP video feed.
The solution above using nvoverlaysink seems to solve the performance issue.
However the nvoverlaysink does not seem to take any notice of the window size set earlier in the
pipeline (nveglglessink does).

Is there a way to control nvoverlaysink’s window size. I notice parameters for the overlay but
no obvious window size parameter in the doc.

Hi gjsmith,

Please help to open a new topic for your issue. Thanks