[Nano] Deepstream-test3-app not working as expected for multiple video source

With Deepstream SDK 4.0, I have successfully run 8 channel 1080p video at 30 FPS with deepstream-app sample program. On deepstream-app.c file, I see the line

/**
 * Function to add components to pipeline which are dependent on number
 * of streams. These components work on single buffer. If tiling is being
 * used then single instance will be created otherwise < N > such instances
 * will be created for < N > streams
 */
static gboolean
create_processing_instance (AppCtx * appCtx, guint index)

which I though equivalent to deepstream-test3-app where N number of sources could be processed with this snippet

for (i = 0; i < num_sources; i++) {
    GstPad *sinkpad, *srcpad;
    gchar pad_name[16] = { };
    GstElement *source_bin = create_source_bin (i, argv[i + 1]);
.......
.......

because it “Configure the stream-muxer to generate a batch of frames and infer on the batch for better resource utilization”
However, my idea got wrong when I tried to run 4 channel 720p video with deepstream-test3-app which drops frame significantly and lags rigorously. Am I getting the idea wrong or is there anything I am missing?

Hi,
We have optimization in config_infer_primary_nano.txt. Please copy it to overwrite dstest3_pgie_config.txt.

nvidia@nvidia-desktop:~/deepstream_sdk_v4.0_jetson/sources/apps/sample_apps/deepstream-test3$ cp -i ../../../../samples/configs/deepstream-app/config_infer_primary_nano.txt dstest3_pgie_config.txt

And modify paths accordingly:

model-file=../../../../samples/models/Primary_Detector_Nano/resnet10.caffemodel
proto-file=../../../../samples/models/Primary_Detector_Nano/resnet10.prototxt
labelfile-path=../../../../samples/models/Primary_Detector_Nano/labels.txt

Thanks for the update. I have tried with the optimized config file for nano. Unfortunately, even with 720p 30fps video, lots of frames are being dropped when running on 4 channels.

I am receiving the warning:

Frame Number = 127 Number of objects = 24 Vehicle Count = 20 Person Count = 4
WARNING from element nvvideo-renderer: A lot of buffers are being dropped.
Warning: A lot of buffers are being dropped.
WARNING from element nvvideo-renderer: A lot of buffers are being dropped.
Warning: A lot of buffers are being dropped.
Frame Number = 128 Number of objects = 8 Vehicle Count = 6 Person Count = 2

Hi,
Please try nvoverlaysink:

#ifdef PLATFORM_TEGRA
-  transform = gst_element_factory_make ("nvegltransform", "nvegl-transform");
+  transform = gst_element_factory_make ("queue", "queue");
#endif
-  sink = gst_element_factory_make ("nveglglessink", "nvvideo-renderer");
+  sink = gst_element_factory_make ("nvoverlaysink", "nvvideo-renderer");

And set ‘sync=false’ to nvoverlaysink:

g_object_set (G_OBJECT(sink), "sync", FALSE, NULL);

Thanks for the update. The modification works fine with 4 channel 720p video @30fps. But adding 8 channel slows down the videos, something not seen with the deepstream-app demo. Maybe on that demo, some more optimizations are used.

Hi,
For eight channels, you may apply interval=4 and try again.
The same setting is in source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt

With non-zero interval value, you shall need nvtracker plugin to get smooth output.

Thanks again DaneLLL for your suggestion. Setting the interval greater than zero reduces video lag/jitters because it inferences the video frames at specified intervals. I would try with nvtracker plugin and let you know.

Hi where do I make these changes in the python file for test 3 as I can’t find these?

Hi,
The samples are identical in C and python. Please check