nvv4l2decoder with queue min-threshold-time

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

With reference to this [url]https://devtalk.nvidia.com/default/topic/1043993/deepstream-sdk/gstreamer-element-queue-minimum-size/post/5296375/#5296375[/url], is there a way to overcome this buffer size limitation? Would it be possible to free up the omxh264dec’s output buffer by copying the data to a new buffer and pushing the new buffer downstream instead?

2

Hi,
Since we are deprecating omx2h64dec, we don’t add new features to the plugin.
We have DeepStream SDK 4.0 released and nvv4l2decoder is implemented for hardware decoding. Please upgrade your system and try the sample applications. If you require this function, please share a patch to existing samples so that we can build and run the sample to understand your request.

3

Hi,

Thanks for the reply. I am using DeepStream 4.0 now, I forgot that omx plugin is not used anymore. I am however, still facing the same issue:

After adding a gstqueue after a secondary GIE, with min-threshold-time to 1 second, the video does not start playing and I only see a black screen.

I am trying to buffer the results of the inference before allowing playback to continue.

4

Hi,
deepstream-test2 should be close to your usecase. Please share a patch on it so that we can build and run the application to reproduce the issue you are facing.

5

Why not hold 1 second in src input ?

6

Hi, sorry for the late reply. Here is the patch for the changes I did.
The program seems to freeze on start.

diff --git a/deepstream_test2_app.c b/deepstream_test2_app.c
index 5c16dca..7eee116 100755
--- a/deepstream_test2_app.c
+++ b/deepstream_test2_app.c
@@ -392,6 +392,18 @@ main (int argc, char *argv[])
   }
 #endif
 
+  GstElement * queue = gst_element_factory_make ("queue", "queue0");
+  if (!queue) {
+    g_printerr ("Failed to create queue\n");
+    return -1;
+  }
+  guint64 time = 1000*1000*1000;
+  g_object_set(G_OBJECT(queue), "min-threshold-time", time, NULL);
+  g_object_set(G_OBJECT(queue), "max-size-time", 0, NULL);
+  g_object_set(G_OBJECT(queue), "max-size-bytes", 0, NULL);
+  g_object_set(G_OBJECT(queue), "max-size-buffers", 0, NULL);
+
+
   /* Set the input filename to the source element */
   g_object_set (G_OBJECT (source), "location", argv[1], NULL);
 
@@ -422,11 +434,11 @@ main (int argc, char *argv[])
   /* decoder | pgie1 | nvtracker | sgie1 | sgie2 | sgie3 | etc.. */
 #ifdef PLATFORM_TEGRA
   gst_bin_add_many (GST_BIN (pipeline),
-      source, h264parser, decoder, streammux, pgie, nvtracker, sgie1, sgie2, sgie3,
+      source, h264parser, decoder, streammux, pgie, nvtracker, sgie1, sgie2, sgie3, queue,
       nvvidconv, nvosd, transform, sink, NULL);
 #else
   gst_bin_add_many (GST_BIN (pipeline),
-      source, h264parser, decoder, streammux, pgie, nvtracker, sgie1, sgie2, sgie3,
+      source, h264parser, decoder, streammux, pgie, nvtracker, sgie1, sgie2, sgie3, queue,
       nvvidconv, nvosd, sink, NULL);
 #endif
 
@@ -462,13 +474,13 @@ main (int argc, char *argv[])
 
 #ifdef PLATFORM_TEGRA
   if (!gst_element_link_many (streammux, pgie, nvtracker, sgie1,
-      sgie2, sgie3, nvvidconv, nvosd, transform, sink, NULL)) {
+      sgie2, sgie3, queue, nvvidconv, nvosd, transform, sink, NULL)) {
     g_printerr ("Elements could not be linked. Exiting.\n");
     return -1;
   }
 #else
   if (!gst_element_link_many (streammux, pgie, nvtracker, sgie1,
-      sgie2, sgie3, nvvidconv, nvosd, sink, NULL)) {
+      sgie2, sgie3, queue, nvvidconv, nvosd, sink, NULL)) {
     g_printerr ("Elements could not be linked. Exiting.\n");
     return -1;
   }
7

I wanted to cache the inference outputs for up to a second at the same time. Is this the right way to do that?

8

Hi,
We will try to reproduce the issue with the patch.

9

Hi,
Please configure

g_object_set(G_OBJECT(nvvidconv), "output-buffers", 60, NULL);

and put the queue between nvvideoconvert and nvdsosd

if (!gst_element_link_many (streammux, pgie, nvtracker, sgie1,
  sgie2, sgie3, nvvidconv, <b>queue</b>, nvosd, transform, sink, NULL)) {
  g_printerr ("Elements could not be linked. Exiting.\n");
  return -1;
}
10

Hi,
For more information, you may try to configure the property in nvv4l2decoder:

num-extra-surfaces  : Additional number of surfaces in addition to min decode surfaces given by the v4l2 driver
                    flags: readable, writable, changeable only in NULL or READY state
                    Unsigned Integer. Range: 0 - 24 Default: 1

The max is 24 which might not be sufficient for 1-second buffering, but you may give it a try.