metadata / msgconv / msgbroker / Codec/ ds-app
1. Sample of adding metadata
https://devtalk.nvidia.com/default/topic/1061083/deepstream-sdk/attaching-custom-type-metadata-to-gstreamer-buffer-on-src-pad-causing-sudden-crash/post/5374690/#5374690
2. Sample of customizing gst-dsexample:
https://devtalk.nvidia.com/default/topic/1061422/deepstream-sdk/how-to-crop-the-image-and-save/post/5375174/#5375174
3. Sample config file of running single RTSP source:
https://devtalk.nvidia.com/default/topic/1058086/deepstream-sdk/how-to-run-rtp-camera-in-deepstream-on-nano/post/5366807/#5366807
5. Sample of accessing NvBufSurface
https://devtalk.nvidia.com/default/topic/1061205/deepstream-sdk/rtsp-camera-access-frame-issue/post/5377678/#5377678
6.Use GST_PAD_PROBE_DROP macro to drop the buffer in the attached probe.
Refer to Pipeline manipulation for the example
static GstPadProbeReturn
event_probe_cb (GstPad * pad, GstPadProbeInfo * info, gpointer user_data)
{
return GST_PAD_PROBE_DROP;
}
7.Add dsexample in ds-test1 app
https://devtalk.nvidia.com/default/topic/1065406/deepstream-sdk/enable-dsexample-in-test-app/?offset=3#5398407
8. Optical flow
Optical flow functionality is supported only on Jetson AGX Xavier and Turing GPUs T4 / RTX 2080 etc. It won’t work on Jetson Nano and GTX
9. How can we set “drop-frame-interval” more than 30 ?
a. Find and download “L4t source” from https://developer.nvidia.com/embedded/downloads#?search=source
gst-nvvideo4linux2_src.tbz2 is in public_sources.tbz2
b. Apply the patches “0001-gstv4l2dec-Fix-high-CPU-usage-in-drop-frame.patch” and " 0002-gst-v4l2dec-Increase-Drop-Frame-Interval.patch"
c. Build a new libgstnvvideo4linux2.so and replace /usr/lib/$(ARCH)/gstreamer-1.0/libgstnvvideo4linux2.so
0001-gstv4l2dec-Fix-high-CPU-usage-in-drop-frame.patch
From 5d8d5a0977473eae89c0f310171d2c7060e24eb6 Mon Sep 17 00:00:00 2001
From: vpagar <vpagar@nvidia.com>
Date: Thu, 5 Dec 2019 16:04:02 +0530
Subject: [PATCH 1/2] gstv4l2dec: Fix high CPU usage in drop-frame
In case of drop-frame-interval, in LL v4l2 implementation a
thread in low level v4l2 lib which sends buffer to block and
a callback thread spins between themselves causing high CPU percentage
usage over the perid.
This CL drops frame at the gstreamer level and LL v4l2 does not handle
dropping frames.
Unit-Test:
gst-launch-1.0 multifilesrc location= sample_720p.h264 \
! h264parse ! nvv4l2decoder drop-frame-interval=3 ! fakesink
and check CPU percentage usage in htop, it should stay stable.
Bug 200562189
Change-Id: I9af22745501d6a9892c341cb640dac16f8641763
---
gst-v4l2/gstv4l2videodec.c | 23 ++++++++++++++++++++++-
gst-v4l2/gstv4l2videodec.h | 1 +
2 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/gst-v4l2/gstv4l2videodec.c b/gst-v4l2/gstv4l2videodec.c
index 5531f9d..f8c62f2 100644
--- a/gst-v4l2/gstv4l2videodec.c
+++ b/gst-v4l2/gstv4l2videodec.c
@@ -593,6 +593,9 @@ gst_v4l2_video_dec_start (GstVideoDecoder * decoder)
gst_v4l2_object_unlock (self->v4l2output);
g_atomic_int_set (&self->active, TRUE);
self->output_flow = GST_FLOW_OK;
+#if USE_V4L2_TARGET_NV
+ self->decoded_picture_cnt = 0;
+#endif
return TRUE;
}
@@ -704,6 +707,11 @@ gst_v4l2_video_dec_set_format (GstVideoDecoder * decoder,
}
}
+#if 0
+ /* *
+ * TODO: From low level library remove support of drop frame interval after
+ * analyzing high CPU utilization in initial implementation.
+ * */
if (self->drop_frame_interval != 0) {
if (!set_v4l2_video_mpeg_class (self->v4l2output,
V4L2_CID_MPEG_VIDEODEC_DROP_FRAME_INTERVAL,
@@ -712,6 +720,7 @@ gst_v4l2_video_dec_set_format (GstVideoDecoder * decoder,
return FALSE;
}
}
+#endif
#ifndef USE_V4L2_TARGET_NV_CODECSDK
if (self->disable_dpb != DEFAULT_DISABLE_DPB) {
if (!set_v4l2_video_mpeg_class (self->v4l2output,
@@ -1141,10 +1150,21 @@ gst_v4l2_video_dec_loop (GstVideoDecoder * decoder)
gst_caps_unref(reference);
}
- ret = gst_video_decoder_finish_frame (decoder, frame);
+#if USE_V4L2_TARGET_NV
+ if ((self->drop_frame_interval == 0) ||
+ (self->decoded_picture_cnt % self->drop_frame_interval == 0))
+ ret = gst_video_decoder_finish_frame (decoder, frame);
+ else
+ ret = gst_video_decoder_drop_frame (GST_VIDEO_DECODER (self), frame);
if (ret != GST_FLOW_OK)
goto beach;
+
+ self->decoded_picture_cnt += 1;
+#else
+ ret = gst_video_decoder_finish_frame (decoder, frame);
+#endif
+
} else {
GST_WARNING_OBJECT (decoder, "Decoder is producing too many buffers");
gst_buffer_unref (buffer);
@@ -1696,6 +1716,7 @@ gst_v4l2_video_dec_init (GstV4l2VideoDec * self)
self->skip_frames = DEFAULT_SKIP_FRAME_TYPE;
self->nvbuf_api_version_new = DEFAULT_NVBUF_API_VERSION_NEW;
self->drop_frame_interval = 0;
+ self->decoded_picture_cnt = 0;
self->num_extra_surfaces = DEFAULT_NUM_EXTRA_SURFACES;
#ifndef USE_V4L2_TARGET_NV_CODECSDK
self->disable_dpb = DEFAULT_DISABLE_DPB;
diff --git a/gst-v4l2/gstv4l2videodec.h b/gst-v4l2/gstv4l2videodec.h
index 50d07c5..5015c30 100644
--- a/gst-v4l2/gstv4l2videodec.h
+++ b/gst-v4l2/gstv4l2videodec.h
@@ -71,6 +71,7 @@ struct _GstV4l2VideoDec
GstFlowReturn output_flow;
guint64 frame_num;
#ifdef USE_V4L2_TARGET_NV
+ guint64 decoded_picture_cnt;
guint32 skip_frames;
guint32 drop_frame_interval;
gboolean nvbuf_api_version_new;
--
2.17.1
0002-gst-v4l2dec-Increase-Drop-Frame-Interval.patch
From 52665605036144ac20628c95e52fdd82edae71b9 Mon Sep 17 00:00:00 2001
From: vpagar <vpagar@nvidia.com>
Date: Wed, 11 Dec 2019 11:56:25 +0530
Subject: [PATCH 2/2] gst-v4l2dec: Increase Drop Frame Interval
Bug 200575866
Change-Id: If5576683c0fad95832595838d032d3145b88ea36
---
gst-v4l2/gstv4l2videodec.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gst-v4l2/gstv4l2videodec.c b/gst-v4l2/gstv4l2videodec.c
index f8c62f2..00d7740 100644
--- a/gst-v4l2/gstv4l2videodec.c
+++ b/gst-v4l2/gstv4l2videodec.c
@@ -1807,7 +1807,7 @@ gst_v4l2_video_dec_class_init (GstV4l2VideoDecClass * klass)
"Drop frames interval",
"Interval to drop the frames,ex: value of 5 means every 5th frame will be given by decoder, rest all dropped",
0,
- 30, 30,
+ G_MAXUINT, G_MAXUINT,
G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS | GST_PARAM_MUTABLE_READY));
g_object_class_install_property (gobject_class, PROP_NUM_EXTRA_SURFACES,
--
2.17.1
10.use deepstream-app option
refer https://devtalk.nvidia.com/default/topic/1069070/deepstream-sdk/labels-disappear-with-multiple-sources/post/5415523/#5415523