What's the detail implementation of function "configure_source_for_ntp_sync"

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Tesla T4
• DeepStream Version 5.0.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 10.2
• Issue Type( questions, new requirements, bugs) Questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I want to know what did deepstream do to the “gstrtspsrc” element in function “configure_source_for_ntp_sync”.

I have another gstreamer pipeline reading the same RTSP stream with DeepStream (RTCP SR already enabled). I want this two pipeline to have same ntp-timestamp so that I can syncronize the meta information like bounding boxes. FYI, my own pipeline looks like this:

gst-launch-1.0 rtspsrc location="rtsp://rtsp_stream:554/lab1" protocols=GST_RTSP_LOWER_TRANS_TCP ! rtph264depay ! h264parse ! queue ! splitmuxsink location="lab1-splitmuxsink_%05d.ts" muxer=mpegtsmux max-size-time=10000000000

However, I found that my pipeline’s gstreamer timestamp is drifting, i.e., the difference between multifilesink’s message’s timestamp value & gstjitterbuffer’s gstreamer time (gstreamer_time += ((sr_ext_rtptime - base_rtptime) * GST_SECOND / clock_rate)) upon “handle_sync” signal is steadily growing (from seconds to tens of seconeds).

My implementation of time syncronization between gsteamer & RTCP SR is obtained from DeepStream 4.0.1 SDK since this is the only open source version.

So I want to know how DeepSteram application deal with this time drifting internally between gstreamer buffer timestamp & RTCP SR’s gstreamer timestamp.

Please give me some instruction.

As I dig into the detail implementation with help of Gstreamer Dot graph debug option, the source pipeline of DeepStream 5.0.0 test5 app’s rtsp source is like this.

The most important part is “GstRTPSrc” and “GstRtpJitterBuffer” element.

For GstRtpSrc, I set the following properties:

  /* we set the input rtsp to the source element */
  g_object_set (G_OBJECT (appCtx->source), "location", appCtx->uri, NULL);
  g_object_set (G_OBJECT (appCtx->source), "latency", 2000, NULL);
  g_object_set (G_OBJECT (appCtx->source), "drop-on-latency", TRUE, NULL);
  /** 
   * none (0) – Only use RTP timestamps
   * slave (1) – Slave receiver to sender clock
   * buffer (2) – Do low/high watermark buffering
   * auto (3) – Choose mode depending on stream live
   * synced (4) – Synchronized sender and receiver clocks
   * Follow GstRtpJitterBuffer property from DeepStream Pipeline.
  */
  g_object_set (G_OBJECT (appCtx->source), "buffer-mode", 4, NULL);

For GstRtpJitterBuffer, I set following properties:

static void
rtp_bin_new_jitter_buffer (GstBin  *rtpbin,
               GstElement *jitterbuffer,
               guint       session,
               guint       ssrc,
               gpointer    user_data)
{
  // Follow GstRtpJitterBuffer property from DeepStream Pipeline.
  g_object_set (G_OBJECT (jitterbuffer), "max-rtcp-rtp-time-diff", -1, NULL);
  g_object_set (G_OBJECT (jitterbuffer), "faststart-min-packets", 1, NULL);
  /** Request for the `handle-sync` signal
   * in jitterbuffer to lawfully tap
   * RTCP Sender Report
   */
  g_signal_connect(G_OBJECT(jitterbuffer), "handle-sync",
                   G_CALLBACK(rtp_bin_handle_sync),
                   user_data);
}

static void 
cb_rtsp_src_elem_added (GstBin *bin, GstElement *element, gpointer u_data) {
  if(strstr(GST_ELEMENT_NAME(element), "manager"))
  {
      /** RtpBin: Request new-jitterbuffer signal */
      g_signal_connect(G_OBJECT(element), "new-jitterbuffer",
                       G_CALLBACK(rtp_bin_new_jitter_buffer),
                       u_data);
  }
}

int
main (int   argc,
      char *argv[])
{
  ...
  /* rtcp sender report handler */
  g_signal_connect (G_OBJECT (appCtx->source), "element-added", G_CALLBACK(cb_rtsp_src_elem_added), appCtx);
  ...
}

Then the time drifting behavior seems disappear. May be this is related to GstRtpJitterBuffer’s “max-rtcp-rtp-time-diff” property. In order to ensure synchronization between sender & receiver, we have to rely solely on RTCP sender report, no matter how much the diff is.