NTP timestamp value always 0

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) RTX 3090
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

when i change attach-sys-ts to 0, frame’s NTP timestamp always 0.

the wireshark result as below, It does get the rtcp package, but my deepstream app not

when i launch my deepstream app, i has some warnings, it seems streammux does’t get ntp timestamp at the beginning

Warning: gst-library-error-quark: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2023-11-22T07:00:28.407Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5): gstnvstreammux_ntp.cpp(108): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:Stream-muxer

which sample are you testing? which the whole media pipeline? could you share the detailed reproducing steps? how do you know frame’s NTP timestamp always 0?

my app was based on python apps/­deepstream-test3, I add a prob on tracker_src_pad, and get the frame_ts, like

frame_ts = int(frame_meta.ntp_timestamp /1E6)

when streammux.set_property(‘attach-sys-ts’, 1), it print the normal timestamp, but when a change attach-sys-ts to 0, the frame_ts always 0.

I do want to get the source timestamp, I use the command blow to print rtcp ntp timestamp

sudo tshark -i eno1  -f "src host <my source ip>" -Y "rtcp" -T fields -e  rtcp.timestamp.ntp

It seems that has got the ntp timetamp, But the print time is 8 hours later than my system time, which is 2pm

Nov 22, 2023 22:42:35.693338234 CST
Nov 22, 2023 22:42:44.129599044 CST
Nov 22, 2023 22:42:51.226086076 CST
Nov 22, 2023 22:42:55.792491268 CST

So I don’t know what happen, my deepstream app always print the 0 ntp timestamp

could you share more logs? please do the following setting first, then run the app again, then share log. you can use zip to compress the log. Thanks!
export GST_DEBUG=nvstreammux:6
export GST_DEBUG_FILE=/tmp/gst.log

gst.zip (112.9 KB)
Hi! there is my gst.log, Thx!

sorry. I can’t find any “NTP” printing in the logs, even there is no "seems to be out of NTP sync " you mentioned. please use the new method. Thanks!

  1. set env variable.
    export GST_DEBUG=6
    export GST_DEBUG_FILE=/tmp/gst.log
  2. after running in a while, execute /tmp/gst.log |grep nvstreammux >1.log.
  3. zip and share 1.log.

about “seems to be out of NTP sync”, it should be related to “8 hours later than my system time”.

1.zip (449.1 KB)

Hi i found the 1.log in line 56:
0:00:07.694262966 e[335m 3748e[00m 0x7f61bc007920 e[37mDEBUG e[00m e[00m nvstreammux_ntp gstnvstreammux_ntp.cpp:66:gst_nvds_ntp_calculator_new:<Stream-muxer>e[00m Setting NTP correction mode disabled.
that means the frame-duration default to -1? I have not set streammux frame-duration. But the ds streammux doc shows frame-duration default to 0.

Anyway, i set set streammux frame-duration to 0 , the 2.log in line 52 shows correction mode are setting to automatic mode, the ntp_time_epoch_ns always 1970-01-01T00:00:00.000Z(0)

0:00:06.929780127 e[332m 4033e[00m 0x7f7424007920 e[37mDEBUG e[00m e[00m nvstreammux_ntp gstnvstreammux_ntp.cpp:69:gst_nvds_ntp_calculator_new:<Stream-muxer>e[00m Setting NTP correction mode automatic.
2.zip (102.3 KB)

I think the problem comes from here

     /** Add ntp timestamp*/
     GstNvDsNtpCalculator* ntp_calc = sinkPad->get_ntp_calc(
             mux->ntp_calc_mode, mux->frame_duration);

     if (ntp_calc && !gst_nvds_ntp_calculator_have_ntp_sync_values (ntp_calc)) {
        GstQuery *nvquery = gst_nvquery_ntp_sync_new ();
        GstClockTime ntp_time_epoch_ns = 0, ntp_frame_timestamp = 0;
        GstClockTime avg_frame_time = 0;

        gst_pad_peer_query ((GstPad*)(sinkPad->wrapped), nvquery);
        _NtpData ntpdata;
        memset(&ntpdata, 0, sizeof(_NtpData));
        gst_nvquery_ntp_sync_parse (nvquery, &ntpdata);

        if (ntpdata.ntp_time_epoch_ns > 0) {
          ntp_time_epoch_ns = ntpdata.ntp_time_epoch_ns;
          ntp_frame_timestamp = ntpdata.frame_timestamp;
          avg_frame_time = ntpdata.avg_frame_time;
        }
        gst_nvds_ntp_calculator_add_ntp_sync_values (ntp_calc,
            ntp_time_epoch_ns, ntp_frame_timestamp, avg_frame_time);
        gst_query_unref (nvquery);
      }

      GstClockTime ntp_ts = 0;
      if(ntp_calc) {
       ntp_ts =
          gst_nvds_ntp_calculator_get_buffer_ntp (ntp_calc,
                  GST_BUFFER_PTS (buffer));
     }

the source code comes from function gst_nvstreammux_chain , from deepstream 6.3 gst-plugins gst-nvmultistream2 gstnvstreammux.cpp, the core func is gst_pad_peer_query or gst_nvquery_ntp_sync_parse, ntpdata was not successfully parsed

Thanks for the sharing. I will check.

“1970-01-01T00:00:00.000Z” is got from gst_nvquery_ntp_sync_parse, which will parse ntp in jitterbuffer’s data callback. I will try tor reproduce this issue. to narrow down this issue, could you try a local rtsp source which has no timezone issue?

1 Like

please refer to all steps in ntp-timestamp-in-deepstream. especially please add configure_source_for_ntp_sync (uri_decode_bin);.

1 Like

gratitude! My version is deepstream 6.2, and the configure_source_for_ntp_sync python binding is not available yet. I successfully obtained the rtcp ntp timestamp by referring to this link.

Hi! @fanzh I met a new problems, the same deepstream python code for different rtsp source,
the rtsp source1 for my deepstream code can get the rtcp ntp timestamp. But when i change a new rtsp source2 from an new camera, the ntp timestamp value turn to 0.

The strange thing is that the tshark packet capture result is normal

  • the rtsp source1

    sudo tshark -i eno1 -f "src host <rtsp source1>" -Y "rtcp"
    Running as user "root" and group "root". This could be dangerous.
    Capturing on 'eno1'
      331 0.872048653 <rtsp source1> → <my host> RTCP 114 Sender Report   Source description
     2547 6.847002735 <rtsp source1> → <my host> RTCP 114 Sender Report   Source description
     4111 12.045429221 <rtsp source1> → <my host> RTCP 114 Sender Report   Source description
    
  • the rtsp source2

    sudo tshark -i eno1 -f "src host <rtsp source1>" -Y "rtcp"
    Running as user "root" and group "root". This could be dangerous.
    Capturing on 'eno1'
      331 0.872048653 <rtsp source1> → <my host> RTCP 114 Sender Report   Source description
     2547 6.847002735 <rtsp source1> → <my host> RTCP 114 Sender Report   Source description
     4111 12.045429221 <rtsp source1> → <my host> RTCP 114 Sender Report   Source description
    

could you share the test steps? could you share the running log according to the method mentioned on Nov 22.

@fanzh

  • step 1
    use gst-launch-1.0 to play rtsp source on my machine
  • step 2
    use tshark to caputre rtcp package

rtsp_source1.log is the Nov 22 source, and rtsp_source2.log from my new camrea

rtsp_source1.log (637.5 KB)
rtsp_source2.log (2.0 MB)

  • streammux config
        streammux.set_property('live-source', 1)
        streammux.set_property('attach-sys-ts', 0)
        streammux.set_property('frame-duration', 0)
    
  • source bin func
    GST_HELPER_LIB = ctypes.CDLL(
        "/opt/nvidia/deepstream/deepstream/lib/libnvdsgst_helper.so"
    )
    def create_source_bin(index, uri):
        print('Creating source bin')
    
        # Create a source GstBin to abstract this bin's content from the rest of the
        # pipeline
        bin_name = 'source-bin-%02d' % index
        print(bin_name)
        nbin = Gst.Bin.new(bin_name)
        if not nbin:
            sys.stderr.write(' Unable to create source bin \n')
    
        # Source element for reading from the uri.
        # We will use decodebin and let it figure out the container format of the
        # stream and the codec and plug the appropriate demux and decode plugins.
        uri_decode_bin = Gst.ElementFactory.make('uridecodebin', 'uri-decode-bin')
        if not uri_decode_bin:
            sys.stderr.write(' Unable to create uri decode bin \n')
        # We set the input uri to the source element
        uri_decode_bin.set_property('uri', uri)
        GST_HELPER_LIB.configure_source_for_ntp_sync(hash(uri_decode_bin))
        # Connect to the 'pad-added' signal of the decodebin which generates a
        # callback once a new pad for raw data has been created by the decodebin
        uri_decode_bin.connect('pad-added', cb_newpad, nbin)
        uri_decode_bin.connect('child-added', decodebin_child_added, nbin)
    
        # We need to create a ghost pad for the source bin which will act as a proxy
        # for the video decoder src pad. The ghost pad will not have a target right
        # now. Once the decode bin creates the video decoder and generates the
        # cb_newpad callback, we will set the ghost pad target to the video decoder
        # src pad.
        Gst.Bin.add(nbin, uri_decode_bin)
        bin_pad = nbin.add_pad(
            Gst.GhostPad.new_no_target('src', Gst.PadDirection.SRC))
        if not bin_pad:
            sys.stderr.write(' Failed to add ghost pad in source bin \n')
            return None
        return nbin
    

the rtsp source2 from new camera tshark detail is that

sudo tshark -i eno1 -f "src host <rtsp source2>" -Y "rtcp"
 -T fields -e  rtcp.timestamp.ntp
Running as user "root" and group "root". This could be dangerous.
Capturing on 'eno1'
Dec  5, 2023 17:29:09.425999999 CST
Dec  5, 2023 17:29:14.424999999 CST
Dec  5, 2023 17:29:19.422999999 CST
Dec  5, 2023 17:29:24.420999999 CST

from the log, all NTP values are “1970-01-01T00:00:00.000Z(0)”. could you share a RTCP package screenshot as you did in the issue description?

Hi @fanzh ! this is my screenshot