Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) Jetson Xavier AGX • DeepStream Version deepstream 6.2 (editted) • JetPack Version (valid for Jetson only) 5.1 [L4T 35.2.1] • TensorRT Version NONE • NVIDIA GPU Driver Version (valid for GPU only) • Issue Type( questions, new requirements, bugs) Questions • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) • Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
Before starting, I would like to notice that I already set attach-sys-ts to FALSE on nvstreammux to attach at RTSP source. Also checked the NTP timestamp from the framemeta and it works normally at the first moment.
Hi,
SITUATION STARTS HERE.
I was checking NTP_timestamp with the 1) systemtime of the main server & 2) the buffer_pts of the framemeta data together.
At the first moment, the increasement of the systemtime & NTP_timestamp & the buffer_pts were exactly same.
However, after the long operation, including a multiple times of pause with gst_element_set_state() gst function, the NTP_timestamp values suddenly jumps up one time and remain permenantly. It seems the NTP_timestamp’s base time is updated.
QUESTION. how-to-correct-ntp-timestamp-by-using-every-rtcp-sender-report
According to this topic, NTP_timestamp value seems to consists of 2 values.
NTP_timestamp = basetime + buffer_pts
where the basetime is the synchronized value from RTCP sender report at the first configure_source_for_ntp_sync.
According to the explanation, the basetime shouldn’t be updated.
However, what it shown in my case, within the long period, it updates.
Contrary to this, when I call configure_source_for_ntp_sync function during the pipeline is running or while it is in the pause state, the basetime of NTP_timestamp does not updates.
So my question is, When does the NTP_timestamp’s basetime ( NTP_timestamp - buffer_pts ) updates? If I can update the basetime, how can I do it??
BACKGROUNDS
I am currently synchronizing the rtsp stream frame to the other sensor data in MainServerUnit.
To synchronize it together, I try to obtain the delay of each stream, since when the video frame was defined.
Until now, what I did to calculate the latency is
Calculate and define the system_time_difference between the MainServerUnit and CameraServerUnit when the RTCP sender report packet received.
Define the NTP_timestamp of each framemeta and MainServerUnit’s system time at the same moment.
compensate the NTP_timestamp with the system_time_difference
subtract to the MainServerUnit’s system time.
This value will be the delayed time of each frame.
Our system consists of MainServerUnit & 4 Camera streams through RTSP (incl. RTCP SR) via CameraServerUnit CAM1 -----gmsl------| CAM2 -----gmsl------|------ CameraServerUnit ----- RTSP / RTCP -----> MainServerUnit CAM3 -----gmsl------| CAM4 -----gmsl------|
could you share the related log? you can run the app with “export GST_DEBUG=3,nvstreammux:6”.
NTP_timestamp calculation is opensouce in the latter versions. please refer to apply_correction_if_needed_rtcp in DS6.3 or DS7.1. ntp_time will updated to the new value if the the new ntp_time is “is consistent with prev SR”. the final ntp value is calculated based on ntp_time, buf_pts and ntp_frame_timestamp.
Hi! thanks for the reply!
I will try and share the logged situation with your suggested condition when I get it.
Our system is limited to the deepstream 6.2 version
Where can I look into the function of the apply_correction_if_needed_rtcp?
I have another situation to handle, that the NTP time changes(initializes) comparing to the prev SR due to the CameraServerUnit reboots.
log-Camera_Server_rebooted_at_line9300.txt (3.6 MB)
I just export GST_DEBUG=2,nvstreammux:6, as the fixme output was so much.
Within the attached file, I found that after rebooting the CameraServerUnit while the MainServerUnit keep try to reconnect the stream, (where the log shows up to reconnect from line 9300) apply_correction_if_needed_rtcp output shows up as below.
Will the initialized system time of the CameraServerUnit affects the ntp correction?
If it is, is there a way to fix it without restarting the process?
For Your Information, some information about my cout in the log file.
CAM-SYS-OSTIME_DIFF : calculated the differene system time between the CameraServer and MainServer by receiving the RTCP SR at another thread.
SYSTIME : current time when the frame is loaded.
NTPtimestamp : loaded from NVDSFrameMeta
Buffer_PTS : also loaded from NVDSFrameMeta
NTP_clock_basetime : calculated by NTPtimestamp - Buffer_PTS
log-250411-extracted.txt (893.7 KB)
After rebooting the CameraServerUnit, keep pausing the rtspsrc and wait to resume several times, the NTP_timestamp shifted to backward.
It shows on
Source 0 [ NTP_timestamp shifts from 1395982898501 (line 488) ->1395928237422 (line 985)]
Source 2 [ NTP_timestamp shifts from 1395982898574 (line 489) ->1395928237916 (line 1083)]
while the buffer_pts not shifts.
this log is printed in apply_correction_if_needed_rtcp. it is because the new ntp does not meet requirement after calculating. could you share the whole media pipeline? ntp calclulation of nvstreamux is opensource after DS6.2. please download the latter versions by referring to this link.
How does CameraServerUnit and MainServerUnit affect the ntp？NTP correction is based on incoming NTP values. please fix the issue of the abnormal ntp first.
I am using DS6.2 currently. I miswrote before as DS6.1.
I defined the pipeline through the DSL;Deepstream Service Library.
It has 4 rtspsrcbin which gathers into nvstreammux, and this connects to the pgie(segmentation).
I have custompph which pads right after to the pgie(segmentation).
The custompph checks the result of pgie(segmentation) and its ntptime, to check when this frame was sent by CameraServerUnit.
[Situation that affects the ntp]
CameraServerUnit sends the RTCP sender report to the MainServerUnit, where deepstream processor is running.
So the ntptime in the RTCP sender report will based on the CameraServerUnit system time.
My problem occurs when rebooting the CameraServerUnit, while the deepstream processor is still running at the MainServerUnit.
After CameraServerUnit reboots, its systemtime will be initialized to the past, then the ntptime at the rtcp sender report will be changed.
I am trying to still update the ntp timestamp when the inconsistency happens in the above situation. Will this be possible without destroying the pipeline?
Is there a way to loose the condition of requirement?
please find the following code in apply_correction_if_needed_rtcp of /opt/nvidia/deepstream/deepstream-7.1/sources/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp
Thank you,
I updated to deepstream6.3 at jetpack5.1.
After update, I removed the condition, rebuild it and installed.
Now the NTP time is updated always.