I am currently working on a deepstream 6.3 C++ project that involves multiple RTSP inputs from the same device, and I require accurate NTP timestamps for each input. Following the instructions provided in the Nvidia guide on NTP timestamps, I set attach-sys-ts to FALSE and called configure_source_for_ntp_sync on my rtspsrc.
While everything works perfectly with a single stream, I encounter an issue when using two streams simultaneously. Occasionally, one of the streams shows an offset of around 1 second between the actual NTP time and the NTP time provided in the Frame Metadata.
In the warnings you can see that even though both ntp timestamps should be nearly identical there is a shift of slightly less than one second between source 0 and 1. I also looked into the rtcp packages that deepstream receives with tshark that got send and all seem to be correct.
Is it possible to use multiple NTP timestamps in this setup? If so, could you provide guidance on how to resolve this issue and what I need to change?
Thank you in advance for your assistance.
Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) Jetson Orin NX • DeepStream Version 6.3 • JetPack Version (valid for Jetson only) 5.1.2 • TensorRT Version 8.5.2.2 • NVIDIA GPU Driver Version (valid for GPU only) 11.4.315 • Issue Type( questions, new requirements, bugs) questions/bug • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) • Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I successfully reproduced the issue using the DeepStream reference application. Thereby, I managed to get the same problem as in my local application, where the outputted NTP timestamps had discrepancies:
Start the streams which the application gets as input. For this, I used the following script:
#!/bin/bash
# Start the rtsp-simple-server in the background
./rtsp-simple-server rtsp-simple-server.yml &
# Give the server a few seconds to start up
sleep 5
# Start the first ffmpeg command in the background
ffmpeg -re -stream_loop -1 -i /home/user/WatermarkVideos_V3/REAL_TV_20210308_01.mp4 -r 50 -c copy -f rtsp rtsp://localhost:8554/teststream1 &
# Start the second ffmpeg command in the background
ffmpeg -re -stream_loop -1 -i /home/user/WatermarkVideos_V3/REAL_TV_202106_01.mp4 -r 50 -c copy -f rtsp rtsp://localhost:8554/teststream2 &
# Wait for all background processes to complete
wait
I started the application with the debugging flags and my own config:
cd /opt/nvidia/deepstream/deepstream-6.3/sources/apps/sample_apps/deepstream-app
GST_DEBUG=3,nvstreammux:6 ./deepstream-app -c /opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/ntp-rtsp-test.txt > ~/1.log 2> ~/1.log
The config (ntp-rtsp-test.txt) which I have written is appended to my reply, as well as the logfile (1.log) 1.log (2.7 MB) ntp-rtsp-test.txt (4.6 KB)
As the doc shown, if you set attach-sys-ts to FALSE, nvstreammux will attach a calculated NTP timestamp, which is not exactly the same with the NTP of rtcp packets. if using attach-sys-ts=TRUE, “This method requires you to synchronize host system to NTP clock. To attach host system time as NTP timestamp”.
This is not exactly what I had in mind. To be a bit more specific, my plan is to not synchronize the two sources against any NTP time, but to use the NTP time over RTSP to associate the Frame to other Frame information which gets provided by another application that also sends the NTP over RTSP timestamp decoded from the RTCP and RTP packages. Specifically, I just want to use the frame_meta->ntp_timestamp inside a probe to match the Frame to the Frame information which I received from another application.
For this, I only need the variable frame_meta->ntp_timestamp to be correct, which in my experience often had discrepancies where both of the streams reported times that could not match given the knowledge that they came from the same pc.
This can also be seen inside the sample application:
Here I ran the streams and the deepstream application on the same device and if we look at the difference in the application runtime given the two streammux outputs, we notice that the application ran for: 7.215469367− 6.487574630=0.727894737 s, while the difference in the NTP time per source is only 41.397−40.933=0.464 s. From my point of view, this should not happen when both streams come from the same pc.
Can you please explain this discrepancy inside the differences?
I tested more than 2 minutes the new ntp of two sources are different, but the difference did not increase over time. here is the log. log-0729.txt (992 Bytes)
2.the ntp correction code is opensource. pleas refer to gst_nvds_ntp_calculator_get_buffer_ntp of \opt\nvidia\deepstream\deepstream\sources\gst-plugins\gst-nvmultistream2\gstnvstreammux_ntp.cpp. the new ntp like “Cur: 2024-07-18T12:13:05.872Z” is got from the following formula.
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks