Ntp_timestamp is gradually shifting in the future direction

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

• Hardware Platform (Jetson / GPU) dGPU, RTXP2080
• DeepStream Version 6.0.1
• JetPack Version (valid for Jetson only) N/A
• TensorRT Version 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only) 495.29.05
• Issue Type( questions, new requirements, bugs) questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) describe later
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Description

For our product, I want to use faeture of NTP timestamp by RTSP Sender Report
I am testing the feature with deepstream-test5 app.
My problem is that ntp_timestamp is gradually shifting in the future direction compared to the real time.

Reproduction procedure

1. prepare rtsp camera using rtsp-simple-server

docker run --rm --name rtspserver -itd --network=host aler9/rtsp-simple-server
ffmpeg -re -stream_loop -1 -i sample_1080p_h264.mp4 -vf fps=20/1 -vcodec libx264 -f rtsp rtsp://localhost:8554/mystream

note that sample_1080p_h264.mp4 is from deepstream container.

2. launch deepstream app

nvidia-docker run -itd --rm --name tmp --net=host nvcr.io/nvidia/deepstream:6.0.1-triton bash
nvidia-docker exec -it tmp bash

3. modify and build test5

cd /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps
cp -ir deepstream-test5 deepstream-test5.orig
cd deepstream-test5

# change source0 uri to rtsp://127.0.0.1:8554/mystream
# disable source1, tracker, sink0, sink1.
# set attach-sys-ts-as-ntp=0 in [streammux]
vim configs/test5_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

# modify to print generaeted ntp_timestamp
# at the end of generate_ts_rfc3339_from_ts()
vim deepstream_test5_app_main.c

make

diff …/deepstream-test5.orig/deepstream_test5_app_main.c …/deepstream-test5/deepstream_test5_app_main.c

327a328,331
>   // DEBUG
>   static int cnt = 0;
>   cnt++;
>   if (cnt % 100 == 0)  { printf("ts=%s\n", buf); }

4. run app and check result

./deepstream-test5-app -c configs/test5_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt

The output log is like below.
“Thu Aug 25 00:33:38 2022 **PERF: 20.36 (19.86)” is a PERF output and the time is system clock, which matches the actual time.
“ts=2022-08-25T00:33:35.304Z” is a ntp_timestamp generated from RTSP report, which gradually shifts into the future comapred to the PERF log.

ts=2022-08-25T00:33:35.304Z
Thu Aug 25 00:33:38 2022
**PERF:  20.36 (19.86)
ts=2022-08-25T00:33:35.784Z

...

ts=2022-08-25T00:34:58.514Z
Thu Aug 25 00:34:18 2022
**PERF:  19.94 (19.99)
ts=2022-08-25T00:34:59.954Z

...

ts=2022-08-25T00:42:12.704Z
Thu Aug 25 00:37:23 2022
**PERF:  19.77 (20.00)
ts=2022-08-25T00:42:13.544Z

I used tcpdump and wireshark and confirmed that

  • Sender Report is sent to deepstream-app port
  • The Sender Report’s NTP timestamp matches to the actual time, after a lapse of time.
  • if I set callback for jitterbuffer’s handle-sync, it receives Sender Report that matches the actual time. (I refered to DeepStream 4.0.2 source code)
3

could you share some logs? please do export GST_DEBUG=4 first.

4

@fanzh
LOG-test5-20220825.txt (2.3 MB)

attached stdout+stderr log.
app runs 3 minutes and terminated by ctrl-C.

5

@fanzh Hi, is there any updates?

6

…
ts=2022-08-26T09:57:40.906Z
Fri Aug 26 17:56:45 2022
**PERF: 16.51 (17.92)
log.txt (4.9 KB)
Using the similar steps(difference is gst-rtsp-server RTSP source), I can reproduce this issue, as the logs shown, in 5 minutes test, ts(frame_meta->ntp_timestamp) time is faster 1 minute than real system time , will continue to check.

1 Like
8

@fanzh
Did you find out anything about it?
I recognize this issue as a major bug because it is reproduced in the sample code.

9

it is related to NTP correction strategy, we are investigating, for workaround, please set attach-sys-ts-as-ntp=1.

10

Thank you for your investigating.

for workaround, please set attach-sys-ts-as-ntp=1.

In our product, we would like to use the RTSP source time.
Thanks.

12

@fanzh Hi, could you let me know the status of the investigation?

14

still investigating, will update if there is progress.

15

this issue is because of non-monotonically increasing timestamps due to presence of B frames from the RTSP source, please don’t use B frame in RTSP source, like this:
ffmpeg -re -stream_loop -1 -i /opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 -vf fps=30/1 -vcodec libx264 -profile:v baseline -f rtsp rtsp://localhost:8554/mystream
.

17

I confirmed ntp_timestamp is correct when-profile:v baseline while not correct when-profile:v main or -profile:v high.

Tested with aler9/rtsp-simple-server:v0.17.17 (I found later versions do not send Sender Report)

18

thanks for your sharing, you can disable b frame if using -profile:v main or -profile:v high .
we have fix the B frame problem internally, Please wait for the next release.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.