[JP6.2 R36.4.3] - V4L2 EOF Buffer Timestamp Offset

[ramenparty]

Similar post to this.

I am aware that the V4L2 EOF time stamps are from monotonic clock, from drivers/media/platform/tegra/camera/vi/channel.c.

    chan->queue.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC

                   | V4L2_BUF_FLAG_TSTAMP_SRC_EOF;

    ret = vb2_queue_init(&chan->queue);

I am attempting to correlate these monotonic timestamps to CLOCK_REALTIME since I have other sensors on my system I want to sync with. At user-space frame dequeue, I calculated offset_ns by doing

offset_ns = clock_gettime(CLOCK_REALTIME, &real_ts) - clock_gettime(CLOCK_MONOTONIC, &mono_ts)

and then adding that offset to V4L2 mono buffer timestamp. I see that my converted timestamp is ~20 seconds ahead of CLOCK_REALTIME.

Next, in an attempt to understand where the offset is coming from, I printed mono time immediately after the last buffer from V4L2 is received. I confirmed that I also see this 20 second offset in V4L2.

v4l2-ctl -V --stream-mmap --stream-count=10 -d /dev/video0 --verbose; python3 -c 'import time; print(time.monotonic_ns()/1.0e9);'
VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
Format Video Capture:
        Width/Height      : 1920/1536
        Pixel Format      : 'UYVY' (UYVY 4:2:2)
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 5898240
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             : 
                VIDIOC_REQBUFS returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq:      0 bytesused: 5898240 ts: 8420.476914 (ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      1 bytesused: 5898240 ts: 8420.516914 delta: 40.000 ms (ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      2 bytesused: 5898240 ts: 8420.556915 delta: 40.001 ms (ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq:      3 bytesused: 5898240 ts: 8420.596915 delta: 40.000 ms (ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq:      4 bytesused: 5898240 ts: 8420.636915 delta: 40.000 ms fps: 25.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      5 bytesused: 5898240 ts: 8420.676916 delta: 40.001 ms fps: 25.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      6 bytesused: 5898240 ts: 8420.716916 delta: 40.000 ms fps: 25.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq:      7 bytesused: 5898240 ts: 8420.756916 delta: 40.000 ms fps: 25.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq:      8 bytesused: 5898240 ts: 8420.796917 delta: 40.001 ms fps: 25.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      9 bytesused: 5898240 ts: 8420.836917 delta: 40.000 ms fps: 25.00 (ts-monotonic, ts-src-eof)

8401.219419104

Other posts mention looking at the offset to the system time looking at offset_ns

in cat /sys/devices/system/clocksource/clocksource0/offset_ns. However, I do not have this in my path.

So I want to understand what the best way is to correlate the monotonic timestamp from RTCPU with clock realtime on my system.

Thanks

[carolyuu]

*** Please note that this reply is generated by LLM automatically ***

To correlate the monotonic timestamp from V4L2 (which is based on the RTCPU clock) with CLOCK_REALTIME on your Jetson AGX Orin system, you need to understand the relationship between these two clocks.

The monotonic clock (RTCPU) is a clock that increments at a constant rate, unaffected by system time changes, such as NTP adjustments or leap seconds. On the other hand, CLOCK_REALTIME is the system’s wall clock time, which can be adjusted by NTP or other mechanisms.

To correlate these two clocks, you need to determine the offset between them. Unfortunately, the /proc/rtc file is not available on your system, which would have provided the offset between the RTCPU clock and the system time.

However, you can use the debugfs interface to access the clock information on your Jetson AGX Orin device. Specifically, you can use the clk_tree file to get the clock tree information, which includes the offset between the RTCPU clock and the system time.

To do this, follow these steps:

1. Enter the following command to launch debugfs and check the clock:

$ sudo -s
# cat /sys/kernel/debug/bpmp/debug/clk/clk_tree

This will display the clock tree information, including the offset between the RTCPU clock and the system time.

2. Look for the rtc clock in the output, which should be in the format rtc: <offset>. The <offset> value represents the offset between the RTCPU clock and the system time in nanoseconds.
3. Use this offset value to adjust your monotonic timestamp to correlate it with CLOCK_REALTIME.

For example, if the offset value is 20000000000 (20 seconds), you can add this value to your monotonic timestamp to get the corresponding CLOCK_REALTIME timestamp.

Note that this offset value may change over time due to system time adjustments, so you may need to periodically recheck the offset value to ensure accurate correlation between the two clocks.

In summary, to correlate the monotonic timestamp from V4L2 with CLOCK_REALTIME on your Jetson AGX Orin system, you need to:

• Use debugfs to access the clock tree information.
• Look for the rtc clock in the output and extract the offset value.
• Use this offset value to adjust your monotonic timestamp to correlate it with CLOCK_REALTIME.

This approach should provide a more accurate correlation between the two clocks compared to your initial method of calculating the offset based on the last buffer timestamp.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

[ramenparty]

rtc clock in the output of

$ sudo -s
# cat /sys/kernel/debug/bpmp/debug/clk/clk_tree

is not present.

[ShaneCCC]

Please reference to below topic for JP6.

How to get the clock source offset_ns on Jetpack 6

For JP5 , the offset_ns in sysnode is calculated during system boot , After calculation, it is a fixed value. you can check the code in kernel-5.10/kernel/time/timekeeping.c For JP6 , The API provided in last comment is calculated dynamically. I have tested that the dynamic range is not very large： uint64_t readOffsetNs() { unsigned long raw_nsec, tsc_ns; unsigned long cycles, frq; struct timespec tp; asm volatile("mrs %0, cntfrq_el0" : "=r"(frq)); …

[ramenparty]

Thanks for that info @ShaneCCC Few clarifying questions:

Is the buffer timestamp from V4L2 raw monotonic? I see you’re calculating the offset with reference to raw monotonic. The buffer flags list V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC same as my VI driver, but no reference whether RAW or not.

Assuming it is not raw, I am following the method below.

Here is the flow I am using to convert EOF from V4L2 to CLOCK_REALTIME. Is this the correct approach?

1. Step 1 Read All Clocks

   1. Read ARM timer
   2. read raw monotonic
   3. read system time

2. Calculate clock offset

   1. Find ARM-to-RAW offset: offset = cntvct_ns - raw_now_ns

3. Convert camera timestamp

   1. Map V4L2 EOF to RAW domain: eof_raw = eof_v4l2 - offset
   2. Convert RAW to realtime: eof_realtime = eof_raw + (real_now_ns - raw_now_ns)

[ShaneCCC]

Suppose below ts could be SOF instead of EOF. Maybe print it in vi5_fops.c to confirm first.

Then the ts from vi5_fops.c is CLOCK_MONOTONIC_RAW of RTCPU time. Map to kernel CLOCK_MONOTONIC_RAW by subtract offset_ns. (system kernel raw = SOF - readOffsetNs)

cap dqbuf: 0 seq:      4 bytesused: 7558272 ts: 603605.950964 delta: 33.332 ms fps: 30.00 (ts-monotonic, ts-src-eof)

[ramenparty]

Thanks @ShaneCCC I confirmed the V4L2 timestamp I am grabbing is EOF.

cap dqbuf: 3 seq:      3 bytesused: 5898240 ts: 8420.596915 delta: 40.000 ms (ts-monotonic, ts-src-eof)

I am mapping V4L2 buffer timestamp to monotonic raw as you are mentioning.

The V4L2 timestamp is in the RTCPU domain, so:

1. I map it to kernel CLOCK_MONOTONIC_RAW by subtracting the RTCPU←→kernel offset (from cntvct_el0/cntfrq_el0 vs clock_gettime(CLOCK_MONOTONIC_RAW)), to get offset_ns = EOF - offset_ns.

2.then derive CLOCK_REALTIME using a co-sampled (realtime_now − mono_raw_now). With that, the camera timestamps should line up with other sensors disciplined to REALTIME.

Can you confirm this is correct?

[ShaneCCC]

Suppose it’s correct.