VI Capture system time offset

Hi,
I’ve noticed that buffers captured have a timestamp that is about 4 seconds ahead of system time.

Is there any way to change this?

I added a bit of debugging prints and I see that the SOF timestamp is already offset in tegra_ivc_vi_notify_get_capture_status()

See the system timestamp output on the left, and the SOF timestamp displayed on the right in nanoseconds… there is a 4 second delta.

[ 116.856140] tegra-ivc-vi-notify ivc-b000000.rtcpu:vinotify@12c0: tegra_ivc_vi_notify_get_capture_status: 121247362176

Thanks

hello phabsch,

thanks for point out,
may I know which JetPack release you’re working with?
thanks

Running TX2, 28.1.

If my understanding is correct, these values are populated by vinotify which comes from the RTCPU. I don’t understand where these would be synchronized. Is there a way of ensuring they have the same time? Or a way of determining the offset so that we can compensate in software?

hello phabsch,

you’re right, we’re using the timestamp of sensor start of frame to make it as v4l2 buffer timestamp.
since there are stream setup and buffer initialization stages, time offset between v4l2 buffer and system time is expect.
you can enable the dynamic debug prints of vi_notify.c to confirm the SOF timestamp.
also, may I know what’s your use-case about timestamp synchronization.
thanks

Hi JerryChang,
Thanks for the reply. While I understand that you are saying there is a delay between SOF and when we get the buffer, buy why is it 4 seconds IN THE FUTURE?

Note in my original post:
kernel time: 116.856140 sec
SOF time: 121.247362176 sec

We have a system with an IMU and are trying to synchronize physical movement with camera vision.

hello phabsch,

we had several kernel fixes integrate after 28.1,
may I know is it possible to move your works to our latest release?
thanks

Hi Jerry,

I just tried jetpack 3.3 on my Jetson dev kit and the ov5693 and experience the same problem. While I don’t have my debug code in place to see the timestamps per frame, I do see that stopping the streaming displays messages on the console.

./yavta -c10 -n4 -F/tmp/test.raw /dev/video0

[ 4918.955558] tegra-vi4 15700000.vi: timestamp sof 4929813128064 eof 4929840366880 data 0x00000001
[ 4918.965131] tegra-vi4 15700000.vi: capture_id 20 stream 2 vchan 0
[ 5075.432800] tegra-vi4 15700000.vi: Status: 5 channel:00 frame:000A

Note that the kernel time is 4918.9 seconds, while the SOF message comes at 4929.8 seconds.

The duration of the test was under one second, how can the SOF be from 10 seconds in the future?

Note: I used yavta from here: https://github.com/brgl/yavta

hello phabsch,

thanks for point out, we will investigate this internally,
thanks

Hi Jerry, is there any updates on this? Thanks

Current v4l2 buffer stamp on TX2 does not comply with V4L2 standard.

https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/buffer.html

[quote]

struct timeval
timestamp
For capture streams this is time when the first data byte was captured, as returned by the clock_gettime() function for the relevant clock id; see V4L2_BUF_FLAG_TIMESTAMP_* in Buffer Flags. For output streams the driver stores the time at which the last data byte was actually sent out in the timestamp field. This permits applications to monitor the drift between the video and system clock. For output streams that use V4L2_BUF_FLAG_TIMESTAMP_COPY the application has to fill in the timestamp which will be copied by the driver to the capture stream.

[quote]

@JerryChang, any updates?

hello phabsch,

there is bug about RTCPU and kernel system out-of-sync, we’re not root cause yet.
please download [Tegra X2 (Parker Series SoC) Technical Reference Manual], and please refer to [Figure 202: NOTIFY System Diagram] and [Real-Time CPU Transport] chapter for more details.

may I have details about your use-case?
I would suggest you to overwrite the timestamp of vi-notify for a temporary solution.
thanks

Hey, I am working together with phabsch.
We need achieve high accuracy synchronization between cameras with other sensors like IMU and Lidar to do some SLAM applications.

hello dk1900,

my suggestion to unblock your process would be overwrite the timestamp of each capture frame for r28.2.1/ TX2.
for example,
please have modification in below VI driver function, and update the timestamp with linux kernel function.

<top>/kernel/kernel-4.4/drivers/media/platform/tegra/camera/vi/vi4_fops.c

static bool vi_notify_wait(...)
{
..
    *ts = ns_to_timespec((s64)status.sof_ts);

Hi JerryChang,
I do already have that work around in place, but I am seeing problems with it. It seems that the vi_notify_wait portion of code after the nvhost_syncpt_wait_timeout_ext() is prone to delays. I haven’t figured out yet whether this is a kernel thread scheduling issue or something else.

I have 4 cameras synced together running at 60 fps. I can see the image is acquired at the same time on all 4 cameras. I used a chasing LED strobe with 100us between LED increment. When I use the timestamp from the RTCPU the values are really close across buffers (+/- 1us). But if I restamp the value in this function I have seen up to 84ms (yes milliseconds) delta between synchronized frames (the content is still synced).

I am thinking that since this code runs in a separate thread for each channel it is being delayed due to thread context switching.

We could probably work with this and just restamp each of the frames with the lowest of all 4 frames, but I am concerned that we are seeing 84ms between them. This means that we sometimes drop multiple frames (because the next acquisition in our code doesn’t start until all 4 frames have captured).

Hello. Any news about issue?

We faced with same problem: system time and timestamps difference about 10 seconds.

After reading Parker TRM I found TSC used for timestamps.

How to read current TSC value? or set it? Or use as system clock

hello kko-smol,

as I mentioned in comment #14,
you may overwrite the timestamp of each capture frame for r28.2.1/ TX2 to unblock your process so far,
thanks

JerryChang, restamp frames in software is not good idea: it will add delay and jitter under load (code in #14 runs as thread, not in interrupt)

I found, that CLOCK_MONOTONIC_RAW return time, close to v4l2 timestamps, but after some hours it got offset too (looks like it work on close, but different frequency. offset grows with uptime)

We want timestamp error not more 1ms.

hello kko-smol,

sorry for late reply,
may I have your confirmation that “after-some-hour offset” is caused by system suspend/resume?
thanks