getSensorTimestamp, getSensorTimestampTsc and v4l2 timestamp

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
6

Hello ShaneCCC,

thank you for the example.

Here is a demo of the inconsistency between timestamps using v4l2 and timestamps using argus.

v4l2-ctl -c bypass_mode=0 --stream-mmap --stream-count=10 --verbose; ./argus_yuvjpeg -t 1 | grep Timestamp:
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
                VIDIOC_REQBUFS returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq:      0 bytesused: 10303488 ts: 736.999092 (ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      1 bytesused: 10303488 ts: 737.032423 delta: 33.331 ms (ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      2 bytesused: 10303488 ts: 737.065754 delta: 33.331 ms (ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq:      3 bytesused: 10303488 ts: 737.099084 delta: 33.330 ms (ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq:      4 bytesused: 10303488 ts: 737.132415 delta: 33.331 ms fps: 30.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      5 bytesused: 10303488 ts: 737.165746 delta: 33.331 ms fps: 30.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      6 bytesused: 10303488 ts: 737.199076 delta: 33.330 ms fps: 30.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 3 seq:      7 bytesused: 10303488 ts: 737.232407 delta: 33.331 ms fps: 30.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 0 seq:      8 bytesused: 10303488 ts: 737.265738 delta: 33.331 ms fps: 30.00 (ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      9 bytesused: 10303488 ts: 737.299069 delta: 33.331 ms fps: 30.00 (ts-monotonic, ts-src-eof)

CONSUMER:       Sensor Timestamp: 713517057000, LUX: 3038.924072
CONSUMER:       Sensor Timestamp: 713550402000, LUX: 3039.257324
CONSUMER:       Sensor Timestamp: 713583727000, LUX: 3039.812256
CONSUMER:       Sensor Timestamp: 713617057000, LUX: 3040.017578
CONSUMER:       Sensor Timestamp: 713650389000, LUX: 3040.390137
CONSUMER:       Sensor Timestamp: 713683724000, LUX: 3040.666748
CONSUMER:       Sensor Timestamp: 713717054000, LUX: 5867.164062
CONSUMER:       Sensor Timestamp: 713750388000, LUX: 6126.350586
CONSUMER:       Sensor Timestamp: 713783719000, LUX: 6328.284180
CONSUMER:       Sensor Timestamp: 713817046000, LUX: 6476.250977
CONSUMER:       Sensor Timestamp: 713850378000, LUX: 6554.108398
CONSUMER:       Sensor Timestamp: 713883707000, LUX: 6613.342285
CONSUMER:       Sensor Timestamp: 713917040000, LUX: 6668.977539
CONSUMER:       Sensor Timestamp: 713950373000, LUX: 6719.968262
CONSUMER:       Sensor Timestamp: 713983701000, LUX: 6768.780273
CONSUMER:       Sensor Timestamp: 714017034000, LUX: 6819.678223
CONSUMER:       Sensor Timestamp: 714050375000, LUX: 6874.002930
CONSUMER:       Sensor Timestamp: 714083697000, LUX: 6931.125488
CONSUMER:       Sensor Timestamp: 714117038000, LUX: 6992.253418
CONSUMER:       Sensor Timestamp: 714150360000, LUX: 7059.019531

As one can see, the timestamps come from different time domains. The timestamps from v4l2 are about 24 seconds above the ones given by argus. And as I wrote above, and already a year ago https://forums.developer.nvidia.com/t/how-to-obtain-accurate-utc0-frame-timestamps/286100/8 the timestamps provided by vi5_fops are not compliant with v4l2 ! And they are discarded by the v4l2src plugin of gstreamer.

Please make vi5_fops and argus provide natively realtime timestamps, or give us enough information on how to fix that ourselves in the kernel.