Libargus Access to frame index from v4l2

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

I’m looking to get v4l2_buffer.index through libargus to sync frames to an external microcontroller. This is accessible through the v4l2 commands and can be seen as the index when running --verbose on a v4l2-ctl call. We haven’t been able to find a way to get access to this through any of the libargus api calls, any help here would be much appreciated.

V4l2_buffer index is coming from the frame sync packet sent by the image sensor itself. Access to the raw SOF or EOF packets would also work for our application.

3

Suppose you can use below API for Argus.

https://docs.nvidia.com/jetson/l4t-multimedia/classArgus_1_1Ext_1_1ISensorTimestampTsc.html

4

Hey Shane,

Thanks for the quick response. This seems to be the timestamp of when the buffer hits the VI stack (example: sof timestamp: 370197025000 eof timestamp 370211814000). I’m specifically looking for the actual raw packet and/or the frame counter provided by the sensor inside of the frame sync packet. In our case it should be a frame counter of length 0xFFFF.

Output of video driver when verbose is enabled with v4l2-ctl is shown below. Index is taken from the frame sync packet sent by the sensor, you can see this index cycle when the 0xFFFF overflow condition is met (you can artificially lower this number to check on the sensor). It’s important in our case to understand the specific frame correlated with an external microcontroller. Using the sof timestamp would be significantly harder since we would have to correlate clocks between the separate system and this system as well as intercept the video data’s sof and eof packet on the external micro.

    **Index    : 0**
    Type     : Video Capture
    Flags    : mapped
    Field    : None
    Sequence : 4935456
    Length   : 12582912
    Bytesused: 12582912
    Timestamp: 95889.062646s (Monotonic, End-of-Frame)

    **Index    : 1**
    Type     : Video Capture
    Flags    : mapped
    Field    : None
    Sequence : 4935457
    Length   : 12582912
    Bytesused: 12582912
    Timestamp: 95889.079430s (Monotonic, End-of-Frame)
5

Yes, it’s the time hit VI.
I would suggest to get sensor embedded data alternative.
Also below API for getting it.

https://docs.nvidia.com/jetson/l4t-multimedia/classArgus_1_1Ext_1_1ISensorPrivateMetadata.html

6

Thats a great suggestion. We have been previously using the emb from the sensor but our production part currently doesn’t allow emb in the binning mode we are running. We are working with the supplier but it may be impossible to get the sensor to output emb (We’ve been working with them over the last two months but hit a wall and thus are reaching out here).
I know this is a very specific case, any other suggestions or modifications we can do to get that data?

Edit:
To potentially help when enabling debug the Chansel pxl sof message shows the frame number I’m looking at getting within libargus. (from /sys/kernel/debug/tracing/trace) It makes me think that this information is just discarded at some step along the libargus processing of a specific image packet, if we can get a modification to the build to output this or feature request to add this to the Sensor Timestamp that would be amazing (or any other potential solution).

kworker/0:2-17086 [000] … 159530.852617: rtcpu_vinotify_event: tstamp:4985798106641 tag:CHANSEL_PXL_SOF channel:0x23 frame:3 vi_tstamp:4985798026975 data:0x00000001

Thanks for you help so far!

7

The trace log only for debug purpose I don’t think that could be a solution.

8

Okay, I’ll wait for your advice on potential other options.

9

@ShaneCCC Any update?

10

Sorry to tell unable to support your request for current camera framework.
I would suggest using v4l2 API and implement software debayer for it.

Thanks