Hi,
I was trying to get the sof or eof for each captured frame on the kernel side when using argus via nvarguscamerasrc, but so far was unable to find where those timestamps are produced and/or added to the frames.
For v4l2 I see where that is(vi5_capture_dequeue), but when using argus it uses another code path. So far I’m stuck at vi_capture_ivc_status_callback, that from what I can tell is where the buffers are sent to user space, but from what I see they are opaque dma buffers, with no struct behind them.
Is there a way to modify and obtain those timestamps in kernel space when using argus to capture?
I’m using two Xavier NX, one with JP4.6 and another with JP5.1.3
PS: this is related to this topic , found out that most of the discarded frames via the timestamp method are actually fine and most corrupted frames still go pass since their ts is fine. The idea was to use our hw to detect at detect issues during capture issues and use the timestamps as the means to poison the frame and discard them in nvarguscamerasrc.
as you can see of Camera Architecture Stack, v4l2src and nvarguscamerasrc they’re going through different pipelines.
the timestamp within VI driver, (i.e. descr->status.sof_timestamp) it’s the timestamp calculated by RCE, and its sources is TSC timestamp.
hence, you may using APIs, such as Argus::Ext::ISensorTimestampTsc::getSensorSofTimestampTsc to return the VI HW (SOF) timestamp based on TSC.
please see-also Topic 313792 for demonstration by adding some code snippets to Argus sample application, yuvJpeg.
FYI, TSC timestamp is a counter, the period is 32 ns (due to 31.25MHz). you may multiply by 32 to convert TSC to nanoseconds. RCE timestamp is calculated from TSC to obtain the sensor SOF/EOF timestamps.
Hi, thanks for the reply.
Got it, so the VI timestamp is calculated inside argus and we cant modify it before getting it with getSensorSofTimestampTsc.
Is there any other data that we can modify on kernel that can enable us to mark bad frames and wont affect downstream consumers like argus?
Then going back to the prev topic(I cannot seem to be able to reopen, should I open a new one?), is there another way we can check for bad frames, what we noticed is that it seems the jumps are frames that are half populated, like this one, and down the pipeline a buffer is being not cleared and produces the jumps effect.
Hi,
The idea was to every time a frame got pushed and we detected a disconnect via the serdes driver, set the vi timestamp to 0 so we can easily check that timestamp on gstreamer and discard it. But we couldn’t continue this approach since when capturing with the isp ie using argus, we don’t have transparency on what the buffer has or how to access it(that’s what I meant with opaque buffers). So basically we couldn’t continue that approach.
That’s why we now knowing that vi timestamp is produced in argus itself, we are asking if there might be any other frame data accessible in kernel side, that can help us mark a frame.
please dig into below driver for reporting SOF/EOF timestamps,
for instance, $public_sources/kernel_src/kernel/nvidia/drivers/platform/tegra/rtcpu/vi-notify.c
static int tegra_ivc_vi_notify_get_capture_status()
{
...
if (status->sof_ts != 0) {
s64 ts = (s64)status->sof_ts;
if (status->eof_ts != 0) {
s64 ts = (s64)status->eof_ts;
Hi,
Unfortunately seems it only enters in that method when using v4l2, with argus/nvarguscamerasrc it doesn’t go into that method. I added a print and also hard-coded both ts, and when I use the following:
unfortunately, sensor frames were processed by RCE firmware, ISP engine for sending to user-space.
you cannot modify the data on kernel once you’re running through nvarguscamerasrc.
please have post-processing by handling the frames with getSensorSofTimestampTsc APIs.
