Unusable frames

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi,

This is a follow-up discussion from
Jumps in capture with unstable setup and
Argus Timestamps in kernel space

We followed in the instructions and added a check on PORT_PASS_CTRL register at 4x of the FPS.

As a result, we are able to filter out most cases, while some corrupted frames still passes through making our application unreliable.

  1. Is it possible to have a general overview on how frames are generated and an explanation on how an issue like this happens in the first place
  2. We need your guidance to remove this issue or find a reliable way to determine when the frame is corrupted so that we can get rid of it

Thanks,

Yan

3

hello yzhao5,

may I know the latest status?

as mentioned by those two topics..
it should caused by hardware side to send abnormal frames to Argus,
so, please have software approach to evaluate the timestamps to filter out abnormal frames.

4

Hi JerryChang,

We combined the timestamp and register methods suggested in the previous threads, but unfortunately, we can’t filter out all abnormal frames.

Our application requires high reliability from perception data, so we come back to see if there is anything else we can do.

5

hello yzhao5,

since it’s caused by hardware side, we need to check what’s the evidence software side able to capture to distinguish the current failure.
for instance, let’s check.. Argus::Ext::ISensorPrivateMetadataCaps::getMetadataSize()

6

Thanks, will try on our end. The expectation is normal frames will have identical/similar size value while the value from corrupted frames is significantly different?

Seems the function only returns a number, just wanted to confirm the benefit

8

hello yzhao5,

it’s just assumption that we’ll see different results to identify abnormal frames.
please also try… Topic329554_Apr11.7z (56.6 KB) to use debug libnvfusacap.so
it added some debug prints to check the status of each frames.
you ma collect nvargus-daemon logs with.. $ sudo journalctl -b -u nvargus-daemon
the debug logs looks like below..

tegra-ubuntu nvargus-daemon[3611114]: Module_id 30 Severity 2 : DEBUG: getCaptureDescriptorStatus: csiFrameId, errData:  121 0
tegra-ubuntu nvargus-daemon[3611114]: Module_id 30 Severity 2 : DEBUG: getCaptureDescriptorStatus: csiFrameId, errData:  122 0
tegra-ubuntu nvargus-daemon[3611114]: Module_id 30 Severity 2 : DEBUG: getCaptureDescriptorStatus: csiFrameId, errData:  123 0
...
9

Hi JerryChang,

Thanks. We’ll give it a try. One thing I forgot to mention is that we are able to see this issue from two different camera sensors, IMX219 and OnSemi AR0234.

Since IMX219 uses argus while AR0234 uses V4L2, is it possible that, instead of camera, it’s something else causes the issue?

10

hello yzhao5,

may I know what’s the test steps, for example, how can we reproduce this failure on Xavier NX DevKit with IMX219.

11

Hi JerryChang,
I defer to yzhao5 on how to reproduce this issue, but I would like to point out that this issue appears to be more complicated than simply a corrupt frame. Looking at the video in the original ticket, the video jumps backwards by more than a single frame, which make me think there must be a buffer of frames somewhere that is being written to/read from incorrectly or that there is memory corruption. Do you know if there is a circular buffer in the VI or other modules? As yzhao5 mentioned, a general overview on how frames are generated and an explanation on how an issue like this could happen would be extremely helpful.

12

hello phorak,

please refer to Camera Architecture Stack for an overview of Argus and v4l2, they’re went through different pipelines.
there did have queue to handle capture buffers, but.. it would be great if you sharing a sample capture pipeline for reference.

14

Hi JerryChang, what do you mean specifically by a sample capture pipeline? We’ve observed the artifacts both with the Argus and v4l2 pipelines.

15

Hi,
You can try:
Argus:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=$W,height$H,framerate=$F/1' ! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test_c4.mp4 -v

V4l2(needs a non raw cam or a patched v4l2src like the following on v4l2 for a raw producer cam like imx219):

gst-launch-1.0 v4l2src ! 'video/x-raw,bpp=10,width=$W,height$H,framerate=$F/1' ! queue ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=test_c4.mp4 -v

The same can be seen sharing the frames via display, or shmsink as well is not exclusive for when we save the video

To emulate the hw issues we toggle the reset line

1 Like
17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.