Sometimes, everything works just fine. My latency measurement setup shows 65ms on average (glass-to-glass), which is expected for that setup and the display I am using.
However, most of the time, I observe the following two problems:
frames are displayed “out of order”. I have some LED set up that blink in regular intervals, so it is easy to observe it with the naked eye. Also when objects move through the field of view, there is some “rubberbanding” when the out-of-order frames are displayed.
the latency goes up to over 200ms.
this only EVER happened with the --nvsci flag. Without that flag, I never saw that behavior.
I tried to mess with NUM_PACKETS and replace all #define NUM_PACKETS (6U) with #define NUM_PACKETS (3U). If I understand it correctly, that should NOT solve the problem. However, it looks like the problem is less “severe” with NUM_PACKETS=3. I do not understand why.
Is there a known issue in the --nvsci variant of the nvsipl_camera sample?
Do you observe the issue without --icrop option? Do you observe the issue with Sekonix SF3324/SF3325 modules?
Could you provide a patch for printing out messages for the out-of-order / latency issue?
NOTE: I think we are talking about two different but related issues. The latency issue and the “frames are displayed in wrong order”. The latency issue shows up almost always when nvsci option is enabled. The “frames in wrong order” is sporadic and sometimes I have to restart nvsipl_camera 2-3 times until it shows up.
I tried a couple of things.
First, I tried without the --icrop option, as you requested: ./nvsipl_camera --platform-config "IMX390RGGBE5_200FOV_RGGB_CPHY_x2" --link-enable-masks "0x0001 0x0000 0x0000 0x0000" -d 0 -w 1 --disableISP1Output --nvsci
The latency issue persists. The “frames are displayed in wrong order” issue persists.
Then, I tried with the Sekonix SF3325 module, as you requested: ./nvsipl_camera --platform-config "SF3325_CPHY_x2" --link-enable-masks "0x0001 0x0000 0x0000 0x0000" -d 0 -w 1 --disableISP1Output --nvsci
The latency issue persists. The “frames are displayed in wrong order” issue persists.
Finally, I tried the patch attached to this answer. The patch is the only change I did to the original sample code then I ran this command: ./nvsipl_camera --platform-config "SF3325_CPHY_x2" --link-enable-masks "0x0001 0x0000 0x0000 0x0000" -d 0 -w 1 --disableISP1Output --nvsci
And I got the following results:
For 15 tries, the “frames are out of order” issue showed up only once.
Some of the 15 tries had the expected average latency of 65ms.
Some of the 15 tries had an average latency of 98ms.
some of the 15 tries had an average latency of 130ms.
In the case, we can start with checking the latency issue.
I meant if you can provide a patch for us to observe the latency issue (tell by some debugging messages)?
Hi @VickNV ,
I could not get an idea of the latency issue with debugging, just with end-to-end measurement. We have a setup where we point the camera at a blinking diode and a photosensor attached to the video output allows us to measure the latency. I find it hard to “trace” a specific frame to be able to measure the latency for that specific frame. My suspicion is, that frames are taken from the queues in the wrong order or even some kind of random order. That would explain why sometimes the latency is ok and sometimes not.
I hope the information provided is still enough for you to find something.
Please help to try if you see the same issue on group b with --link-enable-masks “0x0000 0x0001 0x0000 0x0000”.
I just found the preview looks smoother on group b.
I had some other duties that required a bit of my time, but now I had enough time to give you more info:
I tried group B, same issue.
I made a video. You see on the left screen the faulty behavior (with the --nvsci flag). The right screen shows three different cameras from a different device, I just put them on so you can compare with a “normal” image. You can clearly see my hand doing a smooth movement on the small image and the “jumping around” effect on the left screen because frames are displayed not in the correct order. I recorded with a 1/4 slow-mo.
@VickNV any new information? I would think it would be quite easy to reproduce the issue on your end. Is there still someone available working on the nvsipl sample on your end?
So it could be related to some issue in buffer transportation over NvSciStreams or the way it is implemented in the nvsipl_camera sample app. However, it may not be solved real soon. Especially, DRIVE OS 5.2.6 will be the last release for Xavier generation.