Argus frame Jitter In Jetson NX

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
, ,
1

Hi guys,
I need to grab frames from a 120Hz Bayer camera via Argus API. I have tried multiple approaches (listed below) but faced FPS jitter. For a 120Hz camera, I expect to receive (grab) frames every 8 millisecond but the time delay between successive frames is variable e.g. 10,11,12,13,14,7, and 5 millisecond. My approaches are listed below:
Approach1: I used nvarguscamerasrc GStreamer plugin for frame capturing which led to frame grabbing jitter.
Approach2: I used Argus API and libnvargus_socketclient.so library to grab frames but the jitter problem persists.
Approach3: I used even /usr/src/jetson_multimedia_api/samples/unittest_samples/camera_unit_sample source code for frame grabbing but it has jitter in frame grabbing through v4l2 library.
Approach4: I used /usr/src/jetson_multimedia_api/samples/v4l2cuda source code and MMAP mode to capture and change pixel format to RG10 for frame grabbing. The jitter is less than Argus solution.

To summarize, approaches 2, 3, and 4 all have jitter but the jitter severity in the 4th approach is less than others. However, the 4th approach doesn’t pass the frames through ISP. How can we utilize ISP after capturing frame using the 4th approach both MMAP and DMA modes? I mean the source code using ISP API in Tegra is needed with the hope of having less jitter. Additionally, it would be useful if I can access source code of /usr/lib/aarch64-linux-gnu/libv4l2.so which is related to the 3rd approach.
Any solution for decreasing jitter in Argus is highly appreciated.
Thanks so much.

3

Same here. Appreciate if anyone could give a suggestion to reduce the jitter. I’m using Jetson Orin NX 8GB. Jetson Linux v35.4.1.

4

hello hhami.20401,

for checking the sensor capability.
could you please give it a try to disable preview and shows frame-rate only,
for instance,
$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

5

Hi guys,
Thanks for your attention, This is a result for 1280*720, 100 FPS, that shows it has jitter.
result.txt (15.1 KB)
and time between two frame in milisec in this config is here
res.txt (175.8 KB)
It also has jitter.

Thanks so much

6

hello hhami.20401,

it shows no frame drops.
anyways, this could be minor, please try revise your pipeline by using framerate=100/1, for testing your 100-fps camera sensor.

8

Hi,

By the way, I think I might meet a similar issue. I found the glass to glass latency is periodically fluctuating. Please find the attachments where I enclosed two sets of glass to glass latency measurements. One has an increasing trend, and the other one illustrates the dropping edge, the keep on increasing. How can I make the latency stable and stay minimum?

Here is my setup:

  • Jetson Orin NX 8GB, Jetson Linux 35.4.1
  • Capture frame with libargus
  • Rendering with NvEglRenderer
  • Jetson_clocks is on. VIC, VI, ISP frequencies all boosted.
  • Video 4K@60FPS

image
image927×326 22.5 KB

image
image921×322 23 KB

Best,
Wenhai.

9

Thanks for your reply.
I corrected the pipeline, It still has jitter problem and result are here.
res.txt (31.0 KB)
result.txt (17.7 KB)
How can we correct it ? why v4l UYVY capture is more stable and has less jitter.
Thanks so much.

10

hello hhami.20401,

the average frame-rate report a consistency frame-rate.
it should be very small amounts, right?
you may also give it a try by setting aelock, awblock for verification.

11

hello calvin0328,

I’m also curious about your G2G latency, which has dropped periodically.
hence… may I also know what’s the environment setups. how you evaluate your glass to glass latency.
according to above, could you please also give it a try by setting aelock, awblock for verification.

12

Hi , thanks for attention
it has jitter right now with settings aelock, awblock property.
Thanks so much.

13

hello hhami.20401,

do you have dumped data for reference?
BTW, please also have clocks boosted for verification.
for example,

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
14

Hi , thanks for attention
it has jitter right now with even setting

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

Thanks so much.

15

hello hhami.20401,

we may need to review your test results for further checks.
per your previous sharing results… it seems the average frame-rate report a consistency frame-rate.

18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.