Glass to glass latency 4k@60fps on Orin AGX

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

Hi,

I’m currently testing the 4k@60fps performance on Jetson Orin AGX.
Hardware setup: Orin AGX 32GB; Camera IMX 715, 4k60fps
Software version: Jetpack version 5.1

We run some simple tests. Firstly, we revise sample camera_unit_sample to use DRM renderer to do the display. Knowing from here link, the v4l2 protocol is able to stream frames after Jetson ISP. Our 4k60fps glass to glass latency is 88ms.

Next, we directly fetch the bayer data with v4l2src API, and simply display to the screen. The glass to glass latency is about 66ms.

My questions are:

  1. Can I conclude that the ISP processing time is around 88ms - 66ms → 22ms?
  2. Is there any further space to improve the glass to glass latency? from our test results, even if eliminating the ISP pipeline, the latency is still 66ms. What aspects could affect this?
  3. Because we haven’t fully studied the Argus API, so we only use camera_unit_sample to stream the ISP frames. Is this the optimal way to get the frames with lower latency? Or we have to implement Argus API?

Thanks a lot!

Best,
Wenhai.

2

Suppose 88ms is very good result for the G2G. I don’t think there has much space to improve it.

Thanks

3

Hi Shane:

Thanks for your prompt reply. We are looking into the latency without ISP, 66ms. It’s around 4 frames, which is a bit of longer to our application, can I check with you what dose it consists of? Is there anything we can to do to narrow it down? For example, to concurrently write-in, copy-out the buffer, etc.

Best,
Wenhai.

4

Check the nbuffers and try reduce it to try.

static int vi5_channel_setup_queue(struct tegra_channel *chan,
276  	unsigned int *nbuffers)
277  {
278  	int ret = 0;
279  
280  	*nbuffers = clamp(*nbuffers, CAPTURE_MIN_BUFFERS, CAPTURE_MAX_BUFFERS);
281  
282  	ret = tegra_channel_alloc_buffer_queue(chan, *nbuffers);
283  	if (ret < 0)
284  		goto done;
285  
286  	chan->capture_reqs_enqueued = 0;
287  
288  done:
289  	return ret;
290  }
5

I will try it, thanks a lot!

6

Hi Shane:

Is there any instruction about how to debug the kernel sources? Do I have to add a “printf” after the line 280, then rebuild the kernel sources?

Best,
Wenhai.

8

Yes, need add pr_info() and rebuild the kernel for it.

Thanks

9

Hi Shane,

Thanks a lot! Could you pls suggest how I can see those log info after rebuild the kernel? Will it be automatically printed out while running my script?

Best,
Wenhai.

10

Just check the message by “sudo dmesg | tail” after launch the camera command.

11

Hi Shane,

I’m unable to re-flash the devices, is there any other way to debug the kernel source code instead of flashing the device completely? Or, do I have to re-flash the device to make the change effective?

Best,
Wenhai.

12

You can modify the /boot/extlinux/extlinux.conf to load your kernel by define the FDT

13

I just noticed a tricky issue. The current kernel Image in extlinux.conf is the customized Image by the camera vender. After replacing to my modified one, the cam device cannot be detected. Do you have other suggestion to do it under this scenario?

Best,
Wenhai.

14

Ask the vendor to give loadable module sensor driver.(*.ko file)
Then you can insmod after boot to system.

15

Hi,

Just tried to contact with vender, but pending for their feedbacks. I was wondering whether I can insert the module which contains my modified vi5_fops to the kernel? Is it possible to achieve this? If it is, which *.ko file is related to it? Thanks!

Best,
Wenhai.

16

I don’t have idea to build the vi5_fops as ko file it involves too much to make it.

17

Hi Shane,

Thanks for the advice, I managed to get the kernel Image recompiled. just checked, the nbuffers is 2. According to this topic (Vi_capture_status function take about one-frame time latency), buffer number of 2 seems already the optimal setting… I guess if I further reduce it to 1, it will drop frames.

Can I check with you which part configures and determines this buffer number? Is it dynamically determined?

Best,
Wenhai.

18

And btw Shane, is there any difference if I use DMA or MMAP as the buffer to store the frames?

19

The buffer looks like acquire by v4l2 APP request by VIDIOC_REQBUFS,
BTW, MMAP is allocate the DMA buffer.

I would suggest to check the timestamp to narrow down the frame done after VI to memory then the less could be the display then check if any chance to improve.

Below is the timestamp relative topic.

20

Will have a check. Most appreciate your help and suggestion!

21

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.