MIPI acquisition dropping frames

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

I am bringing up a new specialized camera sensor connected to the CSI connector on the bottom of a Jetson Orin devkit and I am seeing errors in the kernel logs about the discarded frames and I am hoping someone can help me determine the root cause of the CHANSEL_SHORT_FRAME events in the kernel traces that I am seeing. I have double checked the values in the device tree for the sensor and confirmed that they match the sensor and that csi_pixel_bit_depth="16", active_w="1024", active_h="1024", line_length="1064" (with horizontal blanking), mode_type="yuv", pixel_phase="yuyv", pix_clk_hz="40000000", embedded_metadata_height="0" are all correct. Is it possible to bypass the CRC checks and examine the raw data for errors on the Jetson.

I am using v4l2-ctl to read from the device:

root@orin:~$ v4l2-ctl -d /dev/video0 --verbose --set-fmt-video=width=1024,height=1024,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test_frame.raw
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 1024/1024
        Pixel Format      : 'YUYV' (YUYV 4:2:2)
        Field             : None
        Bytes per Line    : 2048
        Size Image        : 2097152
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             : 
                VIDIOC_REQBUFS returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QUERYBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_QBUF returned 0 (Success)
                VIDIOC_STREAMON returned 0 (Success)
cap dqbuf: 0 seq:      0 bytesused: 2097152 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 1 seq:      1 bytesused: 2097152 ts: 0.000000 (error, ts-monotonic, ts-src-eof)
cap dqbuf: 2 seq:      2 bytesused: 2097152 ts: 0.000000 (error, ts-monotonic, ts-src-eof)

and I see the following output in the kernel logs (5.15.185-rt-tegra):

[ 1160.612123] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 1160.640442] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 1160.668912] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 1160.697359] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072

When i enabled tracing in the kernel with the commands below I was able to capture trace.log (74.9 KB) where I found CHANSEL_SHORT_FRAME entries.

sudo su
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace_pipe

I was able to add some extra log messages to vi5_fops.c to also output the status and notify_bits

image
image1112×129 100 KB

but I have been unable to understand exact details on the error.

3

CHANSEL_SHORT_FRAME tell the output size don’t expected.

Do you boost the clocks to confirm.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
4

I tried with booted clock but got the same issue.

capture_output.txt (4.0 KB)

5

Is there a way to see the actual number of bytes received from the sensor? That information could help me understand if its a few bytes off, a few lines off, or even confirm that the sensor is sending fixed sized frames.

6

No,

But you can modify the driver to reduce the height to narrow down it.

8

After testing a few different configurations I was able to get an image to read from an FPGA test pattern generator connected to the same CSI port but I now have two questions.

  1. Im guessing my original issue stemmed a mismatched format header: RAW16 output from the sensor while the Jetson was configured to expect a 16 bit yuv format. The question I have is will the Orin or the onboard Falcon processor drop the frame if there is a configuration mismatch even if we are bypassing the ISP and reading via V4l2?

  2. Using an FPGA based test pattern generator connected to the same CSI port I have been testing multiple formats. The first that I was able to get working was:

Width Height Horizontal Blanking
(Pixels)		 Pixel Clock
(Mhz)		 Pixel
Format		 Bit
Depth
3840 2160 256 40MHZ RGB888 24

This yielded frames of 33177600 bytes with each pixel represented by 4 bytes. Then when I attempted to generate a smaller test frame that matched one of my sensors:

Width Height Horizontal Blanking
(Pixels)		 Pixel Clock
(Mhz)		 Pixel
Format		 Bit
Depth
1620 1240 256 40MHZ RGB888 24

I got another timeout error reading from the device. From what I tested and could find on the forums it seems there is either a 64-byte alignment (and/or maybe a 256-byte alignment) requirement for the Jetson to read an image. Can you confirm if there is such a byte alignment requirement and if so how to calculate the frame size/horizontal blanking pixels needed to align the lines/frame?

9
  1. No the NVCSI/VI don’t check the format to drop the frame.
  2. You can set the preferred_stride by v4l2-ctl like below for the 256 alignment.
v4l2-ctl -c preferred_stride=1792