Custom camera sensor board bring-up problems

I’ve got a custom image sensor board based on the OnSemi ar0823. I am working to get it to operate with the NVIDIA Jetson Orin Nano 8GB dev kit and I’m hitting problems. This board is compatible with the CAM1/J21 header on the Orin Nano dev kit and is configured for 4-lanes at 891Mbps. I’ve been able to connect a IMX219 Raspberry Pi v2 camera on this same J21 header and that works, so I know the dev kit works. Further, I’ve been able to get the custom board to work on a completely different, non-NVIDIA platform so I know the hardware is good.

My Orin Nano driver is based on the nv_imx219.c code and my driver for the other vendor’s platform. The image sensor board is directly connected to CAM1/J21 on the dev kit (no SerDes involved). My driver is able to recognize and successfully probe as I expect it to.

However, when I attempt to capture a single raw frame via v4l2-ctl --set-fmt-video=width=3840,height=2168,pixelformat=BA12 --stream-mmap --stream-count=1 -d /dev/video0 --stream-to=/tmp/ar0823.raw, I see the following errors appear in dmesg until I hit ctrl-C to kill the v4l2-ctl process:

[  110.377962] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 1, flags: 0, err_data 262144
[  113.122368] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  113.122390] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  113.123776] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  113.175732] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 85, flags: 0, err_data 262144
[  115.682230] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  115.682246] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  115.684092] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel

I’ve confirmed that the driver is in a state where it’s transmitting frames so it appears to be outputting data as I expect it to. The error log I’m seeing, however, is not a lot to go on and I’m not sure how to proceed.

I’ve tried maxing out the clocks via this clocking script:

echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

…and there is no difference in behavior.

I’ve tried reducing the pix_clk_hz setting in my device tree and when I try to capture a raw frame, I see the following:

[   68.443845] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 1, flags: 0, err_data 512
[   68.507158] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 3, flags: 0, err_data 512
[   68.548164] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 4, flags: 0, err_data 131072
[   68.639997] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 5, flags: 0, err_data 512
[   68.657075] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 8, flags: 0, err_data 512
[   68.711956] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 9, flags: 0, err_data 512
[   68.930842] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 15, flags: 0, err_data 512
[   69.030623] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 17, flags: 0, err_data 512
[   69.119594] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 22, flags: 0, err_data 512
[   69.338126] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 28, flags: 0, err_data 512
[   69.397740] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 29, flags: 0, err_data 512
[   69.512315] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 33, flags: 0, err_data 512
[   69.541283] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 34, flags: 0, err_data 512
[   69.580679] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 35, flags: 0, err_data 131072

When I max the clocks using the same clocking script above, I then see the first error trace I’ve listed above (the one with the err_data 262144 entries).

The Orin Nano dev kit is running the latest Jetpack 6.1 / r36.4:

nvidia@katana-orin:~/projects/nvidia-orin-nano/scripts$ cat /etc/nv_tegra_release 
# R36 (release), REVISION: 4.0, GCID: 37537400, BOARD: generic, EABI: aarch64, DATE: Fri Sep 13 04:36:44 UTC 2024
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

I’ve tried to enable tracing, but the Orin Nano apparently lacks /sys/kernel/debug/camrtc and this script doesn’t produce any trace information:

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace

I’ve enabled #define DEBUG in csi5_fops.c and vi5_fops.c, but that isn’t giving me a lot of insight either.

I’m a bit at a loss of what my next steps are. Any ideas on what I can do to further troubleshoot the issue?

Thanks!

I just discovered this page:

https://elinux.org/Jetson/l4t/Camera_BringUp#Steps_to_enable_more_debug_messages

…and learned that I need to sudo modprobe rtcpu_debug to enable the underlying trace module for the trace script. I plan to proceed tracing things that way and will update this thread if I get things figured out.

This line appeared in the tracelog:

kworker/1:2-179     [001] .......   176.778840: rtcpu_vinotify_error: tstamp:6429556953 cch:0 vi:1 tag:CHANSEL_NOMATCH channel:0x04 frame:3 vi_tstamp:205745672768 data:0x0000000000000589

I looked this error up on the forum and it turns out the culprit was a setting in my device tree. I originally had embedded_metadata_height="2"; configured. This was a carry-over from the IMX219 device tree. I changed this value to a 0 and I was able to successfully capture the raw output.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.