This is somewhat a follow-up to Camera Porting Issues where we have a new carrier board to attach the camera to. The new carrier includes an Orin NX SOM, instead of the AGX Xavier SOM of the previous post. The new SOM has nv_tegra_release as “# R36 (release), REVISION: 4.0” whereas the old SOM had “R35 (release), REVISION: 2.1”. The video input worked well with the AGX Xavier, but we are only having partial success with the Orin NX.
We have worked on porting the camera drivers and device tree to the new platform. On the Orin NX we now have the updated drivers built and the updated device tree entries for our camera installed and being used. The camera is visible as /dev/video0.
Video frames are correctly received by the Orin NX at 1280x720, 1920x1080, and 3840x2160 resolutions. However, frames at resolutions of 720x480 and 720x576 are not received. I am looking for assistance in getting those low-resolution inputs to also work correctly.
The camera we are using always uses four MIPI CSI-2 lanes, even at 720x480 and 720x576 resolutions. I don’t know if this is acceptable with Orin NX. It worked with AGX Xavier.
I have attached our device tree file for our camera, as well as the output from some v4l2 commands and other seemingly relevant tests.
postable-output.log.txt (13.5 KB)
tegra194-camera-fury-a00.dtsi.txt (14.8 KB)
The trace log tell didn’t receive any validate data from the sensor.
Maybe probe the MIPI signal to confirm the output signal follow the MIPI spec.
Thanks
Thanks for your comments, ShaneCCC.
The video source was working exactly as it is at the 720x480 resolution with the AGX Xavier SOM and R35.2.1. It is working at the higher resolutions with the Orin NX SOM and R36.4.0. So it seems that the MIPI signaling should be valid.
I don’t actually have a way to probe the MIPI signal as it is sandwiched between boards.
Are there other diagnostics that could be run? Is there some kind of adjustment in the device-tree than needs to be done at lower resolutions that I might be missing?
Thanks again,
Chris
Maybe try the cil_settletime and discontinuous_clk.
Note the discontinuous_clk need to modify the sensor to match the output clock mode.
I tweaked some device-tree parameters and am now getting the following line about every 33ms in the kernel log:
[ 206.340575] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 256, flags: 0, err_data 256
That seems like it might indicate a little progress.
I also got a trace in case that was helpful, and uploaded a new version of the .dtsi file.
Any further suggestions?
trace.txt (33.4 KB)
tegra194-camera-fury-a00.dtsi.txt (14.9 KB)
I played with the device tree a little more. If I set cil_settletime to between ~32 and ~63 I get lines like this:
[ 206.340575] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 256, flags: 0, err_data 256
However, if cil_settletime is greater than ~68 or less than ~29 (including 0) I get the lines like this:
[ 513.677109] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 5000 ms
[ 513.677130] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 513.678819] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 518.797081] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 5000 ms
[ 518.797110] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 518.798699] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
So I think the range of cil_settletime should be between 32 and 63. However, there seems to be something else not yet configured correctly. Any suggestions?
Thanks again!
I have tried varying other mode parameters, including discontinuous_clk but still get every frame discarded as shown above. There doesn’t seem to be any reason in those messages for discarding the frames.
Is there any way to determine why the frames are being discarded (lines too long? lines too short? too many lines? missing sync bytes?
Thanks!
Chris
Do you check if able to dump the frame data to confirm for setting the settle to between ~32 and ~63 ?
I’m not sure I understand why you’re asking what your asking. I thought when when the log says:
[ 206.340575] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 256, flags: 0, err_data 256
that it’s discarding the frames. My v4l2-ctl command to read the frames never indicates a frame is received. There are no ‘>’ characters printed. So I don’t think I can dump the frame data, meaning the contents of a received frame.
Are you asking for a dump of the frame parameters? Aren’t they specified on the command line such as
v4l2-ctl --set-fmt-video=width=720,height=480 --stream-mmap
I do see a line in the trace (uploaded above) that indicates the format:
tegra_channel_capture_setup: vnc_id 0 W 720 H 480 fmt 13
It says “fmt 13”. However, my device-tree only specifies 6 formats, though a v4l2-ctl list yields:
$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'UYVY' (UYVY 4:2:2)
Size: Discrete 3840x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
[1]: 'NV16' (Y/CbCr 4:2:2)
Size: Discrete 3840x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
[2]: 'UYVY' (UYVY 4:2:2)
Size: Discrete 3840x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 4096x2160
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
Please clarify what you’re asking about dumping the frame data.
Thanks very much,
Chris
Please report only one pixel format to try.
Thanks, ShaneCCC, for your suggestion.
Based on your suggestion I reduced the number of frame formats to 1, so that listing formats now yields:
$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'UYVY' (UYVY 4:2:2)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
[1]: 'NV16' (Y/CbCr 4:2:2)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
[2]: 'UYVY' (UYVY 4:2:2)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
which didn’t seem to help the Orin NX accept 480p frames… Then I realized that you indicated trying only a single pixel format. So perhaps I tried the wrong thing.
There are three entries the the VIDIOC_ENUM_FMT table above. [0] and [2] are UYVY and [1] is NV16. Were you wanting me to remove entries [1] and [2]? If so, can you point me at how to remove them from this configuration, since I don’t see where those are mentioned at all in the driver code or device tree.
Remove it from the device tree.
It’s not clear what you mean by “it”. Remove what from the device tree? Do you mean remove the NV16 format from the device tree, or remove something else?
I don’t see NV16 anywhere in the device tree, regardless of capitalization. Please clarify.
I did remove the other frame formats from the device tree, so that only the 720x480 format is present. I don’t see how to remove pixel formats, however.
The drivers/media/platform/tegra/camera/sensor_common.c to parser your device tree to add the pixel format.
- Dump your device tree to confirm if any others pixel format is reported.
- Add the message to sensor_common.c to clarify why NV16 a duplicate UYVY format.
Thank-you, ShaneCCC.
I Searched in the active device tree (/proc/device-tree) for pixel formats:
$ find . -name *pixel* -print
./bus@0/i2c@3180000/fury_video@54/mode0/pixel_t
./bus@0/i2c@3180000/fury_video@54/mode0/pixel_phase
./bus@0/i2c@3180000/fury_video@54/mode0/dynamic_pixel_bit_depth
./bus@0/i2c@3180000/fury_video@54/mode0/csi_pixel_bit_depth
./tegra-camera-platform/vi_peak_byte_per_pixel
./tegra-camera-platform/min_bits_per_pixel
./tegra-camera-platform/isp_peak_byte_per_pixel
Then looking in those pixel files:
$ grep -a . ./bus@0/i2c@3180000/fury_video@54/mode0/pixel
./bus@0/i2c@3180000/fury_video@54/mode0/csi_pixel_bit_depth:16
./bus@0/i2c@3180000/fury_video@54/mode0/dynamic_pixel_bit_depth:16
./bus@0/i2c@3180000/fury_video@54/mode0/pixel_phase:uyvy
./bus@0/i2c@3180000/fury_video@54/mode0/pixel_t:yuv_uyvy16
So the device tree for my camera devices has only one mode and it’s UYVY.
I also grep’ed for pixel formats and found UYVY, but not NV16:
$ grep -ri uyvy .
grep: ./bus@0/i2c@3180000/fury_video@54/mode0/pixel_t: binary file matches
grep: ./bus@0/i2c@3180000/fury_video@54/mode0/pixel_phase: binary file matches
but there were no matches when grepping for NV16. So I believe the device-tree itself isn’t what’s making that NV16 format show up with the VIDIOC_ENUM_FMT ioctl.
I looked at drivers/media/platform/tegra/camera/sensor_common.c and it doesn’t mention NV16. I don’t believe I need NV16, but it’s showing up in the VIDIOC_ENUM_FMT output. Do I need to remove NV16 from the supported formats? Our camera doesn’t output NV16, only UYVY. Is something inside the Orin NX offering to create NV16 from the UYVY?
I’m still trying to figure out what is wrong with the 720x480 format, which looks the same as the higher resolution, but isn’t being accepted.
Could you check --list-formats-ext on JP5.x to confirm if it’s regression.
Thanks
I’m working far remotely. I will see if I can resurrect that system to run the test. I will continue to pursue just working on the Orin NX for now.
I did notice that in the places where frame sizes are called out in the csi.c files that the show resolutions in format arrays, that the resolutions that are working for me are listed, but not the 720x480 or 720x576. For example:
nvidia-oot/drivers/media/platform/tegra/camera/csi/csi.c
static struct v4l2_frmsize_discrete tegra_csi_tpg_sizes = {
{320, 240},
{1280, 720},
{1920, 1080},
{3840, 2160}
};
kernel-jammy-src/drivers/staging/media/tegra-video/csi.c
static const struct v4l2_frmsize_discrete tegra_csi_tpg_sizes = {
{ 1280, 720 },
{ 1920, 1080 },
{ 3840, 2160 },
};
Are these all the fixed resolution inputs that are supported? I tried just adding the ones I needed but that didn’t seem to fix the issue.
It doesn’t matter. It’s for test pattern generate for nvhost_nvcsi_t194.ko for test.
Thanks
1 Like
I got someone to run the test for me remotely. Here’s what was on the AGX Xavier which had JP5.x on it. It looks the same as above with the NV16 also present as [1], and the redundant UYVY as [2]:
Have no idea why your driver have this kind of problem.
First, could you confirm by JP5
Second, you can add message to print the width/height/format in vi5_setup_surface() to confirm the settings while run the v4l2-ctl to capture.
