MIPI / Video Interface resolution limit

We are using an 48 MP sensor (8000x6000) and have successfully developed and deployed the sensor driver on Nano.
We are not using the ISP at all, only using RAW 10 bit data from v4l2.

When moving to Xavier NX we realized that all of our binning and cropping modes work, but that there actually seems to be a 6144 horizontal limit even on the MIPI / Video Interface. Anything up to 6144 works, anything above will result in dropped frames.

Is there any way to overcome / reconfigure this on Xavier NX?
We are NOT using the ISP, we just need to get the 8000x6000 RAW data in just as it works on Nano.

@Thomas_K, I will be clarifying this matter and reach out to you soon.

@Thomas_K

when you say the frames are being dropped, does it mean that random frames are lost or no frame capture happens whatsoever?

What is the sensor mode frame rate?

Hi Mike,

“almost all” frames get discarded and debugging seems to indicate that the EMBED_LONG_LINE flag in the VI is set at least in some of the cases, I did not go into full detail.
A random frame might come through from time to time but is all 0x00 in that case - maybe this is just a side effect of error handling/dumping a buffer when the VI gets re-initialized and stopped repeatedly.

There is NO proper frame at all, however.

BR,
Thomas

@Thomas_K

What is your framerate?

I tried different modes:

  1. It would be 10 fps @ 2.5 Gbps MIPI speed (4 lane) normally
  2. Dropping the frame rate to 6 fps and 1.5 Gbps MIPI speed, using the binary driver build from Nano, does not work either (while it works fine on Nano)

I should note that the line length is set to a fixed value of 14704 pixels (sensor limitation) in the cases here.

The difference is just a change in active pixels, i.e. cropping to <= 6144 active pixels horizontal or >=6145 active pixels horizontal.
<=6144 active pixels works fine with any framerate or MIPI lane speed (1.5 or 2.5 Gbps DPHY).

=6145 active pixels has frame drops on Xavier NX, but works fine on Nano.

@Thomas_K
Try to boost the NVCSI/VI clock and boost to system performance to try.

sudo nvpmodel -m 2  #confirm with the /etc/nvpmodel.conf
sudo jetson_clocks&
sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

Hi Shane,

I had tried this before to eliminate any side effects but there is no impact.

The problem persists, even when lowering the framerate to 2 fps or lowering the MIPI speed. 6145 at low clock speeds does not work while 6144 works at fundamentally higher frame rates, so this does not seem to be clock related.
The 6144 limitation was also mentioned elsewhere for the ISP, so 6144 in fact seems to be the magic number (and no clock side effect)…

BR,
Thomas

Does any information from the trace log?

Jetson TX2/28.1 Camera BringUp - eLinux.org

Updated:
dmesg.txt (62.4 KB) trace.txt (119.5 KB)

The trace log show the short frame that tell the output line less than expect.

tag:CHANSEL_SHORT_FRAME

Hi Shane,

I’m sorry, I have uploaded the wrong dmesg/trace. I have corrected it in the original post now.

This dmesg/trace is using exactly the same binary sensor driver and configuration that works on Nano, with 8000 horizontal resolution.

The log show the CHANSEL_FAULT that tell the EMBED_LONG_LINE/EMBED_SPURIOUS, please confirm how many embedded data line are output to modify the embedded_metadata_height in device tree.

tag:CHANSEL_FAULT channel:0x23 frame:2 vi_tstamp:1626650507 data:0x00013008

Hi Shane,

it should be two. This is not changed anywhere, it is two by default in the sensor and two in the device tree.
Same as in all the other modes or on Nano.

If I reduce the horizontal active pixels in the sensor driver (->crop) and only change “active_w” in the device tree, everything works. No other changes made anywhere.

I could try disabling the embedded metadata, though.

BTW, nano didn’t check the metadata line like TX2/Xavier.

I set EBD to zero in the sensor as well as in the device tree. The embedded data line error message seems to be gone now, but the rest of the behavior did not change.

dmesg2.txt (62.8 KB) trace2.txt (140.0 KB)