JetPack 5.0.2 double camera capture not working on Xavier NX

Hi everyone,

I’ve been working on developing a driver for two IMX678 sensors for a Jetson Xavier NX on JetPack 5.0.2. The capture so far is working great when capturing with each camera separately in the following scenario:

  • 3456x2180 at 25 fps resolution.
  • 10 bits per pixel.
  • 2 lanes.
  • 1440000000 Hz datarate

I could make it work by using the information found in this post: Jetpack 5.0.2 camera driver port leads to tegra-capture-vi timeout - #3 by ShaneCCC to calculate the pixel clock value as follows:

Output data rate = (sensor or deserializer pixel clock in hertz) * (bits per pixel) / (number of CSI lanes)

pixel clock in hertz = (output data rate  * number of CSI lanes) / (bits per pixel)

That is, in my case:

pixel clock in hertz = (1440000000 * 2) / 10 = 288000000 Hz

After performing the above calculation, I can dequeue buffers with v4l2-ctl using each camera separately.

Now, when I try to dequeue as follows:

  • First dequeue with /dev/video0
  • Second dequeue with /dev/video1 at the same time /dev/video0 is dequeueing

The dequeue is not successful and I got the following dmesg log messages:

[  119.013416] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  119.013707] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  119.014851] (NULL device *): vi_capture_control_message: NULL VI channel received
[  119.015045] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[  119.015264] (NULL device *): vi_capture_control_message: NULL VI channel received
[  119.015444] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
[  119.015946] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  121.573430] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  103.647810] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  104.925161] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  104.925475] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  104.927297] (NULL device *): vi_capture_control_message: NULL VI channel received
[  104.927511] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=2, csi_port=2
[  104.927742] (NULL device *): vi_capture_control_message: NULL VI channel received
[  104.927880] t194-nvcsi 13e10000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 2 vc- 0
[  104.929720] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel

Furthermore, it does not matter which video device starts, I get the same result.

Also, I tested the driver by increasing the pixel clock value x10 x20 x30, and the same log error messages still and the dequeue not working even with one camera.

Furthermore, I get the trace log messages. Find them attached at the end of this post. Where each file is

  • trace_log_video1_success.txt when could dequeue using /dev/video1

  • trace_log_video0_success.txt when could dequeue using /dev/video0

  • trace_log_dual_camera_not_capturing_log.txt when could not dequeue with both video devices simultaneously.

Question1 for NVIDIA: Does the pixel need to be calculated again based on the number of cameras that are used?
Question2 for NVIDIA: Do you have any documentation about the dual camera support capture on JetPack 5.0.2?

Thanks.
trace_log_video1_success.txt (589.8 KB)
trace_log_video0_success.txt (526.7 KB)
trace_log_dual_camera_not_capturing.txt (548.3 KB)

1 Like

Did you boost the clocks to try?

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

Hi @ShaneCCC

Yes, I did it. However, I am getting the same results, I could not dequeue buffers with v4l2-ctl.

Regards,

Eduardo.

Could you modify the set_exposure/set_frame_rate CID function as dummy function to clarify.

@ShaneCCC: I’m continuing work on this driver: set_exposure is fully implemented, and set_frame_rate is a simple stub (return 0).

I have reproduced @EduardoSalazar96’s results above: I am able to stream from either CSI port A or CSI port C separately, using both V4L2 & gst-launch, but I am unable to stream both simultaneously.

While debugging, I’ve confirm that all clocks are at max, and I’ve also drastically reduced the data-rate from both sensors (6.25FPS with 2x line & frame timing along w/ reduced CSI data rate). So far the failure mode is precisely the same.

Interestingly, I’ve found that port A appears to override port C: If I start port C first with v4l2-ctl, observe frames are arriving, then start port A: Port C will start dropping all frames, port A shows captured frames. Upon stopping port A, port C will resume frame capture!

What should I be investigating next?

Did you run on NX devkit?
I just verify with two IMX477 without problem on J5.0.2

Looks like that’s where we’re headed next. I will also compare against IMX477 to see if I can spot anything relevant. Thanks!

Good news: We’ve solved the problem for the customer.
Bad news: We’re not entirely sure why. :-(

We have two slightly different hardware setups:

  • Setup “A” is an NX devkit using the same serial interfaces (@EduardoSalazar96’s original report above) where the sensors are fully separated from each other. (Separate input clocks, no external XVS/XHS)
  • Setup “B” is the customer’s hardware where the sensors share the same input clock (split using fanout buffer), and have their XVS/XHS signals bridged.

On setup “B”, I got very lucky and managed to observe the CSI lanes of both sensors during an exceedingly rare moment where both sensors managed to stream: In this moment, the start & end of every pixel row was perfectly aligned down to the same clock pulse (1GHz, 5GS/s scope). Typically showed between 10 & 60% phase-offset between the two sensors on different runs, with no jitter or variance during each run itself.

Based on this result, I implemented slave-mode for the 2nd sensor, and now we are able to stream simultaneously without any errors or timeouts. What’s puzzling, is that setup “A” has no shared XVS/XHS, or even input clocks! :-/

XNX’s VI bandwidth is about 460Mx8ppc = 3.68Gbps
Maybe try to low the frame rate to 20 to verify.