The RealSense D457 camera connected to the Orin NX with MAX96724 has no frame input

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
1

I use Orin NX + custom board to connected D457 camera (GMSL).
MAX96724 input from Link A, Output MIPI port B (4 lane).
MIPI Port B is connected to CSI-2 and CSI-3 of the Orin NX.
“I used v4l2-ctl for streaming, but the following error occurred. However, the I2C communication is working fine.”
[ 798.947065] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 798.947091] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 798.948731] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel

Device tree file:
dts.txt (11.4 KB)
Kernel log file:
dmesg.txt (124.6 KB)

3

hello XuanZZZ,

please give it a try to configure lane_polarity = "0"; for using CSI-2/3 of Orin NX.
BTW,
may I also confirm which Jetpack release version you’re working with.

4

I am working with JetPack 6.2

5

The issue still occurs after configuring lane_polarity = "0";

6

hello XuanZZZ,

is it possible to confirm that SerDes chip is working normally for sending MIPI signaling to CSI channel?
for instance, you may arrange hardware resources to probe the high speed signaling.

or…
is this porting issue? had you verify the camera setup with previous Jetpack release version?

7

Hello Jerry,

If the deserializer is set to 1500 Mbps per lane, does deskew need to be enabled?

8

hello XuanZZZ,

it’s 1.5Gbps as deskew threshold.

9

Hi Jerry,

So, I don’t need to set deskew_initial_enable and deskew_periodic_enable in the device tree when the deserializer is set to 1.5 Gbps per lane?

10

hello XuanZZZ,

deskew calibration is a must if data-rate > 1.5 Gbps.

you can ignore these DT properties, deskew_initial_enable, deskew_periodic_enable.
when you have data-rate > 1.5 Gbps, it’s the camera firmware to wait for deskew signal from the sensor side continuously.
moreover,
we are only looking for initial deskew calibration; but, it should also be ok to configure the deserializer to send periodic skew calibration bursts. the effect would be that the initial skew calibration can be retried every time the skew calibration burst is sent.

11

Hi Jerry,

I can receive frames when modifying the deserializer settings, but I can only start one stream at a time.
When I start a second stream, the frames either drop or freeze. Is there any configuration to enable MIPI for multiple streams?

[ 207.323588] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 207.350593] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 262144
[ 207.351325] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 64
[ 207.356848] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 207.384091] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 262144
[ 207.384681] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 64
…

12

hello XuanZZZ,

may I know what’s the modification you’ve done?

13

Hi Jerry,

I changed 4 lanes to 2 lanes (Deser to OrinNX).

14

hello XuanZZZ,

please double check SerDes Pixel Clock to review pixel clock rate settings, it must be set correctly to avoid potential issues.

15

Hi Jerry,

I set the deserializer to 2 lanes at 1.5Gbps per lane and configured the following properties in the device tree, but the issue still occurs.

num_lanes = “2”;
csi_pixel_bit_depth = “16”;
pix_clk_hz = “74250000”;
serdes_pix_clk_hz = “187500000”;

I have four video nodes from a single sensor and have mapped them to four vi channels. How can I check for issues?

16

hello XuanZZZ,

may I know what’s your test pipeline?
did you have the deskew calibration done received by rce-fw?

please follow the steps below to enable VI tracing logs.

modprobe rtcpu_debug    #for JP-6.1 or later.
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace
17

having same issue let me know when get the right answer

18

I am using v4l2-ctl to test the stream, as shown below:

v4l2-ctl -d /dev/video2 --stream-mmap  # RGB  
v4l2-ctl -d /dev/video4 --stream-mmap  # Depth

When I open video4, there are no frames, and video2 starts dropping frames.
Please check the logs.
trace.txt (5.8 MB)

I’m not sure what RCE-FW is? I also enabled deskew calibration in the deserializer, but the issue is not resolved.

19

hello XuanZZZ,

I assume there’s no issue to stream video2 individually, right?
may I know the format types of video4?

20

Hi Jerry,

Yes, stream every video node(IMU/Y8/RGB/Depth) is no issue individually.
(video4 is vc-0, video2 is vc-1)

The video4 is V4L2_PIX_FMT_Z16, define from Intel.

           .name           = "Depth data 16-bit (D16)",
           .guid           = UVC_GUID_FORMAT_D16,
           .fcc            = V4L2_PIX_FMT_Z16,
21

hello XuanZZZ,

this format is not supported by VI, moreover, it’ll need to extend to 32-bit for processing.
for instance, let me taking an example with RGB888 memory format (it’s not supported as well), you’ll need to add luminance format to RGBA32 as workarounds.