ISP crashed with AGX Xavier on customized carrier board

We encounter some ISP crash problems for Jetson AGX Xavier on our customized carried board.
Hardware configuration is as follows:
4x[sensor(IMX390)–> Maxim max9295(serializer)]-------->Maxim max96712(deserializer)–----> Xavier MIPI CSI2(4 lane/4 VC)

We use gstreamer to capture 8 cameras simultaneously, system works fine in most case, but sometimes one or more cameras timeout to get data, and finally the pipeline exit.
The pipeine as follows:

  gst-launch-1.0 -v nvarguscamerasrc sensor-id=$cam maxperf=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1' ! \
        nvvidconv ! fpsdisplaysink video-sink=fakesink

when the ISP crashed, sometimes the kernel no error output, sometimes output errors , can be found in file “isp_error.log”.
isp_error.log (24.7 KB)

If we open debug info, all cameras will crashed.

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

echo file csi4_fops.c +p >/sys/kernel/debug/dynamic_debug/control
echo file channel.c +p >/sys/kernel/debug/dynamic_debug/control
echo file vi5_fops.c +p >/sys/kernel/debug/dynamic_debug/control
echo file capture.c +p >/sys/kernel/debug/dynamic_debug/control
echo file capture_vi_channel.c +p >/sys/kernel/debug/dynamic_debug/control
echo file mc_common.c +p >/sys/kernel/debug/dynamic_debug/control
echo file mipi_cal.c +p >/sys/kernel/debug/dynamic_debug/control
echo file camera_common.c +p >/sys/kernel/debug/dynamic_debug/control

We have tested our system with JetPack 4.2.3, 4.4, and 4.5.1, and ISP crashed problems still exist even with JP 4.5.1.
We also set max performances for Xavier:

	sudo nvpmodel -m 0
	sudo jetson_clocks

We also test with v4l-utils bypass isp, everything works fine.
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap

Regards and thank you for your time.

Please confirm 6 cameras case due to current reference sensor board only have 6 sensors design for verifying.

Thanks

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.