Xavier CSI-2 capture issue with IMX577 camera

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
24

hello Ax1,

there’s no SOF and EOF signaling coming to VI engine.
please examine your device tree property settings while you’d verify the sensor driver with v4l2 standard controls.

here’re several suggestions for you to investigate this further.

  1. you may double check Sensor Pixel Clock configuration. It must be set correctly to avoid potential issues.
  2. please Maximizing Jetson Xavier Performance by execute jetson_clocks.sh script file.
  3. please also have a try to add set_mode_delay_ms property into your DT, it’s configuration for adding wait time for the first frame after capture starts.
  4. please check Debugg Tips session for the steps to examine your sensor settings.
25

Hi Jerry,

For #3, I added set_mode_delay_ms=“3000” and gst-launch command shows error message: “no camera available”. The video window pops up and closes immediately.

I actually added set_mode_delay_ms to video0 only, but gst-launch on video1 also shows same message.

Is set_mode_delay_ms supposed to be under tegra-camera-platform {} as well?

===========================================================
imx477_a@10 {
compatible = “nvidia,imx477”;
/* I2C device address */
reg = <0x10>;

/* V4L2 device node location */
devnode = "video0";

/* Physical dimensions of sensor */
physical_w = "3.674";
physical_h = "2.738";

sensor_model = "imx477";

set_mode_delay_ms = "3000";

/* Define any required hw resources needed by driver */
/* ie. clocks, io pins, power sources */
avdd-reg = "vana";
iovdd-reg = "vif";

Thanks.

26

hello Ax1,

set_mode_delay_ms should be added in sensor mode property, is used to override the default capture timeout value.
you should review your DT properties for such “no camera available" error messages,
thanks

27

Hi Jerry,

Thanks. I assume the sensor mode is the following. I will give it a try and update the results.

mode2 { // IMX477_MODE_1920X1080
	mclk_khz = "24000";
	num_lanes = "4";
	tegra_sinterface = "serial_a";
	discontinuous_clk = "no";
	dpcm_enable = "false";
	cil_settletime = "0";
	dynamic_pixel_bit_depth = "10";
	csi_pixel_bit_depth = "10";
	mode_type = "bayer";
	pixel_phase = "rggb";

	active_w = "1920";
	active_h = "1080";
	pixel_t = "bayer_rggb";
	readout_orientation = "90";
	line_length = "4512";
	inherent_gain = "1";
	mclk_multiplier = "200";
	pix_clk_hz = "840000000";
	
	set_mode_delay_ms = "3000";

	min_gain_val = "1.0";
	max_gain_val = "354";
	min_hdr_ratio = "1";
	max_hdr_ratio = "160";
	min_framerate = "2.84";
	max_framerate = "60";
	min_exp_time = "5.373";
	max_exp_time = "352112";
	embedded_metadata_height = "2";
};

Thanks.

28

Hi Jerry,

I tried the following, but none of them helps:

  1. Set set_mode_delay_ms = 3000, no difference in behavior.
  2. Set post_crop_frame_drop = 1, no difference
  3. Set max_framerate = 30 (instead of 60), no difference
  4. Execute jetson_clocks script to maximize performance, no difference

I haven’t tried to experiment with the clock settings yet. However, I would like to reiterate that without the MIPI CSI-2 Repeater device, the video streaming using gst-launch works fine.

Just to be clear, here is the block diagram of our setup with Leopard Imaging camera module:

image
image870×297 7.45 KB

Without MIPI CSI-2 Repeater, the setup works fine. Can we conclude the clock setttings are correct?

With CSI-2 Repeater, only CSI-2 packets are going thru the repeater. We also knew that it works with v4l2-ctl. So it looks like something is not configured properly on Xavier side when receiving CSI-2 packets. Are there any particular patterns Xavier VI expects to see when performing video streaming? Or could it be any timing requirements for this setup?

Also note that the CSI-2 Repeater has continuous clock out on DPHY TX, and discontinuous_clk = “no” in sensor mode setting.

Can you please suggest where else to investigate further? Thanks.

Alternatively, since v4l2-ctl works, is there a way to use v4l2 commands to stream video from camera and display it on the screen?

Thanks.

29

hello Ax1,

may I know which JetPack and L4T release version you’re working with?
please check release tag for the info, i.e. # cat /etc/nv_tegra_release
thanks

31

Hi Jerry,

It is R32.3.1

Thanks.

32

hello Ax1,

here’s changes to configure CSI clock modes in case sensor start streaming early,
could you please have a try to replace SCF binary with the attachment, it’s based-on l4t-r32.3.1, Topic165343_Jan28_libnvscf.zip (2.7 MB)
you may overwrite it with /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so, and perform a warm-reboot to take effects.
thanks

33

Hi Jerry,

That helps to get the video streaming back with gstreamer.

I could be wrong, but my guess is that Xavier VI sees continuous CSI-2 clock coming in and expects CSI-2 packets, but there is none. That causes timeout and stalls the video stream.

Can you please provide more details about what changes in the new libnvscf file?

Thanks.

34

hello Ax1,

glad to know the pre-built update in post #32 works.
it’s modification to configure LP-bypass mode for continuous clock sensors, setting CSI CIL configure in case sensor start streaming early.

FYI,
this change already complete code-review process,
you should expect it’ll include in the next public release, (i.e. JetPack-4.6)
thanks

35

Hi Jerry,

Thanks very much for your great support!

Best regards.