No CSI traffic is received from CSI custom sensor

Hi

I am working with a custom CSI sensor, and a custom CSI driver based on ov5693.
When running the following command:
v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap

I can see CSI traffic with scope, and packet counts are incremented from the custom sensor, but no traffic is received in the Xavier.

Below are the trace logs:

# tracer: nop
#
# entries-in-buffer/entries-written: 36/36   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
        v4l2-ctl-9689  [003] ....   714.864755: tegra_channel_open: vi-output, imx324 30-001a
        v4l2-ctl-9689  [000] ....   714.879859: tegra_channel_set_power: imx324 30-001a : 0x1
        v4l2-ctl-9689  [000] ....   714.879893: camera_common_s_power: status : 0x1
        v4l2-ctl-9689  [000] ....   714.880546: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9689  [000] ....   714.880553: csi_s_power: enable : 0x1
        v4l2-ctl-9689  [000] ....   714.889371: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9689  [000] ....   714.897982: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9689  [000] ....   714.897985: csi_s_stream: enable : 0x1
        v4l2-ctl-9689  [000] ....   714.897987: tegra_channel_set_stream: imx324 30-001a : 0x1
        v4l2-ctl-9689  [001] ....   718.252872: tegra_channel_close: vi-output, imx324 30-001a
        v4l2-ctl-9689  [000] ....   718.252948: tegra_channel_set_stream: enable : 0x0
        v4l2-ctl-9689  [000] ....   718.252951: tegra_channel_set_stream: imx324 30-001a : 0x0
        v4l2-ctl-9689  [000] ....   718.253634: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9689  [000] ....   718.253638: csi_s_stream: enable : 0x0
        v4l2-ctl-9689  [000] ....   718.261559: tegra_channel_set_power: imx324 30-001a : 0x0
        v4l2-ctl-9689  [000] ....   718.261574: camera_common_s_power: status : 0x0
        v4l2-ctl-9689  [000] ....   718.261953: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9689  [000] ....   718.261960: csi_s_power: enable : 0x0
        v4l2-ctl-9702  [003] ....   902.080095: tegra_channel_open: vi-output, imx324 30-001a
        v4l2-ctl-9702  [000] ....   902.089496: tegra_channel_set_power: imx324 30-001a : 0x1
        v4l2-ctl-9702  [000] ....   902.089517: camera_common_s_power: status : 0x1
        v4l2-ctl-9702  [001] ....   902.095719: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9702  [001] ....   902.095775: csi_s_power: enable : 0x1
        v4l2-ctl-9702  [000] ....   902.105351: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9702  [000] ....   902.118149: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9702  [000] ....   902.118152: csi_s_stream: enable : 0x1
        v4l2-ctl-9702  [000] ....   902.118154: tegra_channel_set_stream: imx324 30-001a : 0x1
        v4l2-ctl-9702  [001] ....   907.538030: tegra_channel_close: vi-output, imx324 30-001a
        v4l2-ctl-9702  [000] ....   907.538131: tegra_channel_set_stream: enable : 0x0
        v4l2-ctl-9702  [000] ....   907.538134: tegra_channel_set_stream: imx324 30-001a : 0x0
        v4l2-ctl-9702  [000] ....   907.538803: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9702  [000] ....   907.538806: csi_s_stream: enable : 0x0
        v4l2-ctl-9702  [000] ....   907.540655: tegra_channel_set_power: imx324 30-001a : 0x0
        v4l2-ctl-9702  [000] ....   907.540661: camera_common_s_power: status : 0x0
        v4l2-ctl-9702  [000] ....   907.540823: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9702  [000] ....   907.540825: csi_s_power: enable : 0x0

I don’t see any errors in the trace log.

Could this issue be related to working with CSI virtual channel ?

Thanks,
Ron

Did you enable the trace by below command?

sudo su

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 3 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

Hi Shane,

Those are the exact commands I was using for the trace logs.

The system was working before, the only changes which were done:

  • Adding the custom sensor a virtual channel (I added to the device tree sensor and VI as well)
  • Moving from Jetpack R32.4.3 to R32.6.1
  • Adding Ser-Des between the sensor and the Jetson.

The counters which are increasing are at the Deserializer CSI toward the Jetson.

Could this be related to the CSI virtual-channel ?
Could this be related to Ser-Des CSI running at 1Gbps while the sensor is running at 600Mbps ?
Could this be related to the sensor working at CSI continuous clock, while the Ser-Des is working at non-continuous clock?

Thanks,
Ron

But I didn’t saw any trace log about the VI notify like below link.

https://elinux.org/Jetson/l4t/Camera_BringUp

Me neither - that is why I asked the question.

In the past CSI traffic passed successfully, and all trace logs appeared.
What could be the result for no vinotify logs at all ?

How about from verify without VC/Ser-Des first.

The Ser-Des must have VC for its internal routing.

One thing I noticed was that the Deserializer CSI is working at 1.1G per lane.
But it seems that the nvcsi rate is only 400M

/home/valens/radarDemo# cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
400000000

Also, what does it mean that when running
v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw --verbose

The system hangs on VIDIOC_STREAMON forever, and doesn’t exit on timeout ?

400M is NVCSI logic working clock doesn’t matter with the lane speed. If there’s concern for the NVCSI/VI bandwidth you can boost them for sensor bring up stage.

The v4l2-ctl command is capture frame data and save to file, if capture failed it would stuck there.

I already tried with boost - without successs.

Usefully when there was no frame start/end or invalid line length - the v4l2-ctl would timeout after a few seconds, and perform VIDIOC_STREAMOFF.

But now it just hangs on VIDIOC_STREAMON - what does that mean ? Is there any CSI data/clock ?

That could be the NVCSI/VI driver try to recovery and in the recovery loop.
The root cause still the sensor output invalidate data cause it.

Hi Shane,

I connected the system to CSI analyzer, and the traffic after the Deserializer received successful and correctly.

The sensor is transmitting 2048x384 raw12 over 4 lanes CSI at rate 1.1G each lane.

Below is my DT configuration:

/ {
	host1x {
		vi@15c10000 {
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					imx324_vi_in0: endpopoint {
                                                vc-id = <0>;
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&imx324_csi_out0>;
					};
				};
			};
		};

		nvcsi@15a00000 {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						imx324_csi_in0: endpoint@0 {
							port-index = <0>;
							bus-width = <4>;
							remote-endpoint = <&imx324_imx324_out0>;
						};
					};
                                        port@1 {
                                                reg = <1>;
                                                imx324_csi_out0: endpoint@1 {
                                                        remote-endpoint = <&imx324_vi_in0>;
                                                };
                                        };
				};
			};
		};
	};

	i2c@3180000 {
		tca9546@70 {
			i2c@0 {
			imx324_a@1a {
				compatible = "nvidia,imx324";

				reg = <0x1a>;

                                /* V4L2 device node location */
                                devnode = "video0";

				/* Physical dimensions of sensor */
				physical_w = "15.0";
				physical_h = "12.5";

				sensor_model ="imx324";

				/* Defines number of frames to be dropped by driver internally after applying */
				/* sensor crop settings. Some sensors send corrupt frames after applying */
				/* crop co-ordinates */
				post_crop_frame_drop = "0";

				/* Convert Gain to unit of dB (decibel) befor passing to kernel driver */
				//use_decibel_gain = "true";

				/* enable CID_SENSOR_MODE_ID for sensor modes selection */
				use_sensor_mode_id = "true";

				/**
				* A modeX node is required to support v4l2 driver
				* implementation with NVIDIA camera software stack
				*
				* mclk_khz = "";
				* Standard MIPI driving clock, typically 24MHz
				*
				* num_lanes = "";
				* Number of lane channels sensor is programmed to output
				*
				* tegra_sinterface = "";
				* The base tegra serial interface lanes are connected to
				*
				* vc_id = "";
				* The virtual channel id of the sensor.
				*
				* discontinuous_clk = "";
				* The sensor is programmed to use a discontinuous clock on MIPI lanes
				*
				* dpcm_enable = "true";
				* The sensor is programmed to use a DPCM modes
				*
				* cil_settletime = "";
				* MIPI lane settle time value.
				* A "0" value attempts to autocalibrate based on mclk_multiplier
				*
				* active_w = "";
				* Pixel active region width
				*
				* active_h = "";
				* Pixel active region height
				*
				* dynamic_pixel_bit_depth = "";
				* sensor dynamic bit depth for sensor mode
				*
				* csi_pixel_bit_depth = "";
				* sensor output bit depth for sensor mode
				*
				* mode_type="";
				* Sensor mode type, For eg: yuv, Rgb, bayer, bayer_wdr_pwl
				*
				* pixel_phase="";
				* Pixel phase for sensor mode, For eg: rggb, vyuy, rgb888
				*
				* readout_orientation = "0";
				* Based on camera module orientation.
				* Only change readout_orientation if you specifically
				* Program a different readout order for this mode
				*
				* line_length = "";
				* Pixel line length (width) for sensor mode.
				* This is used to calibrate features in our camera stack.
				*
				* mclk_multiplier = "";
				* Multiplier to MCLK to help time hardware capture sequence
				* TODO: Assign to PLL_Multiplier as well until fixed in core
				*
				* pix_clk_hz = "";
				* Sensor pixel clock used for calculations like exposure and framerate
				*
				*
				*
				*
				* inherent_gain = "";
				* Gain obtained inherently from mode (ie. pixel binning)
				*
				* min_gain_val = ""; (floor to 6 decimal places)
				* max_gain_val = ""; (floor to 6 decimal places)
				* Gain limits for mode
				* if use_decibel_gain = "true", please set the gain as decibel
				*
				* min_exp_time = ""; (ceil to integer)
				* max_exp_time = ""; (ceil to integer)
				* Exposure Time limits for mode (us)
				*
				*
				* min_hdr_ratio = "";
				* max_hdr_ratio = "";
				* HDR Ratio limits for mode
				*
				* min_framerate = "";
				* max_framerate = "";
				* Framerate limits for mode (fps)
				*
				* embedded_metadata_height = "";
				* Sensor embedded metadata height in units of rows.
				* If sensor does not support embedded metadata value should be 0.
				*/
				mode0 {/*mode AWR2243_MODE_2048X384*/
					vc_id="0";
					mclk_khz = "24000";
					num_lanes = "4";
					tegra_sinterface = "serial_a";
                                        phy_mode = "DPHY";
					discontinuous_clk = "yes";
					dpcm_enable = "false";
					cil_settletime = "0";
					dynamic_pixel_bit_depth = "12";
					csi_pixel_bit_depth = "12";
					mode_type = "bayer";
					pixel_phase = "rggb";

					active_w = "2048";
					active_h = "384";
					readout_orientation = "0";
					line_length = "2048";
					inherent_gain = "1";
					mclk_multiplier = "15.3";  
					pix_clk_hz = "367200000";  

					gain_factor = "10";
					min_gain_val = "0"; /* dB */
					max_gain_val = "300"; /* dB */
					step_gain_val = "1"; /* 0.1 */
					default_gain = "0";
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					framerate_factor = "1000000";
					min_framerate = "20000000";
					max_framerate = "30000000";
					step_framerate = "1";
					default_framerate = "25000000";
					exposure_factor = "1000000";
					min_exp_time = "59"; /*us, 2 lines*/
					max_exp_time = "55556";
					step_exp_time = "1";
					default_exp_time = "556";/* us */
					embedded_metadata_height = "0";
				};
		
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						imx324_imx324_out0: endpoint {
							port-index = <0>;
							bus-width = <4>;
                                                        vc-id = <0>;
							remote-endpoint = <&imx324_csi_in0>;
							};
						};
					};
				gmsl-link {
					src-csi-port = "a";
					dst-csi-port = "a";
					serdes-csi-link = "a";
					csi-mode = "1x4";
					num-lanes = <4>;
					streams = "ued-u1", "raw12";
					};
				};				
			}; /*i2c@0*/
		};
	};
};

/ {

	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		/**
		* Physical settings to calculate max ISO BW
		*
		* num_csi_lanes = <>;
		* Total number of CSI lanes when all cameras are active
		*
		* max_lane_speed = <>;
		* Max lane speed in Kbit/s
		*
		* min_bits_per_pixel = <>;
		* Min bits per pixel
		*
		* vi_peak_byte_per_pixel = <>;
		* Max byte per pixel for the VI ISO case
		*
		* vi_bw_margin_pct = <>;
		* Vi bandwidth margin in percentage
		*
		* max_pixel_rate = <>;
		* Max pixel rate in Kpixel/s for the ISP ISO case
		*
		* isp_peak_byte_per_pixel = <>;
		* Max byte per pixel for the ISP ISO case
		*
		* isp_bw_margin_pct = <>;
		* Isp bandwidth margin in percentage
		*/
		num_csi_lanes = <4>;
		max_lane_speed = <1100000>;    
		min_bits_per_pixel = <12>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		/**
		 * The general guideline for naming badge_info contains 3 parts, and is as follows,
		 * The first part is the camera_board_id for the module; if the module is in a FFD
		 * platform, then use the platform name for this part.
		 * The second part contains the position of the module, ex. "rear" or "front".
		 * The third part contains the last 6 characters of a part number which is found
		 * in the module's specsheet from the vender.
		 */
		modules {
			module0 {
				badge = "imx324_front";
				position = "front";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "imx324 30-001a";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/tca9546@70/i2c@0/imx324_a@1a";
				};
			};
		};
	};
};

I boosted the clock, and open the debug traces.
Below is the trace log:

     kworker/0:3-2165  [000] ....   214.645951: rtos_queue_peek_from_isr_failed: tstamp:7146539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   214.757939: rtos_queue_peek_from_isr_failed: tstamp:7151539895 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   214.925941: rtos_queue_peek_from_isr_failed: tstamp:7156539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.093982: rtos_queue_peek_from_isr_failed: tstamp:7161539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.261946: rtos_queue_peek_from_isr_failed: tstamp:7166539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.429940: rtos_queue_peek_from_isr_failed: tstamp:7171539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.597947: rtos_queue_peek_from_isr_failed: tstamp:7176539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.765994: rtos_queue_peek_from_isr_failed: tstamp:7181539896 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.877959: rtos_queue_peek_from_isr_failed: tstamp:7186539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.049964: rtos_queue_peek_from_isr_failed: tstamp:7191539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.218003: rtos_queue_peek_from_isr_failed: tstamp:7196539901 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.385938: rtos_queue_peek_from_isr_failed: tstamp:7201539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.721967: rtos_queue_peek_from_isr_failed: tstamp:7211539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.837977: rtos_queue_peek_from_isr_failed: tstamp:7216539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.005976: rtos_queue_peek_from_isr_failed: tstamp:7221539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.173941: rtos_queue_peek_from_isr_failed: tstamp:7226539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.509993: rtos_queue_peek_from_isr_failed: tstamp:7236539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.677982: rtos_queue_peek_from_isr_failed: tstamp:7241539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.957954: rtos_queue_peek_from_isr_failed: tstamp:7251539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   218.125936: rtos_queue_peek_from_isr_failed: tstamp:7256539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   218.237955: rtos_queue_peek_from_isr_failed: tstamp:7259287510 queue:0x0bcbbbb8
        v4l2-ctl-9515  [003] ....   227.820424: tegra_channel_open: vi-output, imx324 30-001a
        v4l2-ctl-9515  [001] ....   227.832877: tegra_channel_set_power: imx324 30-001a : 0x1
        v4l2-ctl-9515  [001] ....   227.832913: camera_common_s_power: status : 0x1
        v4l2-ctl-9515  [001] ....   227.834070: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9515  [001] ....   227.834079: csi_s_power: enable : 0x1
        v4l2-ctl-9515  [001] ....   227.843742: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9515  [001] ....   227.854937: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9515  [001] ....   227.854941: csi_s_stream: enable : 0x1
        v4l2-ctl-9515  [001] ....   227.854943: tegra_channel_set_stream: imx324 30-001a : 0x1
        v4l2-ctl-9515  [001] ....   232.740804: tegra_channel_close: vi-output, imx324 30-001a
        v4l2-ctl-9515  [001] ....   232.740888: tegra_channel_set_stream: enable : 0x0
        v4l2-ctl-9515  [001] ....   232.740891: tegra_channel_set_stream: imx324 30-001a : 0x0
        v4l2-ctl-9515  [001] ....   232.742899: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9515  [001] ....   232.742903: csi_s_stream: enable : 0x0
        v4l2-ctl-9515  [000] ....   232.745340: tegra_channel_set_power: imx324 30-001a : 0x0
        v4l2-ctl-9515  [000] ....   232.745347: camera_common_s_power: status : 0x0
        v4l2-ctl-9515  [000] ....   232.745545: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9515  [000] ....   232.745548: csi_s_power: enable : 0x0

It seems like the Jetson doesn’t receive FS even though it is being transmitted on the lines.

Also, when running the command:

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=2048, height=384,

I get the following:

format=NV12' ! appsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2048 x 384 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 0.000000, max 30.000000; Exposure Range min 59000, max 55556000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 2048 H = 384
   seconds to Run    = 0
   Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: Done Success
Got EOS from element "pipeline0".
Execution ended after 0:00:04.006548640
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up

Any idea?

Yes, from the trace shows didn’t receive any validate data from MIPI bus.
I would suggest to configure SER/DES output test partner to debug.

Issue was resolved.
This was caused by Xavier getting CSI clock before data.

Thanks,
Ron