No CSI traffic is received from CSI custom sensor

Hi

I am working with a custom CSI sensor, and a custom CSI driver based on ov5693.
When running the following command:
v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap

I can see CSI traffic with scope, and packet counts are incremented from the custom sensor, but no traffic is received in the Xavier.

Below are the trace logs:

# tracer: nop
#
# entries-in-buffer/entries-written: 36/36   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
        v4l2-ctl-9689  [003] ....   714.864755: tegra_channel_open: vi-output, imx324 30-001a
        v4l2-ctl-9689  [000] ....   714.879859: tegra_channel_set_power: imx324 30-001a : 0x1
        v4l2-ctl-9689  [000] ....   714.879893: camera_common_s_power: status : 0x1
        v4l2-ctl-9689  [000] ....   714.880546: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9689  [000] ....   714.880553: csi_s_power: enable : 0x1
        v4l2-ctl-9689  [000] ....   714.889371: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9689  [000] ....   714.897982: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9689  [000] ....   714.897985: csi_s_stream: enable : 0x1
        v4l2-ctl-9689  [000] ....   714.897987: tegra_channel_set_stream: imx324 30-001a : 0x1
        v4l2-ctl-9689  [001] ....   718.252872: tegra_channel_close: vi-output, imx324 30-001a
        v4l2-ctl-9689  [000] ....   718.252948: tegra_channel_set_stream: enable : 0x0
        v4l2-ctl-9689  [000] ....   718.252951: tegra_channel_set_stream: imx324 30-001a : 0x0
        v4l2-ctl-9689  [000] ....   718.253634: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9689  [000] ....   718.253638: csi_s_stream: enable : 0x0
        v4l2-ctl-9689  [000] ....   718.261559: tegra_channel_set_power: imx324 30-001a : 0x0
        v4l2-ctl-9689  [000] ....   718.261574: camera_common_s_power: status : 0x0
        v4l2-ctl-9689  [000] ....   718.261953: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9689  [000] ....   718.261960: csi_s_power: enable : 0x0
        v4l2-ctl-9702  [003] ....   902.080095: tegra_channel_open: vi-output, imx324 30-001a
        v4l2-ctl-9702  [000] ....   902.089496: tegra_channel_set_power: imx324 30-001a : 0x1
        v4l2-ctl-9702  [000] ....   902.089517: camera_common_s_power: status : 0x1
        v4l2-ctl-9702  [001] ....   902.095719: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9702  [001] ....   902.095775: csi_s_power: enable : 0x1
        v4l2-ctl-9702  [000] ....   902.105351: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9702  [000] ....   902.118149: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9702  [000] ....   902.118152: csi_s_stream: enable : 0x1
        v4l2-ctl-9702  [000] ....   902.118154: tegra_channel_set_stream: imx324 30-001a : 0x1
        v4l2-ctl-9702  [001] ....   907.538030: tegra_channel_close: vi-output, imx324 30-001a
        v4l2-ctl-9702  [000] ....   907.538131: tegra_channel_set_stream: enable : 0x0
        v4l2-ctl-9702  [000] ....   907.538134: tegra_channel_set_stream: imx324 30-001a : 0x0
        v4l2-ctl-9702  [000] ....   907.538803: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9702  [000] ....   907.538806: csi_s_stream: enable : 0x0
        v4l2-ctl-9702  [000] ....   907.540655: tegra_channel_set_power: imx324 30-001a : 0x0
        v4l2-ctl-9702  [000] ....   907.540661: camera_common_s_power: status : 0x0
        v4l2-ctl-9702  [000] ....   907.540823: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9702  [000] ....   907.540825: csi_s_power: enable : 0x0

I don’t see any errors in the trace log.

Could this issue be related to working with CSI virtual channel ?

Thanks,
Ron

Did you enable the trace by below command?

sudo su

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 3 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

Hi Shane,

Those are the exact commands I was using for the trace logs.

The system was working before, the only changes which were done:

  • Adding the custom sensor a virtual channel (I added to the device tree sensor and VI as well)
  • Moving from Jetpack R32.4.3 to R32.6.1
  • Adding Ser-Des between the sensor and the Jetson.

The counters which are increasing are at the Deserializer CSI toward the Jetson.

Could this be related to the CSI virtual-channel ?
Could this be related to Ser-Des CSI running at 1Gbps while the sensor is running at 600Mbps ?
Could this be related to the sensor working at CSI continuous clock, while the Ser-Des is working at non-continuous clock?

Thanks,
Ron

But I didn’t saw any trace log about the VI notify like below link.

https://elinux.org/Jetson/l4t/Camera_BringUp

Me neither - that is why I asked the question.

In the past CSI traffic passed successfully, and all trace logs appeared.
What could be the result for no vinotify logs at all ?

How about from verify without VC/Ser-Des first.

The Ser-Des must have VC for its internal routing.

One thing I noticed was that the Deserializer CSI is working at 1.1G per lane.
But it seems that the nvcsi rate is only 400M

/home/valens/radarDemo# cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
400000000

Also, what does it mean that when running
v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw --verbose

The system hangs on VIDIOC_STREAMON forever, and doesn’t exit on timeout ?

400M is NVCSI logic working clock doesn’t matter with the lane speed. If there’s concern for the NVCSI/VI bandwidth you can boost them for sensor bring up stage.

The v4l2-ctl command is capture frame data and save to file, if capture failed it would stuck there.

I already tried with boost - without successs.

Usefully when there was no frame start/end or invalid line length - the v4l2-ctl would timeout after a few seconds, and perform VIDIOC_STREAMOFF.

But now it just hangs on VIDIOC_STREAMON - what does that mean ? Is there any CSI data/clock ?

That could be the NVCSI/VI driver try to recovery and in the recovery loop.
The root cause still the sensor output invalidate data cause it.

Hi Shane,

I connected the system to CSI analyzer, and the traffic after the Deserializer received successful and correctly.

The sensor is transmitting 2048x384 raw12 over 4 lanes CSI at rate 1.1G each lane.

Below is my DT configuration:

/ {
	host1x {
		vi@15c10000 {
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					imx324_vi_in0: endpopoint {
                                                vc-id = <0>;
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&imx324_csi_out0>;
					};
				};
			};
		};

		nvcsi@15a00000 {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						imx324_csi_in0: endpoint@0 {
							port-index = <0>;
							bus-width = <4>;
							remote-endpoint = <&imx324_imx324_out0>;
						};
					};
                                        port@1 {
                                                reg = <1>;
                                                imx324_csi_out0: endpoint@1 {
                                                        remote-endpoint = <&imx324_vi_in0>;
                                                };
                                        };
				};
			};
		};
	};

	i2c@3180000 {
		tca9546@70 {
			i2c@0 {
			imx324_a@1a {
				compatible = "nvidia,imx324";

				reg = <0x1a>;

                                /* V4L2 device node location */
                                devnode = "video0";

				/* Physical dimensions of sensor */
				physical_w = "15.0";
				physical_h = "12.5";

				sensor_model ="imx324";

				/* Defines number of frames to be dropped by driver internally after applying */
				/* sensor crop settings. Some sensors send corrupt frames after applying */
				/* crop co-ordinates */
				post_crop_frame_drop = "0";

				/* Convert Gain to unit of dB (decibel) befor passing to kernel driver */
				//use_decibel_gain = "true";

				/* enable CID_SENSOR_MODE_ID for sensor modes selection */
				use_sensor_mode_id = "true";

				/**
				* A modeX node is required to support v4l2 driver
				* implementation with NVIDIA camera software stack
				*
				* mclk_khz = "";
				* Standard MIPI driving clock, typically 24MHz
				*
				* num_lanes = "";
				* Number of lane channels sensor is programmed to output
				*
				* tegra_sinterface = "";
				* The base tegra serial interface lanes are connected to
				*
				* vc_id = "";
				* The virtual channel id of the sensor.
				*
				* discontinuous_clk = "";
				* The sensor is programmed to use a discontinuous clock on MIPI lanes
				*
				* dpcm_enable = "true";
				* The sensor is programmed to use a DPCM modes
				*
				* cil_settletime = "";
				* MIPI lane settle time value.
				* A "0" value attempts to autocalibrate based on mclk_multiplier
				*
				* active_w = "";
				* Pixel active region width
				*
				* active_h = "";
				* Pixel active region height
				*
				* dynamic_pixel_bit_depth = "";
				* sensor dynamic bit depth for sensor mode
				*
				* csi_pixel_bit_depth = "";
				* sensor output bit depth for sensor mode
				*
				* mode_type="";
				* Sensor mode type, For eg: yuv, Rgb, bayer, bayer_wdr_pwl
				*
				* pixel_phase="";
				* Pixel phase for sensor mode, For eg: rggb, vyuy, rgb888
				*
				* readout_orientation = "0";
				* Based on camera module orientation.
				* Only change readout_orientation if you specifically
				* Program a different readout order for this mode
				*
				* line_length = "";
				* Pixel line length (width) for sensor mode.
				* This is used to calibrate features in our camera stack.
				*
				* mclk_multiplier = "";
				* Multiplier to MCLK to help time hardware capture sequence
				* TODO: Assign to PLL_Multiplier as well until fixed in core
				*
				* pix_clk_hz = "";
				* Sensor pixel clock used for calculations like exposure and framerate
				*
				*
				*
				*
				* inherent_gain = "";
				* Gain obtained inherently from mode (ie. pixel binning)
				*
				* min_gain_val = ""; (floor to 6 decimal places)
				* max_gain_val = ""; (floor to 6 decimal places)
				* Gain limits for mode
				* if use_decibel_gain = "true", please set the gain as decibel
				*
				* min_exp_time = ""; (ceil to integer)
				* max_exp_time = ""; (ceil to integer)
				* Exposure Time limits for mode (us)
				*
				*
				* min_hdr_ratio = "";
				* max_hdr_ratio = "";
				* HDR Ratio limits for mode
				*
				* min_framerate = "";
				* max_framerate = "";
				* Framerate limits for mode (fps)
				*
				* embedded_metadata_height = "";
				* Sensor embedded metadata height in units of rows.
				* If sensor does not support embedded metadata value should be 0.
				*/
				mode0 {/*mode AWR2243_MODE_2048X384*/
					vc_id="0";
					mclk_khz = "24000";
					num_lanes = "4";
					tegra_sinterface = "serial_a";
                                        phy_mode = "DPHY";
					discontinuous_clk = "yes";
					dpcm_enable = "false";
					cil_settletime = "0";
					dynamic_pixel_bit_depth = "12";
					csi_pixel_bit_depth = "12";
					mode_type = "bayer";
					pixel_phase = "rggb";

					active_w = "2048";
					active_h = "384";
					readout_orientation = "0";
					line_length = "2048";
					inherent_gain = "1";
					mclk_multiplier = "15.3";  
					pix_clk_hz = "367200000";  

					gain_factor = "10";
					min_gain_val = "0"; /* dB */
					max_gain_val = "300"; /* dB */
					step_gain_val = "1"; /* 0.1 */
					default_gain = "0";
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					framerate_factor = "1000000";
					min_framerate = "20000000";
					max_framerate = "30000000";
					step_framerate = "1";
					default_framerate = "25000000";
					exposure_factor = "1000000";
					min_exp_time = "59"; /*us, 2 lines*/
					max_exp_time = "55556";
					step_exp_time = "1";
					default_exp_time = "556";/* us */
					embedded_metadata_height = "0";
				};
		
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						imx324_imx324_out0: endpoint {
							port-index = <0>;
							bus-width = <4>;
                                                        vc-id = <0>;
							remote-endpoint = <&imx324_csi_in0>;
							};
						};
					};
				gmsl-link {
					src-csi-port = "a";
					dst-csi-port = "a";
					serdes-csi-link = "a";
					csi-mode = "1x4";
					num-lanes = <4>;
					streams = "ued-u1", "raw12";
					};
				};				
			}; /*i2c@0*/
		};
	};
};

/ {

	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		/**
		* Physical settings to calculate max ISO BW
		*
		* num_csi_lanes = <>;
		* Total number of CSI lanes when all cameras are active
		*
		* max_lane_speed = <>;
		* Max lane speed in Kbit/s
		*
		* min_bits_per_pixel = <>;
		* Min bits per pixel
		*
		* vi_peak_byte_per_pixel = <>;
		* Max byte per pixel for the VI ISO case
		*
		* vi_bw_margin_pct = <>;
		* Vi bandwidth margin in percentage
		*
		* max_pixel_rate = <>;
		* Max pixel rate in Kpixel/s for the ISP ISO case
		*
		* isp_peak_byte_per_pixel = <>;
		* Max byte per pixel for the ISP ISO case
		*
		* isp_bw_margin_pct = <>;
		* Isp bandwidth margin in percentage
		*/
		num_csi_lanes = <4>;
		max_lane_speed = <1100000>;    
		min_bits_per_pixel = <12>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		/**
		 * The general guideline for naming badge_info contains 3 parts, and is as follows,
		 * The first part is the camera_board_id for the module; if the module is in a FFD
		 * platform, then use the platform name for this part.
		 * The second part contains the position of the module, ex. "rear" or "front".
		 * The third part contains the last 6 characters of a part number which is found
		 * in the module's specsheet from the vender.
		 */
		modules {
			module0 {
				badge = "imx324_front";
				position = "front";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "imx324 30-001a";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/tca9546@70/i2c@0/imx324_a@1a";
				};
			};
		};
	};
};

I boosted the clock, and open the debug traces.
Below is the trace log:

     kworker/0:3-2165  [000] ....   214.645951: rtos_queue_peek_from_isr_failed: tstamp:7146539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   214.757939: rtos_queue_peek_from_isr_failed: tstamp:7151539895 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   214.925941: rtos_queue_peek_from_isr_failed: tstamp:7156539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.093982: rtos_queue_peek_from_isr_failed: tstamp:7161539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.261946: rtos_queue_peek_from_isr_failed: tstamp:7166539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.429940: rtos_queue_peek_from_isr_failed: tstamp:7171539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.597947: rtos_queue_peek_from_isr_failed: tstamp:7176539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.765994: rtos_queue_peek_from_isr_failed: tstamp:7181539896 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   215.877959: rtos_queue_peek_from_isr_failed: tstamp:7186539894 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.049964: rtos_queue_peek_from_isr_failed: tstamp:7191539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.218003: rtos_queue_peek_from_isr_failed: tstamp:7196539901 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.385938: rtos_queue_peek_from_isr_failed: tstamp:7201539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.721967: rtos_queue_peek_from_isr_failed: tstamp:7211539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   216.837977: rtos_queue_peek_from_isr_failed: tstamp:7216539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.005976: rtos_queue_peek_from_isr_failed: tstamp:7221539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.173941: rtos_queue_peek_from_isr_failed: tstamp:7226539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.509993: rtos_queue_peek_from_isr_failed: tstamp:7236539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.677982: rtos_queue_peek_from_isr_failed: tstamp:7241539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   217.957954: rtos_queue_peek_from_isr_failed: tstamp:7251539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   218.125936: rtos_queue_peek_from_isr_failed: tstamp:7256539892 queue:0x0bcbbbb8
     kworker/0:3-2165  [000] ....   218.237955: rtos_queue_peek_from_isr_failed: tstamp:7259287510 queue:0x0bcbbbb8
        v4l2-ctl-9515  [003] ....   227.820424: tegra_channel_open: vi-output, imx324 30-001a
        v4l2-ctl-9515  [001] ....   227.832877: tegra_channel_set_power: imx324 30-001a : 0x1
        v4l2-ctl-9515  [001] ....   227.832913: camera_common_s_power: status : 0x1
        v4l2-ctl-9515  [001] ....   227.834070: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9515  [001] ....   227.834079: csi_s_power: enable : 0x1
        v4l2-ctl-9515  [001] ....   227.843742: tegra_channel_set_stream: enable : 0x1
        v4l2-ctl-9515  [001] ....   227.854937: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x1
        v4l2-ctl-9515  [001] ....   227.854941: csi_s_stream: enable : 0x1
        v4l2-ctl-9515  [001] ....   227.854943: tegra_channel_set_stream: imx324 30-001a : 0x1
        v4l2-ctl-9515  [001] ....   232.740804: tegra_channel_close: vi-output, imx324 30-001a
        v4l2-ctl-9515  [001] ....   232.740888: tegra_channel_set_stream: enable : 0x0
        v4l2-ctl-9515  [001] ....   232.740891: tegra_channel_set_stream: imx324 30-001a : 0x0
        v4l2-ctl-9515  [001] ....   232.742899: tegra_channel_set_stream: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9515  [001] ....   232.742903: csi_s_stream: enable : 0x0
        v4l2-ctl-9515  [000] ....   232.745340: tegra_channel_set_power: imx324 30-001a : 0x0
        v4l2-ctl-9515  [000] ....   232.745347: camera_common_s_power: status : 0x0
        v4l2-ctl-9515  [000] ....   232.745545: tegra_channel_set_power: 15a00000.nvcsi--1 : 0x0
        v4l2-ctl-9515  [000] ....   232.745548: csi_s_power: enable : 0x0

It seems like the Jetson doesn’t receive FS even though it is being transmitted on the lines.

Also, when running the command:

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=2048, height=384,

I get the following:

format=NV12' ! appsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2048 x 384 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 0.000000, max 30.000000; Exposure Range min 59000, max 55556000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 2048 H = 384
   seconds to Run    = 0
   Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...
CONSUMER: Done Success
Got EOS from element "pipeline0".
Execution ended after 0:00:04.006548640
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up

Any idea?

Yes, from the trace shows didn’t receive any validate data from MIPI bus.
I would suggest to configure SER/DES output test partner to debug.

Issue was resolved.
This was caused by Xavier getting CSI clock before data.

Thanks,
Ron

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.