Vi error “no reply from camera processor” for custom camera driver

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
,
1

I am writing a custom camera driver on Nvidia Xavier NX platform for a camera sensor from Infineon. i2c communication with sensor is already working fine. My sensor is connected on csi channel 3. There are three other camera sensors connected to Jetson and hence my video device is registered as /dev/video3 and comes on i2c-12
My driver is based upon L4T 32.6.1 (kernel: 4.9.253-tegra)

Below is my issue when I try to start streaming:

Driver is inserted:    
#                                       
[  113.249405] irs 12-003d: camera_common_parse_ports: port 3 num of lanes 2
[  113.249716] irs 12-003d: irs_probe: name irs_d
[  113.249822] irs 12-003d: camera_common_create_debugfs irs_d
[  113.249993] irs 12-003d: camera_common_mclk_enable: enable MCLK with 24000000 Hz
[  113.250565] irs 12-003d: camera_common_dpd_disable: csi 3
[  113.301200] tegra194-vi5 15c10000.vi: subdev irs 12-003d bound
[  113.301503] tegra194-vi5 15c10000.vi: notify complete, all subdevs registered
[  113.303616] tegra194-vi5 15c10000.vi: creating links for entity 15a00000.nvcsi--1
[  113.303854] tegra194-vi5 15c10000.vi: processing endpoint /host1x/nvcsi@15a00000/channel@3/ports/port@0/endpoint
[  113.305292] tegra194-vi5 15c10000.vi: skipping sink port /host1x/nvcsi@15a00000/channel@3:0
[  113.305519] tegra194-vi5 15c10000.vi: processing endpoint /host1x/nvcsi@15a00000/channel@3/ports/port@1/endpoint
[  113.305945] tegra194-vi5 15c10000.vi: skipping channel port /host1x/nvcsi@15a00000/channel@3:1
[  113.306121] tegra194-vi5 15c10000.vi: creating links for entity irs 12-003d
[  113.306290] tegra194-vi5 15c10000.vi: processing endpoint /cam_i2cmux/i2c@3/irs@3d/ports/port@0/endpoint
[  113.306546] tegra194-vi5 15c10000.vi: creating irs 12-003d:0 -> 15a00000.nvcsi--1:0 link
[  113.306757] tegra194-vi5 15c10000.vi: creating links for channels
[  113.306908] tegra194-vi5 15c10000.vi: processing endpoint /host1x/vi@15c10000/ports/port@3/endpoint
[  113.309556] tegra194-vi5 15c10000.vi: creating link for channel 15c10000.vi-output-3
[  113.317428] tegra194-vi5 15c10000.vi: creating 15a00000.nvcsi--1:1 -> 15c10000.vi-output-3:0 link
[  113.326948] irs 12-003d: camera_common_g_fmt++
# 
 
 
 Streaming:
 
 #v4l2-ctl -d /dev/video3 --set-fmt-video=width=224,height=1557,pixelformat=BG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=data.raw
[  399.579177] irs 12-003d: camera_common_mclk_enable: enable MCLK with 24000000 Hz
[  399.580357] irs 12-003d: camera_common_dpd_disable: csi 3
[  399.585888] irs 12-003d: camera_common_try_fmt: size 224 x 1557
[  399.586095] irs 12-003d: camera_common_s_fmt(12296) size 224 x 1557
VIDIOC_S_FMT: failed: Invalid argument
[  399.592044] tegra194-vi5 15c10000.vi: vi_capture_init++
[  399.592542] tegra194-vi5 15c10000.vi: vi_channel_power_on_vi_device
[  399.593363] tegra194-vi5 15c10000.vi: chan flags 4163
[  399.593522] tegra194-vi5 15c10000.vi: chan mask ffffffffffffffff
[  399.593674] tegra194-vi5 15c10000.vi: queue depth 4
[  399.593804] tegra194-vi5 15c10000.vi: request size 704
[  399.610691] tegra194-vi5 15c10000.vi: 6 GoS tables configured.
[  399.610926] tegra194-vi5 15c10000.vi: gos[0] = 0xc2100000
[  399.611080] tegra194-vi5 15c10000.vi: gos[1] = 0xc2101000
[  399.611223] tegra194-vi5 15c10000.vi: gos[2] = 0xc2102000
[  399.611360] tegra194-vi5 15c10000.vi: gos[3] = 0xc2103000
[  399.611505] tegra194-vi5 15c10000.vi: gos[4] = 0xc2104000
[  399.611736] tegra194-vi5 15c10000.vi: gos[5] = 0xc2105000
[  399.611896] tegra194-vi5 15c10000.vi: vi_capture_ivc_send_control: sending chan_id 64 msg_id 16
[  399.614482] tegra194-vi5 15c10000.vi: vi_capture_ivc_send_control: response chan_id 64 msg_id 17
[  399.615295] tegra194-vi5 15c10000.vi: vi_capture_request: sending chan_id 0 msg_id 1 buf:0
[  399.615954] tegra194-vi5 15c10000.vi: vi_capture_request: sending chan_id 0 msg_id 1 buf:1
[  399.615998] tegra194-vi5 15c10000.vi: vi_capture_status: waiting for status, timeout:5000 ms
[  399.632250] tegra194-vi5 15c10000.vi: vi_capture_request: sending chan_id 0 msg_id 1 buf:2
[  399.640756] tegra194-vi5 15c10000.vi: vi_capture_request: sending chan_id 0 msg_id 1 buf:3
[  399.663669] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2

[  404.827707] tegra194-vi5 15c10000.vi: no reply from camera processor
[  404.827867] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 5000 ms
[  404.828051] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
[  404.828226] tegra194-vi5 15c10000.vi: vi_channel_power_off_vi_device
[  404.831634] tegra194-vi5 15c10000.vi: vi_capture_shutdown--

Below is my device tree entry:
I do not have regulator (avdd-reg, iovdd-reg etc.) in my device tree node. I am not sure if I should have one?

host1x {
        vi@15c10000 {
            num-channels = <4>;
            ports {
                #address-cells = <1>;
                #size-cells = <0>;
                
                	vi_port3: port@3 {		
			reg = <3>;
			status = "okay";
			irs_vi_in0: endpoint {
				status = "okay";
				port-index = <3>;  //CSI 3 <–> CSI D <–> port-index= 3
				bus-width = <2>;
				remote-endpoint = <&irs_csi_out0>;
			};
		};
            };
    };


  nvcsi@15a00000 {
            num-channels = <4>;
            #address-cells = <1>;
            #size-cells = <0>;
            
             csi_chan3: channel@3 {
		reg = <3>;
		status = "okay";
		ports {
			#address-cells = <1>;
			#size-cells = <0>;
			csi_chan3_port0: port@0 {       //===> Not sure @0 (number matters)?
				reg = <0>;
				status = "okay";
				irs_csi_in0: endpoint {
					port-index = <3>;    //CSI 3 <–> CSI D <–> port-index= 3
					bus-width = <2>;
					status = "okay";
					remote-endpoint = <&irs_out0>;
				};
			};
			csi_chan3_port1: port@1 {       //===> Not sure @1 (number matters)?
				reg = <1>;
				status = "okay";
				irs_csi_out0: endpoint {
				status = "okay";
				remote-endpoint = <&irs_vi_in0>;
				};
			};
		};
	};
  };
  
  
   cam_i2cmux {
   
   i2c_3:i2c@3 {		
			irs_cam3: irs@3d {
			compatible = "infineon,irs";
			/* I2C device address */
			reg = <0x3d>; 

			/* MCLK */ /*TODO*/ /*The default value is extperiph1.*/
			clocks = <&bpmp_clks TEGRA194_CLK_EXTPERIPH1>;
                        clock-names = "extperiph1";
                        mclk = "extperiph1";

			/* V4L2 device node location */
			devnode = "video3";

			use_sensor_mode_id = "false";
			
			// This node is needed by the Tegra framework. 
                // All properties are set by the driver on runtime.
		
                mode0 { 
                    num_lanes                = "2";
                    tegra_sinterface         = "serial_d";     //=====> should be "serial_d" for csi 3?
                    phy_mode                 = "DPHY";

                    discontinuous_clk        = "no";
                    mclk_khz                 = "24000";
                    pix_clk_hz               = "3487680";
                    cil_settletime           = "0";

                    dpcm_enable              = "false";
                    mclk_multiplier          = "1";

                    inherent_gain            = "1";
                    min_hdr_ratio            = "1";
                    max_hdr_ratio            = "1";
                    readout_orientation      = "0";

                    active_w                 = "1557";
                    active_h                 = "224";
                    line_length              = "1000";
                    pixel_t                  = "bayer_bggr12";

                    gain_factor              = "1";
                    min_gain_val             = "0";
                    max_gain_val             = "100000";
                    step_gain_val            = "1";
                    default_gain             = "0";

                    exposure_factor          = "1";
                    min_exp_time             = "0";
                    max_exp_time             = "100000000";
                    step_exp_time            = "1";
                    default_exp_time         = "10000";

                    framerate_factor         = "1";
                    min_framerate            = "0";
                    max_framerate            = "1000";
                    step_framerate           = "1";
                    default_framerate        = "1000";

                    embedded_metadata_height = "2";
                };

			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					irs_out0: endpoint {
						port-index = <3>;  //CSI 3 <–> CSI D <–> port-index= 3
						bus-width = <2>;
						remote-endpoint = <&irs_csi_in0>;
					};
				};
			};
    		};
    };

Issue media-ctl command (Only intended sensor output snipped):

#media-ctl -p -d /dev/media0
Media controller API version 0.1.0

Media device information
------------------------
driver          tegra194-vi5
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  0.0.0

Device topology


- entity 52: 15a00000.nvcsi--1 (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
pad0: Sink
<- "irs 12-003d":0 [ENABLED]
pad1: Source
-> "vi-output, irs 12-003d":0 [ENABLED]

- entity 55: irs 12-003d (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev7
pad0: Source
[  164.202113] irs 12-003d: camera_common_g_fmt++
[fmt:SBGGR12_1X12/224x1557 field:none colorspace:srgb]
-> "15a00000.nvcsi--1":0 [ENABLED]

- entity 57: vi-output, irs 12-003d (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
pad0: Sink
<- "15a00000.nvcsi--1":1 [ENABLED]

#

I also do not implement v4l2-ctrls like gain, exposure setting !!

# v4l2-ctl --all -d /dev/video3 --verbose
[  767.556552] irs 12-003d: camera_common_mclk_enable: enable MCLK with 24000000 Hz
[  767.557022] irs 12-003d: camera_common_dpd_disable: csi 3
VIDIOC_QUERYCAP: ok
Driver Info (not using libv4l2):
Driver name   : tegra-video
Card type     : vi-output, irs 12-003d
Bus info      : platform:15c10000.vi:3
Driver version: 4.9.253
Capabilities  : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps   : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 2
**Video input : 0 (Camera 3: no power)**   ======> This doesn't looks to be good
Format Video Capture:
Width/Height      : 224/1557
Pixel Format      : 'BG12'
Field             : None
Bytes per Line    : 448
Size Image        : 697536
Colorspace        : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization      : Default (maps to Full Range)
Flags             :

Camera Controls

                        fuse_id 0x009a2007 (str)    : min=0 max=32 step=2 value='' flags=read-only, has-payload
           sensor_configuration 0x009a2032 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
         sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
      sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only
[  767.698587] irs 12-003d: camera_common_dpd_enable: csi 3
[  767.704313] irs 12-003d: camera_common_mclk_disable: disable MCLK
#

I will really appriciate any help !! Thank you.

3

hello sau,

please refer to Debugging Tips session for some steps to review your driver.
in order to resolve clock configuration issue, you may enable below commands to configure clocks to boost all the clocks.
for example,

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
4

Hello @JerryChang
Thank you for your reply. I have executed the command as mentioned above:

# echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
# echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
# echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
# cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
460800000
# cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
576000000
# cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate314000000
#

Also enable debug (from my side) for few files:

echo 'file camera_common.c +p' > /sys/kernel/debug/dynamic_debug/control
echo 'file capture_vi_channel.c +p' > /sys/kernel/debug/dynamic_debug/control
echo 'file capture.c +p' > /sys/kernel/debug/dynamic_debug/control
echo 'file sensor_common.c +p' > /sys/kernel/debug/dynamic_debug/control
5

There are also few point I want to present here:

  1. I do not write mode tables from my driver.

  2. I might not have correct data for active_h, active_w and pixel_clk_hz, active_w, active_h, line_length, min/max_gain_val, cil_settletimein the mode0 {}; node.
    I have a user application which should set gain, exposure etc (through sending i2c commands and control through /dev/video3).
    But I am not sure here that what is absolutely minimum required for the sensor, in order to start/test streaming. This means without these minimum sensor settings I see the errors which I am seeing right now.

  3. There are properties like num_lanes = “2”, tegra_sinterface = “serial_d” (corresponding to CSI channel 3), pixel_t and mclk_khz, I feel I have put correctly.

  4. I will request you to please once review the device tree I prepared.

  5. I am concerned at the point that “Video input : 0 (Camera 3: no power)” while querying the proprieties of /dev/video3. I am not sure what “no power” corresponds to here and from which level this is represented here.

Thank you once again.

6

hello sau,

  1. please check kernel init logs during booting-up; there’s sensor registration during kernel initialization, it might report an error if you have regulator related issue,
  2. you must have min/max property settings since it’s device tree to report sensor hardware capability to VI driver, please looking for sensor specification and complete the mode settings.
  3. mode table should be ok, but you should ensure you’ve complete register/signal configuration for camera sending out MIPI signaling.
  4. furthermore, please see-also V4L2 Sensor Driver Development Tutorial, a training video for writing a complete V4L2 compliant driver for an image sensor to connect to the NVIDIA Jetson platform over MIPI CSI-2.
8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.