Mipi driver works on jetson nano but fails in jetson tx2

Hello,

I have been developing a custom v4l2 driver, first for Jetson nano and now I want to implement the driver inside a TX2-4GB device. The goal is to receive mipi frames from Lattice FPGA.

I am working with L4T 32.3.1.

I was able to succesfully communicate nano with FPGA using I2C and receive frames, but now I am using the same driver source code, and kernel tree file inside TX2 but the streaming is not working.

The parameters like mclk_multiplier, pix_clk_hz and cil_settle_time are the same, beacuse the streaming is not changing at all.

To receive streams in the jetson nano I connect with the FPGA developer kit using the J13/Cam0 connector.
To receive streams in the TX2 I am using a custom carrier board connected to the FPGA developer kit.

In the TX2, the driver is able to detect the FPGA and /dev/video0 is enabled, I can see in the logs that it is working fine, but when I start the streaming I have the error:

tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel

with an oscilloscope I am able to see that FPGA is streaming so I gess the problem may be in the TX2.
Right now I am not able to connect the FPGA developer kit with the TX2 developer kit because I dont have the J22 connector.

Is there any difference with the Jetson nano and the TX2 in the mipi driver configuration?

Thank you

Enable the trace log to check more information.

https://elinux.org/Jetson_TX2_Camera_BringUp

Hello ShaneCCC

I have enable more logs with

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

also tried with

export enableCamPclLogs=5
export enableCamScfLogs=5
echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

In dmesg I dont see any difference.

In TX2 /sys/kernel/debug/tracing/trace I see

libBeamCamContr-8506  [005] ....  1678.646454: tegra_channel_open: vi-output, bmg_ldr 2-003e
 libBeamCamContr-8506  [000] ....  1678.647648: tegra_channel_set_power: bmg_ldr 2-003e : 0x1
 libBeamCamContr-8506  [000] ....  1678.650464: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x1
 libBeamCamContr-8506  [000] ....  1678.650466: csi_s_power: enable : 0x1
 libBeamCamContr-8506  [005] ....  1678.759940: tegra_channel_capture_setup: vnc_id 0 W 3664 H 150 fmt 10
 vi-output, bmg_-8531  [003] ....  1678.760036: tegra_channel_set_stream: enable : 0x1
 vi-output, bmg_-8531  [005] ....  1678.761276: tegra_channel_set_stream: 150c0000.nvcsi--1 : 0x1
 vi-output, bmg_-8531  [005] ....  1678.761279: csi_s_stream: enable : 0x1
 vi-output, bmg_-8531  [005] ....  1678.761305: tegra_channel_set_stream: bmg_ldr 2-003e : 0x1
     kworker/3:0-7670  [003] ....  1678.782436: rtos_queue_peek_from_isr_failed: tstamp:52768450966 queue:0x0b4b4500
     kworker/3:0-7670  [003] ....  1678.782439: rtcpu_start: tstamp:52768452195
     kworker/3:0-7670  [003] ....  1678.782441: rtos_queue_send_from_isr_failed: tstamp:52768461522 queue:0x0b4a7258

The error means that 0 package from the MIPI bus has been received, but as I said, if I connect an oscilloscope to the mipi lanes I can see the clock and the frames data.

In Jetson nano /sys/kernel/debug/tracing/trace I see

 libBeamCamContr-8639  [002] ....  373.778635: tegra_channel_open: vi-output, bmg_ldr 7-003e
 libBeamCamContr-8639  [000] ....  373.794248: tegra_channel_set_power: bmg_ldr 7-003e : 0x1
 libBeamCamContr-8639  [000] ....  373.797083: tegra_channel_set_power: 150c0000.nvcsi--2 : 0x1
 libBeamCamContr-8639  [000] ....  373.797087: csi_s_power: enable : 0x1
 vi-output, bmg_-8686  [003] .n..   406.048046: tegra_channel_set_stream: enable : 0x1
 vi-output, bmg_-8686  [001] ....   406.048206: tegra_channel_set_stream: nvcsi--2 : 0x1
 vi-output, bmg_-8686  [001] .... 406.048210: csi_s_stream:enable : 0x1
 vi-output, bmg_-8686  [001] .... 406.048220: tegra_channel_set_stream: bmg_ldr 7-003e : 0x1
 vi-output, bmg_-8686  [001] .... 406.116260: tegra_channel_capture_frame:  sof:405.970194130

bmg_ldr is the name of my custom mipi driver it uses the I2C address 0x3e

As you can see in the TX2 it tries to set a image format W 3664 H 150 fmt 10 but it is not the one I am setting, even it is one of the possible that I have defined in the kernel tree, actually I am setting the format W 14464 H 150* which is also defined in the kernel tree.

In the kernel tree, the first mode *mode0 is defined as shown:

mode0 { //! BMG_LDR_MODE_3664x150_10FPS 

					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					phy_mode = "DPHY";
					discontinuous_clk = "yes";
					dpcm_enable = "false";
					cil_settletime = "0";

					active_w = "3664";
					active_h = "150";
					mode_type = "bayer";
					pixel_phase = "bggr";	
					csi_pixel_bit_depth = "8";								
					readout_orientation = "0";
					line_length = "3664";
					inherent_gain = "1";
					mclk_multiplier = "9.33";
					pix_clk_hz = "83333333";


					gain_factor = "1";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "0";  
					max_gain_val = "250";
					step_gain_val = "1";
					default_gain = "1";  
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; //! 2.0 fps 
					max_framerate = "10000000"; //! 5.0 fps 
					step_framerate = "1";
					default_framerate = "10000000"; //! 5.0 fps 
					min_exp_time = "1000000"; //! us 
					max_exp_time = "10000000"; //! us 
					step_exp_time = "1";
					default_exp_time = "1000000"; //! us 

					embedded_metadata_height = "0";
				};

I am using the same application in both jetson TX2 and nano to start the streaming, so I dont understand why the TX2 is setting a different capture configuration/mode

Thank you

Get the trace log after run the v4l2-ctl or some others APP to capture by “cat /sys/kernel/debug/tracing/trace” again

Hi ShaneCCC,

First I have modified the driver to look like this:

static int bmg_ldr_set_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_pad_config *cfg,
		struct v4l2_subdev_format *format){
   int ret;

   printk(KERN_ERR "bmg_ldr_set_fmt W=%d H=%d \n", (int)format->format.width, (int)format->format.height);

   if(format->format.code != 12289){
       return -1;
  }
  //!check if format is available
  ret = camera_common_s_fmt(sd, &format->format);

  return ret;
}

static struct v4l2_subdev_pad_ops bmg_ldr_subdev_pad_ops = {
	.set_fmt = bmg_ldr_set_fmt,
	.get_fmt = bmg_ldr_get_fmt,
	.enum_mbus_code = bmg_ldr_enum_mbus_code,
	.enum_frame_size = bmg_ldr_enum_framesizes,
	.enum_frame_interval = bmg_ldr_enum_frameintervals,
};

Then I use the command:

v4l2-ctl -d 0 --set-fmt-video width=14464,height=150,pixelformat='BA81'

The response in the dmesg -w for the Jetson nano is:

[133.693302] bmg_ldr_set_fmt W=14464 H=150
[133.698295] bmg_ldr_set_fmt camera_common_s_fmt response 0
[133.704006] bmg_ldr_set_fmt W=14464 H=150
[133.708878] bmg_ldr_set_fmt camera_common_s_fmt response 0

The response in the dmesg -w for the Jetson TX2 is:

[ 1503.723620] bmg_ldr_set_fmt W=14464 H=150 
[ 1503.727830] bmg_ldr_set_fmt camera_common_s_fmt response 0
[ 1503.733626] bmg_ldr_set_fmt W=3664 H=150 
[ 1503.738006] bmg_ldr_set_fmt camera_common_s_fmt response 0

As I said before the configuration 3664x150 should be supported, but with that configuration the lattice is not working, but that is a different issue, I know that lattice works with 14464x150, and it works fine in the Jetson nano.

Why does the TX2 sets a different format if the driver code and kernel tree are the same?

Thank you

TX2 CSI/VI driver have check more think like embedded data or frame size. That’s why we need to check the trace log if can get more information.

Hello ShaneCCC

I have enabled the debug as said in post 3

Here you can see the v4l2-ctl commands I am using to initialize the Lattice streaming

Here you can see the /var/syslog messages

Here you can see /var/kern.log messages

Here you can see dmesg -w messages

Here you can see /sys/kernel/debug/tracing/trace messages

As I said before, I still dont understand why the CSI/VI driver is changing the v4l2-format configuration from 14464x150 to 3664x150.

I have also tried with a very simple cpp program but the result is the same, when the xioctl function executes it is setting the wrong format

#include <sys/ioctl.h>
#include <fcntl.h>
#include <iostream>
#include <unistd.h>
#include <linux/videodev2.h>
#include <string.h>

#define CLEAR(x) memset(&(x), 0, sizeof(x))

int xioctl(int fd, int request, void *arg){
    int result = 0;
    do{
        result = ioctl(fd, request, arg);
    }
    while(-1 == result && EINTR == errno);

    return result;
}

void openDevice(int &fd){

   fd = open("/dev/video0", O_RDWR | O_NONBLOCK, 0);
   
}

int closeDevice(int &fd){
   return close(fd);
}

int getVideoFormat(int fd){
   
    int response = 0;
    v4l2_format fmt;
    CLEAR(fmt);


    response = xioctl(fd, VIDIOC_G_FMT, &fmt);

    if(response){
       printf("W = %d\n",(int)fmt.fmt.pix.width);
       printf("H = %d\n",(int)fmt.fmt.pix.height);
    } 
  
    return response; 
    
}

int setVideoFormat(int fd){
    int response = 0;

    v4l2_format fmt;

    fmt.fmt.pix.pixelformat= V4L2_PIX_FMT_SBGGR8;
    fmt.fmt.pix.height = 150;
    fmt.fmt.pix.width = 14464;//!3664;14464;
    fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    fmt.fmt.pix.field = V4L2_FIELD_NONE;
    fmt.fmt.pix.bytesperline = 14464;

    response = xioctl(fd, VIDIOC_S_FMT, &fmt);

    return response;

}

int main(){
   
   int response = 0;
   printf("Starting V4L2-Driver test\n");
   printf("Opening device\n");
   
   int fd = 0;   
   openDevice(fd);

   if(fd == -1){
	printf("Error opening device\n");
        return -1;
   }
   else{
       printf("video device open succesfully\n");
   }


   printf("Set format\n");
   response = setVideoFormat(fd);

   if(response == 0){
     printf("video format succesful\n");
   }else{
     printf("Error setting video format %d\n", response);
   }

   //!printf("Get current format\n");

   //!response = getVideoFormat(fd);

   response = closeDevice(fd);
   if(response != 0){
      printf("Error closing device\n");
   }else{
      printf("Video device close succesfully\n");
      fd = -1;
   }
   
   return response; 
}

Thank you

I think bmg_ldr_set should be your sensor driver to changing the v4l2-format configuration from 14464x150 to 3664x150

Hello ShaneCCC
To configure the device resolution I use I2C registers in the lattice, this is working in jetson nano, so if v4l2-ctl, or to be more specific camera_common.c sets the mode, as I understand, to CSI/VI driver, please correct me if I am wrong.

I have seen an error in my kernel tree configuration, the value

use_sensor_mode_id = "true";

has to be “false” in order to the v4l2-ctl or any application that uses VIDIOC_S_FMT to set the correct format specified in the structure.

So now I can see that when I set the format, the camera_common.c is setting the right format, but I still getting the error:

[  228.803295] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  228.809968] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel

Here you can see that tegra_channel_capture_setup has the right configuration:

root@beamagine-dev:/sys/kernel/debug/dynamic_debug# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 292/292   #P:4
#
#                              _-----=> irqs-off
#                             / _----=> need-resched
#                            | / _---=> hardirq/softirq
#                            || / _--=> preempt-depth
#                            ||| /     delay
#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
#              | |       |   ||||       |         |
        v4l2-ctl-8028  [004] ....   902.843116: tegra_channel_open: vi-output, bmg_ldr 2-003e
        v4l2-ctl-8028  [003] ....   902.845883: tegra_channel_set_power: bmg_ldr 2-003e : 0x1
        v4l2-ctl-8028  [003] ....   902.848733: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x1
        v4l2-ctl-8028  [003] ....   902.848738: csi_s_power: enable : 0x1
        v4l2-ctl-8028  [000] ....   902.852062: tegra_channel_close: vi-output, bmg_ldr 2-003e
        v4l2-ctl-8028  [000] ....   902.852078: tegra_channel_set_power: bmg_ldr 2-003e : 0x0
        v4l2-ctl-8028  [000] ....   902.855048: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x0
        v4l2-ctl-8028  [000] ....   902.855055: csi_s_power: enable : 0x0
        v4l2-ctl-8049  [003] ....   909.916895: tegra_channel_open: vi-output, bmg_ldr 2-003e
        v4l2-ctl-8049  [003] ....   909.920724: tegra_channel_set_power: bmg_ldr 2-003e : 0x1
        v4l2-ctl-8049  [003] ....   909.923907: tegra_channel_set_power: 150c0000.nvcsi--1 : 0x1
        v4l2-ctl-8049  [003] ....   909.923945: csi_s_power: enable : 0x1
        v4l2-ctl-8049  [004] ....   909.942935: tegra_channel_capture_setup: vnc_id 0 W 14464 H 150 fmt 10
 vi-output, bmg_-8050  [003] ....   909.943167: tegra_channel_set_stream: enable : 0x1
     kworker/3:0-7900  [003] ....   909.954602: rtos_queue_peek_from_isr_failed: tstamp:28741195709 queue:0x0b4b4500
     kworker/3:0-7900  [003] ....   909.954641: rtcpu_start: tstamp:28741198085
     kworker/3:0-7900  [003] ....   909.954670: rtos_queue_send_from_isr_failed: tstamp:28741257215 queue:0x0b4a7258
     kworker/3:0-7900  [003] ....   909.954675: rtos_queue_send_from_isr_failed: tstamp:28741257355 queue:0x0b4aad68
     kworker/3:0-7900  [003] ....   909.954682: rtos_queue_send_from_isr_failed: tstamp:28741257493 queue:0x0b4ac998
     kworker/3:0-7900  [003] ....   909.954686: rtos_queue_send_from_isr_failed: tstamp:28741257634 queue:0x0b4ae518
     kworker/3:0-7900  [003] ....   909.954694: rtos_queue_send_from_isr_failed: tstamp:28741257775 queue:0x0b4af2d8
     kworker/3:0-7900  [003] ....   909.954697: rtos_queue_send_from_isr_failed: tstamp:28741257912 queue:0x0b4b0098
     kworker/3:0-7900  [003] ....   909.954706: rtos_queue_send_from_isr_failed: tstamp:28741258050 queue:0x0b4b0e58
     kworker/3:0-7900  [003] ....   909.954710: rtos_queue_send_from_isr_failed: tstamp:28741258187 queue:0x0b4b1c18
     kworker/3:0-7900  [003] ....   909.954735: rtos_queue_send_failed: tstamp:28741258812 queue:0x0b4a7258
     kworker/3:0-7900  [003] ....   909.954739: rtos_queue_send_from_isr_failed: tstamp:28741262936 queue:0x0b4a7258
     kworker/3:0-7900  [003] ....   909.954743: rtos_queue_send_from_isr_failed: tstamp:28741263073 queue:0x0b4aad68
     kworker/3:0-7900  [003] ....   909.954747: rtos_queue_send_from_isr_failed: tstamp:28741263212 queue:0x0b4ac998
     kworker/3:0-7900  [003] ....   909.954751: rtos_queue_send_from_isr_failed: tstamp:28741263351 queue:0x0b4ae518
     kworker/3:0-7900  [003] ....   909.954754: rtos_queue_send_from_isr_failed: tstamp:28741263488 queue:0x0b4af2d8
     kworker/3:0-7900  [003] ....   909.954758: rtos_queue_send_from_isr_failed: tstamp:28741263626 queue:0x0b4b0098
     kworker/3:0-7900  [003] ....   909.954762: rtos_queue_send_from_isr_failed: tstamp:28741263763 queue:0x0b4b0e58
     kworker/3:0-7900  [003] ....   909.954766: rtos_queue_send_from_isr_failed: tstamp:28741263899 queue:0x0b4b1c18
     kworker/3:0-7900  [003] ....   909.954770: rtos_queue_send_failed: tstamp:28741265338 queue:0x0b4a7258
 vi-output, bmg_-8050  [000] ....   909.957919: tegra_channel_set_stream: 150c0000.nvcsi--1 : 0x1
 vi-output, bmg_-8050  [000] ....   909.957922: csi_s_stream: enable : 0x1
 vi-output, bmg_-8050  [000] ....   909.957947: tegra_channel_set_stream: bmg_ldr 2-003e : 0x1

Here I attach the kernel tree I am using:

in hardware/nvidia/platform/t18x/common/kernel-dts/t18x-common-modules/tegra186-camera-bmg-ldr6cam-a00.dtsi

 /*
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

/* camera control gpio definitions */
/ {
	host1x {
		vi@15700000 {
			status = "okay";
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					status ="okay";
					reg = <0>;
					bmg_ldr_vi_in0: endpoint {
						status = "okay";
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&bmg_ldr_csi_out0>;
					};
				};
			};
		};
        
		nvcsi@150c0000 {
			status = "okay";
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			channel@0 {
				status = "okay";
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status = "okay";
						reg = <0>;
						bmg_ldr_csi_in0: endpoint@0 {
							status = "okay";
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&bmg_ldr_out0>;
						};
					};
					port@1 {
						status = "okay";
						reg = <1>;
						bmg_ldr_csi_out0: endpoint@1 {
							status = "okay";
							remote-endpoint = <&bmg_ldr_vi_in0>;
						};
					};
				};
			};
		};
	};

	i2c@3180000 {
		status = "okay";
		bmg_ldr_a@3e {
			status = "okay";
			devnode = "video0";
			compatible = "beamagine,bmg_ldr";
			reg = <0x3e>;

			physical_w = "14.408";
			physical_h = "200";

			sensor_model = "bmg_ldr";

			use_sensor_mode_id = "false";


			mode0 { //! BMG_LDR_MODE_3664x150_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
				cil_settletime = "0";
				
				active_w = "3664";
				active_h = "150";
				mode_type = "bayer";
				pixel_phase = "bggr";
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "3664";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "5496000";*/
				
				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 5.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 5.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "10000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1000000"; //! us 
				
				embedded_metadata_height = "0";
			};
			mode1 { //! BMG_LDR_MODE_7264x150_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";

				active_w = "7264";
				active_h = "150";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "7264";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "10896000";*/

				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 
				
				embedded_metadata_height = "0";
			};
			mode2 { //! BMG_LDR_MODE_10864x150_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
									
				active_w = "10864";
				active_h = "150";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "10864";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "16296000";*/
				
				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 
				
				embedded_metadata_height = "0";
			};
			mode3 { //! BMG_LDR_MODE_14464x150_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
									
				active_w = "14464";
				active_h = "150";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "14464";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "21696000";*/
				
				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 
				
				embedded_metadata_height = "0";
			};
			mode4 { //! BMG_LDR_MODE_3664x200_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
									
				active_w = "3664";
				active_h = "200";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "3664";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "7328000";*/
				
				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 5.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 5.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "10000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1000000"; //! us 
				
				embedded_metadata_height = "0";
			};
			mode5 { //! BMG_LDR_MODE_7264x200_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
					
				active_w = "7264";
				active_h = "200";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "7264";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "14528000";*/


				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 

				embedded_metadata_height = "0";
			};
			mode6 { //! BMG_LDR_MODE_10864x200_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
					
				active_w = "10864";
				active_h = "200";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "10864";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "21728000";*/

				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 

				embedded_metadata_height = "0";
			};
			mode7 { //! BMG_LDR_MODE_14464x200_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
					
				active_w = "14464";
				active_h = "200";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "14464";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*pix_clk_hz = "28928000";*/
				/*mclk_multiplier = "29.16";*/

				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 

				embedded_metadata_height = "0";
			};
			mode8 { //! BMG_LDR_MODE_11104x150_10FPS 

				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_a";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
                cil_settletime = "0";
					
				active_w = "11104";
				active_h = "150";
				mode_type = "bayer";
				pixel_phase = "bggr";	
				csi_pixel_bit_depth = "8";								
				readout_orientation = "0";
				line_length = "11104";
				inherent_gain = "1";
				mclk_multiplier = "9.33";
				pix_clk_hz = "83333333";
				/*mclk_multiplier = "29.16";*/
				/*pix_clk_hz = "16656000";*/

				gain_factor = "1";
				framerate_factor = "1000000";
				exposure_factor = "1000000";
				min_gain_val = "0";  
				max_gain_val = "250";
				step_gain_val = "1";
				default_gain = "1";  
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "2000000"; //! 2.0 fps 
				max_framerate = "10000000"; //! 10.0 fps 
				step_framerate = "1";
				default_framerate = "10000000"; //! 10.0 fps 
				min_exp_time = "1000000"; //! us 
				max_exp_time = "1000000"; //! us 
				step_exp_time = "1";
				default_exp_time = "1"; //! us 

				embedded_metadata_height = "0";
			};
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
                status = "okay";
				port@0 {
					status = "okay";
					reg = <0>;
					bmg_ldr_out0: endpoint {
						status = "okay";
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&bmg_ldr_csi_in0>;
					};
				};
			};
		};
	};



	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";

		/**
		* Physical settings to calculate max ISO BW
		*
		* num_csi_lanes = <>;
		* Total number of CSI lanes when all cameras are active
		*
		* max_lane_speed = <>;
		* Max lane speed in Kbit/s
		*
		* min_bits_per_pixel = <>;
		* Min bits per pixel
		*
		* vi_peak_byte_per_pixel = <>;
		* Max byte per pixel for the VI ISO case
		*
		* vi_bw_margin_pct = <>;
		* Vi bandwidth margin in percentage
		*
		* max_pixel_rate = <>;
		* Max pixel rate in Kpixel/s for the ISP ISO case
		*
		* isp_peak_byte_per_pixel = <>;
		* Max byte per pixel for the ISP ISO case
		*
		* isp_bw_margin_pct = <>;
		* Isp bandwidth margin in percentage
		*/
		num_csi_lanes = <2>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <10>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <240000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		/**
		 * The general guideline for naming badge_info contains 3 parts, and is as follows,
		 * The first part is the camera_board_id for the module; if the module is in a FFD
		 * platform, then use the platform name for this part.
		 * The second part contains the position of the module, ex. “rear” or “front”.
		 * The third part contains the last 6 characters of a part number which is found
		 * in the module's specsheet from the vender.
		 */
		modules {
			status = "okay";
			module0 {
				status = "okay";
				badge = "bmg_ldr_0";
				position = "front";
				orientation = "1";
				drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
					devname = "bmg_ldr 0 2-003e";
					proc-device-tree = "/proc/device-tree/host1x/i2c@3180000/bmg_ldr_a@3e";
				};
				
			};
		};
	};
};

in harwdare/nvidia/platform/t18x/common/kernel-dts/t18x-common-platforms/tegra186-quill-camera-bmg-ldr6-a00.dtsi

/*
 * Copyright (c) 2015-2016, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
#include <t18x-common-modules/tegra186-camera-bmg-ldr6cam-a00.dtsi>
#include "dt-bindings/clock/tegra186-clock.h"

#define CAM0_RST_L     TEGRA_MAIN_GPIO(R, 5)
#define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

/* camera control gpio definitions */
/ {
	i2c@3180000 {
		bmg_ldr_a@3e {
			/*clocks = <&tegra_car TEGRA186_CLK_EXTPERIPH1>, */
			/*       <&tegra_car TEGRA186_CLK_PLLP_OUT0>; */
			/*clock-names = "extperiph1", "pllp_grtba";*/
			
			/*mclk = "extperiph1";*/
		};
	};
};

And in hardware/nvidia/platform/t18x/quill/kernel-dts/tegra186-quill-p3489-0888-a00-00-base.dts
The first lines for the includes are like this:

#include <t18x-common-platforms/tegra186-quill-common-p3489-1000-a00.dtsi>
#include <t18x-common-platforms/tegra186-quill-power-tree-p3489-1000-a00-00.dtsi>

//!#include <t18x-common-platforms/tegra186-quill-camera-modules.dtsi>

#include <t18x-common-platforms/tegra186-quill-camera-bmg-ldr6-a00.dtsi>

#include <t18x-common-modules/tegra186-display-e3320-1000-a00.dtsi>

/* comms dtsi file should be included after gpio dtsi file */
#include <t18x-common-plugin-manager/tegra186-quill-p3489-1000-a00-plugin-manager.dtsi>
#include <t18x-common-modules/tegra186-super-module-e2614-p2597-1000-a00.dtsi>
#include <t18x-common-plugin-manager/tegra186-quill-display-plugin-manager.dtsi>
#include <t18x-common-prod/tegra186-priv-quill-p3489-1000-a00-prod.dtsi>
//!#include <t18x-common-plugin-manager/tegra186-quill-camera-plugin-manager.dtsi>

Is there any difference in developing a driver for the jetson nano and the TX2 regarding the Kernel tree configuration?

Thank you

There’s no any different DTS configure for TX and Nano.
That’s all of the trace log? From the log NVCSI/VI didn’t receive any validate data from MIPI bus.

Hello ShaneCCC

I have talked with the hardware team and they find out a problem with the carrier, now it is solved, but I am able to receive frames only if I do the clock boost with:

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

If I try to read frames without the boost I get the following error:

[  111.492020] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  111.498633] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  111.510854] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERROR_STATUS2VI_VC0 = 0x0000000e
[  111.519795] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x0001000e
[  111.527858] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x0001000e
[  111.884018] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  111.890699] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  111.902821] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x0001000e
[  111.910828] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x0001000e
[  112.236112] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11

Why is this happening?

Thank you

Check the pix_clk_hz in the device tree to increase it. Or add serdes_pix_clk_hz in the device tree and give much bigger than value than pix_clk_hz.

Hello ShaneCCC

I have modified the value of pix_clk_hz using the formula

pixel_clk_hz = sensor data rate per lane (Mbps) * number of lanes / bits per pixel

The values that I had before were using the formula

pixel_clk_hz = sensor output size * frame rate

Now I am able to receive frames without the boost for debug, but I am seeing another problem, I have done a raw data dump with

v4l2-ctl -d 0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=3 --stream-to=out.raw

And looking at the data, I can see that at the end of each line I have 128 bytes of zeros, and as the buffer has an specific size in this case 14464*150 I am losing 19200 bytes wich is more than one line of data.

is there any reason why the TX2 has this strange behaviour with the data at the end of each line of the frame?

Thank you

Does Nano have the same symptom?

Hi ShaneCCC,
No, in the Nano the buffer was normal

Have a try the adjust the preferred_stride by v4l2-ctl to try.

v4l2-ctl --set-ctrl preferred_stride=xxxx

Hello ShaneCCC,

I dont have the preferred_stride control in the driver, so I am getting

unknown control 'preferred_stride'

Also now with v4l2-ctl set format I can see that the bytes per lane value is different from the width, you can see how the value changes for the different image formats in the image below:

So, why is the TX2 setting a different bytes per line size while the Jetson didnt?

Also i tried

v4l2-ctl -d 0 --set-fmt-video width=7264,height=200,pixelformat='BA81',bytesperline=7264

but it was setting the bytes per line as 7424 as you can see in the image.

I can try a workaround knowing how many bytes I have to skip at the end of each line but I would like to know why is this happening,

Thank you

Check this topic for the preferred_stride

Hello ShaneCCC

I have tried what you said, and made the modifications, but I see no difference, the bytes per line value is different from width, and I dont fully understand how the preferred_stride parameter works, what value should I set to the preferred_stride? the line lenght?

Also reading this post you say that VI has the 64-Byte align requirement, so as the VI is changing the resolution I will try to handle the buffers taking into account the bytes per line value to skip the zeros added at the end of each line.

Thank you,

I think you can check the source code to know how the bytes per line calculated.