Hi,
I am working on porting camera driver on nvidia jetson nano platform.
MIPI clock frequency for camera sensor is 135 MHz.
sensor characteristics are as follows:-
Maximum data rate :- 540 Mbps
no.of lanes :- 2
input clock - 45 MHz
Could anyone help me to set these parameters regarding mclk in dtsi file…?
Below is my dtsi file for camera sensor connected over i2c bus-6 :-
tegra210-camera-rbpcv2-cam_sensor.dtsi
i2c@546c0000 {
rbpcv2_cam_sensor_a@64 {
compatible = "nvidia,cam_sensor";
/* I2C device address */
reg = <0x64>;
/* V4L2 device node location */
devnode = "video0";
/* Physical dimensions of sensor */
physical_w = "3.680";
physical_h = "2.760";
sensor_model = "cam_sensor";
use_sensor_mode_id = "true";
/**
* ==== Modes ====
* A modeX node is required to support v4l2 driver
* implementation with NVIDIA camera software stack
*
* == Signal properties ==
*
* phy_mode = "";
* PHY mode used by the MIPI lanes for this device
*
* tegra_sinterface = "";
* CSI Serial interface connected to tegra
* Incase of virtual HW devices, use virtual
* For SW emulated devices, use host
*
* pix_clk_hz = "";
* Sensor pixel clock used for calculations like exposure and framerate
*
* readout_orientation = "0";
* Based on camera module orientation.
* Only change readout_orientation if you specifically
* Program a different readout order for this mode
*
* == Image format Properties ==
*
* active_w = "";
* Pixel active region width
*
* active_h = "";
* Pixel active region height
*
* pixel_t = "";
* The sensor readout pixel pattern
*
* line_length = "";
* Pixel line length (width) for sensor mode.
*
* == Source Control Settings ==
*
* Gain factor used to convert fixed point integer to float
* Gain range [min_gain/gain_factor, max_gain/gain_factor]
* Gain step [step_gain/gain_factor is the smallest step that can be configured]
* Default gain [Default gain to be initialized for the control.
* use min_gain_val as default for optimal results]
* Framerate factor used to convert fixed point integer to float
* Framerate range [min_framerate/framerate_factor, max_framerate/framerate_factor]
* Framerate step [step_framerate/framerate_factor is the smallest step that can be configured]
* Default Framerate [Default framerate to be initialized for the control.
* use max_framerate to get required performance]
* Exposure factor used to convert fixed point integer to float
* For convenience use 1 sec = 1000000us as conversion factor
* Exposure range [min_exp_time/exposure_factor, max_exp_time/exposure_factor]
* Exposure step [step_exp_time/exposure_factor is the smallest step that can be configured]
* Default Exposure Time [Default exposure to be initialized for the control.
* Set default exposure based on the default_framerate for optimal exposure settings]
*
* gain_factor = ""; (integer factor used for floating to fixed point conversion)
* min_gain_val = ""; (ceil to integer)
* max_gain_val = ""; (ceil to integer)
* step_gain_val = ""; (ceil to integer)
* default_gain = ""; (ceil to integer)
* Gain limits for mode
*
* exposure_factor = ""; (integer factor used for floating to fixed point conversion)
* min_exp_time = ""; (ceil to integer)
* max_exp_time = ""; (ceil to integer)
* step_exp_time = ""; (ceil to integer)
* default_exp_time = ""; (ceil to integer)
* Exposure Time limits for mode (sec)
*
* framerate_factor = ""; (integer factor used for floating to fixed point conversion)
* min_framerate = ""; (ceil to integer)
* max_framerate = ""; (ceil to integer)
* step_framerate = ""; (ceil to integer)
* default_framerate = ""; (ceil to integer)
* Framerate limits for mode (fps)
*
* embedded_metadata_height = "";
* Sensor embedded metadata height in units of rows.
* If sensor does not support embedded metadata value should be 0.
*/
mode0 { /* AD903X_MODE_640x960_30FPS */
mclk_khz = "24000";
num_lanes = "2";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "no";
dpcm_enable = "false";
cil_settletime = "0";
active_w = "640";
active_h = "960";
pixel_t = "bayer_rggb12";
readout_orientation = "0";
line_length = "640";
inherent_gain = "1";
mclk_multiplier = "1.87";
pix_clk_hz = "45000000";
gain_factor = "1";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "1"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "30000000"; /* 30.0 fps */
step_framerate = "1";
default_framerate = "30000000"; /* 30.0 fps */
min_exp_time = "13"; /* us */
max_exp_time = "683709"; /* us */
step_exp_time = "1";
default_exp_time = "2495"; /* us */
embedded_metadata_height = "0";
};
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
rbpcv2_cam_sensor_out0: endpoint {
port-index = <0>;
bus-width = <2>;
remote-endpoint = <&rbpcv2_cam_sensor_csi_in0>;
};
};
};
};
};
};
};
/ {
tegra-camera-platform {
compatible = “nvidia, tegra-camera-platform”;
/**
* Physical settings to calculate max ISO BW
*
* num_csi_lanes = <>;
* Total number of CSI lanes when all cameras are active
*
* max_lane_speed = <>;
* Max lane speed in Kbit/s
*
* min_bits_per_pixel = <>;
* Min bits per pixel
*
* vi_peak_byte_per_pixel = <>;
* Max byte per pixel for the VI ISO case
*
* vi_bw_margin_pct = <>;
* Vi bandwidth margin in percentage
*
* max_pixel_rate = <>;
* Max pixel rate in Kpixel/s for the ISP ISO case
*
* isp_peak_byte_per_pixel = <>;
* Max byte per pixel for the ISP ISO case
*
* isp_bw_margin_pct = <>;
* Isp bandwidth margin in percentage
*/
num_csi_lanes = <2>;
max_lane_speed = <1500000>;
min_bits_per_pixel = <12>;
vi_peak_byte_per_pixel = <2>;
vi_bw_margin_pct = <25>;
max_pixel_rate = <240000>;
isp_peak_byte_per_pixel = <5>;
isp_bw_margin_pct = <25>;
/**
* The general guideline for naming badge_info contains 3 parts, and is as follows,
* The first part is the camera_board_id for the module; if the module is in a FFD
* platform, then use the platform name for this part.
* The second part contains the position of the module, ex. "rear" or "front".
* The third part contains the last 6 characters of a part number which is found
* in the module's specsheet from the vendor.
*/
modules {
module0 {
badge = "porg_front_RBPCV2";
position = "rear";
orientation = "1";
drivernode0 {
pcl_id = "v4l2_sensor";
devname = "cam_sensor 6-0064";
proc-device-tree = "/proc/device-tree/host1x/i2c@546c0000/rbpcv2_cam_sensor_a@64";
};
};
};
};
};
tegra210-porg-camera-rbpcv2-cam_sensor.dtsi:-
#include “tegra210-camera-rbpcv2-cam_sensor.dtsi”
#define CAM1_PWDN TEGRA_GPIO(S, 7)
/ {
host1x {
i2c@546c0000 {
rbpcv2_cam_sensor_a@10 {
status = “okay”;
reset-gpios = <&gpio CAM1_PWDN GPIO_ACTIVE_HIGH>;
};
};
};
gpio@6000d000 {
camera-control-output-low {
gpio-hog;
output-low;
gpios = < CAM1_PWDN 0 >;
label = "cam1-pwdn";
};
};
};
hello nikhil.karale,
please refer to Sensor Driver Programming Guide, and check the [Sensor Pixel Clock] chapter for the calculation formulas.
thanks
Hi,
I have calculated mclk parameters as below using formulas given :
mclk = 24 Mhz
pixel_clk_hz = sensor data rate per lane * no of lanes / bits per pixel
= 270 Mbps * 2 / 12
= 45 MHz
mclk_multiplier = pixel_clk_hz / mclk
= 45 MHz / 24 MHz
= 1.875
Is this calculation correct…?
hello nikhil.karale,
please configure pixel_clk_hz property according to your calculation result.
also, since mclk_multiplier property has already deprecated, kernel driver will not consider this setting.
thanks
As per Sensor Driver Programming Guide, default value of mclk is 24 MHz , as input clock for my camera sensor is 45 MHz … how to set mclk as 45 MHz…?
Could you provide values for below parameters:-
clocks
clock-names
clock-frequency
mclk
Currently i have set below values, but its not working…
clocks = <&tegra_car TEGRA210_CLK_CLK_OUT_3>;
clock-names = “extperiph1”;
clock-frequency = <24000000>;
mclk = “extperiph1”;
hello nikhil.karale,
may I know which camera sensor you’re working with.
you’ll need to use an external clock source if you would like to configure mclk larger than 24MHz.
thanks
Hello JerryChang,
Thanks for the reply.
After discussion with hardware team, I found that camera sensor is connected with analog front end (AFE) and AFE is connected with processor. AFE also has 45 MHz clock given externally. So basically AFE is driving camera sensor attached to it (providing clock to sensor , configuring sensor & giving MIPI-CSI-2 output).
I can only program the AFE using I2C bus which seems to be working fine. So in this case what should be the the value of mclk and pixel_clk…?
Hi,
I tried device tree settings as per sensor programming guide,but its not working.
Below diagram shows how camera sensor is interfaced with jetson nano board:-
[url]https://drive.google.com/file/d/15QO3-tTdKjvcGrskKmM1RQWMdWG3_4ES/view[/url]
As camera sensor is not directly interfaced with jetson nano, sensor configuraton is taken care by ADC and also mclk (input clock for sensor) is given externally which drives the sensor.
Currently we are getting video frames containing zero data (all zeros) from camera.
We also observed MIPI data lines and MIPI output clock from the sensor.
I think there is clock mismatch issue as camera is giving output on MIPI lines.
Does it require any other configuration in the the tegra framework…?
hello nikhil.karale,
please check if there’s suspicious kernel failures, you may also refer to Approaches for Validating and Testing the V4L2 Driver session to have commands to access camera sensor.
FYI,
there’s supported camera list of Jetson-Nano, please check Raspberry camera module V1.3 compatibility discussion thread.
also known NOT support cameras, please refer to Topic 1050743
Hi JerryChang,
Thanks for the reply.
I tested the camera driver using v4l2-ctl command ,i got following output on terminal-
terminal log -
nikhil@nikhil-desktop:~$ v4l2-ctl -d /dev/video0 -w --verbose --set-fmt-video=width=640,height=960,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=3 --stream-to=data.raw
Opening in BLOCKING MODE
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
Width/Height : 640/960
Pixel Format : 'RG12'
Field : None
Bytes per Line : 1280
Size Image : 1228800
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Full Range)
Flags :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
Index : 0
Type : Video Capture
Flags : mapped
Field : None
Sequence : 0
Length : 1228800
Bytesused: 1228800
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 1
Type : Video Capture
Flags : mapped
Field : None
Sequence : 1
Length : 1228800
Bytesused: 1228800
Timestamp: 0.000000s (Monotonic, End-of-Frame)
Index : 2
Type : Video Capture
Flags : mapped
Field : None
Sequence : 2
Length : 1228800
Bytesused: 1228800
Timestamp: 0.000000s (Monotonic, End-of-Frame)
VIDIOC_STREAMOFF: ok
kernel dmesg log for the v4l2-ctl command:-
[14402.155220] camxxx 6-0064: camera_common_mclk_enable: enable MCLK with 135000000 Hz
[14402.162951] camxxx 6-0064:
Set mclk
[14402.167989] camxxx 6-0064:
Set mclk err = 0
[14402.173820]
Program AFE
[14402.176363] camxxx_programAfe called
[14402.179999] camxxx 6-0064: camxxx_programAfe
[14402.941401] camxxx_programAfe completed
[14402.945295]
s_stream 1
[14402.947714] vi 54080000.vi: Calibrate csi port 0
[14402.952485] camxxx 6-0064: s_power 1
[14402.956090] camxxx 6-0064: camxxx_set_power_on
[14402.964005] camxxx 6-0064: power on complete
[14402.968296] camxxx 6-0064: s_power 0
[14402.971886] camxxx 6-0064: camxxx_set_power_off
[14402.979207] camxxx 6-0064: power off complete
[14402.983592]
power down succeed
[14402.986637]
s_stream 0
[14402.990961] camxxx 6-0064: camera_common_mclk_enable: enable MCLK with 135000000 Hz
[14402.998846] camxxx 6-0064:
Set mclk
[14403.003833] camxxx 6-0064:
Set mclk err = 0
[14403.063371]
camxxx set_format 0 0
[14403.068199]
call camera_common_try_fmt
[14403.073485] camxxx 6-0064: camera_common_try_fmt: size 640 x 960
[14403.079551]
camxxx set_format 0 0
[14403.085035]
call camera_common_try_fmt
[14403.090380] camxxx 6-0064: camera_common_try_fmt: size 640 x 960
[14403.096466]
camxxx set_format 1 0
[14403.101344]
call camera_common_s_fmt
[14403.106468] camxxx 6-0064: camera_common_s_fmt(12306) size 640 x 960
[14403.112880]
camera_common_s_fmt 12306 640 960
[14403.118826] camxxx 6-0064: camera_common_try_fmt: size 640 x 960
[14403.133617] vi 54080000.vi: cil_settingtime is pulled from device
[14403.139827] vi 54080000.vi: cil_settingtime was autocalculated
[14403.145698] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[14403.152695]
Program AFE
[14403.155142] camxxx_programAfe called
[14403.158763] camxxx 6-0064: camxxx_programAfe
[14403.823206] camxxx_programAfe completed
[14403.827089]
s_stream 1
[14404.002373] video4linux video0: Syncpoint already enabled at capture done!0
[14404.212106] video4linux video0: tegra_channel_capture_done: MW_ACK_DONE syncpoint time out!0
[14404.221042] video4linux video0: TEGRA_VI_CSI_ERROR_STATUS 0x00000004
[14404.228122] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000180
[14404.234835] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
[14404.241116] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
[14404.247003] vi 54080000.vi: cil_settingtime is pulled from device
[14404.253134] vi 54080000.vi: cil_settingtime was autocalculated
[14404.259082] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[14404.266002] camxxx 6-0064: s_power 1
[14404.269591] camxxx 6-0064: camxxx_set_power_on
[14404.277639] camxxx 6-0064: power on complete
[14404.281938] camxxx 6-0064: s_power 0
[14404.285535] camxxx 6-0064: camxxx_set_power_off
[14404.292935] camxxx 6-0064: power off complete
[14404.297314]
power down succeed
[14404.300409]
s_stream 0
hello nikhil.karale,
it seems you got syncpt timeout even your sensor is outputting streaming
[14403.827089]
s_stream 1
[14404.002373] video4linux video0: Syncpoint already enabled at capture done!0
[14404.212106] video4linux video0: tegra_channel_capture_done: MW_ACK_DONE syncpoint time out!0
are you able to arrange hardware engineer to probe the sensor signaling?
thanks
Hello JerryChang,
I probed the MIPI signals, and observed that sensor is driving MIPI clock (135 MHz) and sending data over MIPI data lines.
Is there any way to get rid of the above error. I am not getting why this error is coming if sensor is throwing data and clock over MIPI bus.
Hi JerryChang,
I tried the IMX219 camera sensor (Rasberry Pi camera V2) with jetson nano which works with it.
I probed the MIPI clock signal for IMX219 and observed that there is a sync pulse after every frame on MIPI clock line.
You can check the MIPI clock output for IMX219 sensor (i ran gstreamer with frame rate of 60, width=1280,height = 720) using below link:-
[url]https://drive.google.com/open?id=1HGAR1kr5OFWnX2LqXwsW8MScODWoHKZg[/url]
In case of my custom camera sensor, i am getting below MIPI clock output:-
[url]https://drive.google.com/open?id=1ZZJbsHkKJNNK9oLUVNUaLcAUQSvhhCqm[/url]
Does tegra camera framework require sync pulse on MIPI clock line driven by the sensor…?
hello nikhil.karale,
please note that Tegra camera serial interface is based-on the MIPI CSI-2.0 standard specification.
you should follow that standard for your sensor configuration.
thanks
Hello JerryChang,
Camera sensor is MIPI CSI-2 standard compliant as it works with another (other than NVIDIA) hardware platforms. The sensor data is output through the MIPI CSI-2 Tx interface only.
hello nikhil.karale,
I’ve consult with our hardware engineer to check your signaling,
please modify your custom camera sensor to add sync pulse on MIPI clock.
thanks
Hello JerryChang,
We can not modify camera sensor to add sync pulse on MIPI clock as i dont know much about its design.
Is it possible to modify tegra camera framework so that it will not consider the sync pulse on MIPI clock…? and fill the buffer with data available on MIPI bus.
Is there any way to do so…?
Hello JerryChang,
I went through Sensor programming guide and found below image which shows direct v4l2 interface to camera
[url]https://drive.google.com/open?id=1TWYS-tW2kloyeo8pwrCqfbmmrETmjIQC[/url]
Also found there is “TEGRA_CAMERA_CID_VI_BYPASS_MODE” control handler which can be used to bypass VI settings in kernel driver.
Does this control handler can be used to disable the VI block in framework…?
Could i disable the VI block and access the camera using direct v4l2 interface as shown in figure above…?