hello TrevinN,
for camera mode running with the same resolution, it should have different pixel clocks due to different fps.
here’s one of calculate formula for your reference. please see-also Sensor Pixel Clock section.
i.e. pixel_clk_hz = sensor output size × frame rate
this should also works since the clock settings is higher to drive 30-fps sensor mode.
conversely, it’ll failed if you define 30-fps mode settings but launching 60-fps mode.
hello jerry,
ok.thanks for your answer.
So, is it correct to understand that the 4K60 mode can also be used for 4K30 as long as the pixel clock meets the requirement for 4K30 (greater than 38402160 30)?
that’s correct, and it should works for validation stage.
since it’s clock configuration, you may still use different mode definitions as your formal camera solution.
ok Returning to the issue of abnormal reception at 4K30, I have obtained more logs. The information is as follows:
orker/5:1-942 [005] … 620.156964: rtos_queue_peek_from_isr_failed: tstamp:19711209001 queue:0x0b4b4940
vi-7731 [003] … 641.746323: tegra_channel_open: vi-output, lt6911uxc 0-002b
vi-7731 [000] … 641.748104: tegra_channel_set_power: lt6911uxc 0-002b : 0x1
vi-7731 [000] … 641.748128: camera_common_s_power: status : 0x1
vi-7731 [000] … 641.748176: tegra_channel_set_power: 150c0000.nvcsi–1 : 0x1
vi-7731 [000] … 641.748180: csi_s_power: enable : 0x1
vi-7731 [000] … 641.852506: tegra_channel_capture_setup: vnc_id 0 W 3840 H 2160 fmt 41
vi-7731 [000] … 641.852633: tegra_channel_capture_setup: vnc_id 1 W 3840 H 2160 fmt 41
vi-output, lt69-7732 [003] … 641.852759: tegra_channel_set_stream: enable : 0x1
vi-output, lt69-7732 [000] … 641.855022: tegra_channel_set_stream: 150c0000.nvcsi–1 : 0x1
vi-output, lt69-7732 [000] … 641.855025: csi_s_stream: enable : 0x1
vi-output, lt69-7732 [000] … 641.855104: tegra_channel_set_stream: lt6911uxc 0-002b : 0x1
kworker/5:1-942 [005] … 641.900925: rtos_queue_peek_from_isr_failed: tstamp:20390804790 queue:0x0b4b4940
kworker/5:1-942 [005] … 641.900929: rtcpu_start: tstamp:20390806280
kworker/5:1-942 [005] … 641.900931: rtos_queue_send_from_isr_failed: tstamp:20390813249 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.900932: rtos_queue_send_from_isr_failed: tstamp:20390813357 queue:0x0b4ab1a8
kworker/5:1-942 [005] … 641.900933: rtos_queue_send_from_isr_failed: tstamp:20390813463 queue:0x0b4acdd8
kworker/5:1-942 [005] … 641.900934: rtos_queue_send_from_isr_failed: tstamp:20390813572 queue:0x0b4af718
kworker/5:1-942 [005] … 641.900934: rtos_queue_send_from_isr_failed: tstamp:20390813677 queue:0x0b4b04d8
kworker/5:1-942 [005] … 641.900935: rtos_queue_send_from_isr_failed: tstamp:20390813782 queue:0x0b4b1298
kworker/5:1-942 [005] … 641.900936: rtos_queue_send_from_isr_failed: tstamp:20390813889 queue:0x0b4b2058
kworker/5:1-942 [005] … 641.900937: rtos_queue_send_failed: tstamp:20390814337 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.900938: rtos_queue_send_from_isr_failed: tstamp:20390816302 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.900939: rtos_queue_send_from_isr_failed: tstamp:20390816408 queue:0x0b4ab1a8
kworker/5:1-942 [005] … 641.900940: rtos_queue_send_from_isr_failed: tstamp:20390816514 queue:0x0b4acdd8
kworker/5:1-942 [005] … 641.900940: rtos_queue_send_from_isr_failed: tstamp:20390816623 queue:0x0b4af718
kworker/5:1-942 [005] … 641.900941: rtos_queue_send_from_isr_failed: tstamp:20390816727 queue:0x0b4b04d8
kworker/5:1-942 [005] … 641.900942: rtos_queue_send_from_isr_failed: tstamp:20390816831 queue:0x0b4b1298
kworker/5:1-942 [005] … 641.900943: rtos_queue_send_from_isr_failed: tstamp:20390816935 queue:0x0b4b2058
kworker/5:1-942 [005] … 641.900943: rtos_queue_send_failed: tstamp:20390817851 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.900944: rtos_queue_send_from_isr_failed: tstamp:20390820355 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.900945: rtos_queue_send_from_isr_failed: tstamp:20390820462 queue:0x0b4ab1a8
kworker/5:1-942 [005] … 641.900946: rtos_queue_send_from_isr_failed: tstamp:20390820567 queue:0x0b4acdd8
kworker/5:1-942 [005] … 641.900946: rtos_queue_send_from_isr_failed: tstamp:20390820674 queue:0x0b4af718
kworker/5:1-942 [005] … 641.900947: rtos_queue_send_from_isr_failed: tstamp:20390820778 queue:0x0b4b04d8
kworker/5:1-942 [005] … 641.900948: rtos_queue_send_from_isr_failed: tstamp:20390820883 queue:0x0b4b1298
kworker/5:1-942 [005] … 641.900949: rtos_queue_send_from_isr_failed: tstamp:20390820996 queue:0x0b4b2058
kworker/5:1-942 [005] … 641.900949: rtos_queue_send_failed: tstamp:20390821909 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.900951: rtcpu_vinotify_event: tstamp:20390899742 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:20390899358 data:0x00000001
kworker/5:1-942 [005] … 641.900952: rtcpu_vinotify_event: tstamp:20390900655 tag:CSIMUX_STREAM channel:0xff frame:0 vi_tstamp:20390900284 data:0x00000100
kworker/5:1-942 [005] … 641.900953: rtcpu_vinotify_event: tstamp:20391775947 tag:CHANSEL_PXL_SOF channel:0x00 frame:16192 vi_tstamp:20391774821 data:0x00000001
kworker/5:1-942 [005] … 641.900953: rtcpu_vinotify_event: tstamp:20391776122 tag:CHANSEL_PXL_SOF channel:0x01 frame:16192 vi_tstamp:20391774821 data:0x00000001
kworker/5:1-942 [005] … 641.900954: rtcpu_vinotify_event: tstamp:20391776287 tag:ATOMP_FS channel:0x00 frame:16192 vi_tstamp:20391774824 data:0x00000000
kworker/5:1-942 [005] … 641.900954: rtcpu_vinotify_event: tstamp:20391776400 tag:ATOMP_FS channel:0x01 frame:16192 vi_tstamp:20391774824 data:0x00000000
kworker/5:1-942 [005] … 641.900955: rtcpu_vinotify_event: tstamp:20391776514 tag:CHANSEL_FAULT channel:0x00 frame:16192 vi_tstamp:20391775015 data:0x00000100
kworker/5:1-942 [005] … 641.900956: rtcpu_vinotify_event: tstamp:20391777274 tag:CHANSEL_FAULT channel:0x01 frame:16192 vi_tstamp:20391775015 data:0x00000100
kworker/5:1-942 [005] … 641.900956: rtcpu_vinotify_event: tstamp:20391778032 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:16192 vi_tstamp:20391776653 data:0x08000000
kworker/5:1-942 [005] … 641.900957: rtcpu_vinotify_event: tstamp:20391778141 tag:CHANSEL_FAULT_FE channel:0x01 frame:16192 vi_tstamp:20391776673 data:0x00000001
kworker/5:1-942 [005] … 641.900957: rtcpu_vinotify_event: tstamp:20391778281 tag:ATOMP_FE channel:0x00 frame:16192 vi_tstamp:20391776675 data:0x00000000
kworker/5:1-942 [005] … 641.900958: rtcpu_vinotify_event: tstamp:20391778534 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:16192 vi_tstamp:20391777414 data:0x18000000
kworker/5:1-942 [005] … 641.900958: rtcpu_vinotify_event: tstamp:20391778642 tag:CHANSEL_FAULT_FE channel:0x04 frame:16192 vi_tstamp:20391777414 data:0x00000002
kworker/5:1-942 [005] … 641.900959: rtcpu_vinotify_event: tstamp:20391778781 tag:ATOMP_FE channel:0x01 frame:16192 vi_tstamp:20391777417 data:0x00000000
kworker/4:0-36 [004] … 641.912194: tegra_channel_notify_status_callback:
kworker/5:1-942 [005] … 641.956927: rtos_queue_send_from_isr_failed: tstamp:20392685707 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.956930: rtos_queue_send_from_isr_failed: tstamp:20392685818 queue:0x0b4ab1a8
kworker/5:1-942 [005] … 641.956931: rtos_queue_send_from_isr_failed: tstamp:20392685929 queue:0x0b4acdd8
kworker/5:1-942 [005] … 641.956932: rtos_queue_send_from_isr_failed: tstamp:20392686036 queue:0x0b4af718
kworker/5:1-942 [005] … 641.956933: rtos_queue_send_from_isr_failed: tstamp:20392686141 queue:0x0b4b04d8
kworker/5:1-942 [005] … 641.956933: rtos_queue_send_from_isr_failed: tstamp:20392686248 queue:0x0b4b1298
kworker/5:1-942 [005] … 641.956934: rtos_queue_send_from_isr_failed: tstamp:20392686354 queue:0x0b4b2058
kworker/5:1-942 [005] … 641.956936: rtos_queue_send_failed: tstamp:20392686900 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.956937: rtos_queue_send_from_isr_failed: tstamp:20392689042 queue:0x0b4a7698
kworker/5:1-942 [005] … 641.956937: rtos_queue_send_from_isr_failed: tstamp:20392689148 queue:0x0b4ab1a8
kworker/5:1-942 [005] … 641.956938: rtos_queue_send_from_isr_failed: tstamp:20392689255 queue:0x0b4acdd8
kworker/5:1-942 [005] … 641.956939: rtos_queue_send_from_isr_failed: tstamp:20392689362 queue:0x0b4af718
kworker/5:1-942 [005] … 641.956940: rtos_queue_send_from_isr_failed: tstamp:20392689467 queue:0x0b4b04d8
kworker/5:1-942 [005] … 641.956941: rtos_queue_send_from_isr_failed: tstamp:20392689572 queue:0x0b4b1298
kworker/5:1-942 [005] … 641.956941: rtos_queue_send_from_isr_failed: tstamp:20392689676 queue:0x0b4b2058
kworker/5:1-942 [005] … 641.956942: rtos_queue_send_failed: tstamp:20392690214 queue:0x0b4a7698
kworker/5:1-942 [005] … 642.012940: rtos_queue_peek_from_isr_failed: tstamp:20395805122 queue:0x0b4b4940
kworker/5:1-942 [005] … 642.180918: rtos_queue_peek_from_isr_failed: tstamp:20400805110 queue:0x0b4b4940
kworker/5:1-942 [005] … 642.348933: rtos_queue_peek_from_isr_failed: tstamp:20405805101 queue:0x0b4b4940
kworker/5:1-942 [005] … 642.516907: rtos_queue_peek_from_isr_failed: tstamp:20410805094 queue:0x0b4b4940
kworker/5:1-942 [005] … 642.684976: rtos_queue_peek_from_isr_failed: tstamp:20415805088 queue:0x0b4b4940
kworker/5:1-942 [005] … 642.852983: rtos_queue_peek_from_isr_failed: tstamp:20420805083 queue:0x0b4b4940
kworker/5:1-942 [005] … 643.024978: rtos_queue_peek_from_isr_failed: tstamp:20425805077 queue:0x0b4b4940
kworker/5:1-942 [005] … 643.140935: rtos_queue_peek_from_isr_failed: tstamp:20430805067 queue:0x0b4b4940
my dts config ：
mode0 { // E2832_1920x1080_60Fps
mclk_khz = “12000”;
num_lanes = “4”;
tegra_sinterface = “serial_a”;
phy_mode = “DPHY”;
discontinuous_clk = “yes”;
dpcm_enable = “false”;
cil_settletime = “0”;
active_w = "1920";
active_h = "1080";
mode_type = "rgb";
pixel_phase = "rgb888";
csi_pixel_bit_depth = "24";
readout_orientation = "0";
line_length = "1920";
inherent_gain = "1";
mclk_multiplier = "1";
pix_clk_hz = "1";
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "16"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "30"; /* 60.0 fps */
step_framerate = "1";
default_framerate = "20000000"; /* 60.0 fps */
min_exp_time = "13"; /* us */
max_exp_time = "300000"; /* us */
step_exp_time = "1";
default_exp_time = "666666"; /* us */
};
mode1 { // E2832_3840x2160
mclk_khz = "24000";
num_lanes = "2";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "yes";
dpcm_enable = "false";
cil_settletime = "0";
active_w = "3840";
active_h = "2160";
mode_type = "rgb";
pixel_phase = "rgb888";
csi_pixel_bit_depth = "24";
readout_orientation = "0";
line_length = "3840";
inherent_gain = "1";
mclk_multiplier = "24";
pix_clk_hz = "273715200";
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "16"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "30000000"; /* 60.0 fps */
step_framerate = "1";
default_framerate = "30000000"; /* 60.0 fps */
min_exp_time = "30"; /* us */
max_exp_time = "683709"; /* us */
step_exp_time = "1";
default_exp_time = "33334"; /* us */
};
mode2 { // E2832_1280x720_60Fps
mclk_khz = "24000";
num_lanes = "4";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "yes";
dpcm_enable = "false";
cil_settletime = "0";
active_w = "1280";
active_h = "720";
mode_type = "rgb";
pixel_phase = "rgb888";
csi_pixel_bit_depth = "24";
readout_orientation = "0";
line_length = "1280";
inherent_gain = "1";
mclk_multiplier = "24";
pix_clk_hz = "576000000";
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "16"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "60000000"; /* 60.0 fps */
step_framerate = "1";
default_framerate = "60000000"; /* 60.0 fps */
min_exp_time = "13"; /* us */
max_exp_time = "683709"; /* us */
step_exp_time = "1";
default_exp_time = "16667"; /* us */
};
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
e2832_out0: endpoint {
port-index = <0>;
bus-width = <2>;
remote-endpoint = <&e2832_csi_in0>;
};
};
};
};
};
};
hello TrevinN,
here looks incorrect,
you’ve obtain two SOF in the beginning of stream.
furthermore, what’s the lane configure you’re used? 4-lane or 2-lane.
the device tree settings, such as num_lanes and bus-width should be the same.
“According to the diagram, the SOC is connected to the sensor via 8 lanes. When transmitting at 4K60 using 8 lanes, should the configuration be num_lanes = “8” and bus-width = <8>? However, whether the sensor needs to use 8 lanes to transmit data for other lower resolutions such as 4K30 and 1080P60 . For example, in one modex with a resolution of 1080P, only 4 lanes are used for transmission. Can this configuration, with num_lanes = “4” and bus-width = <8>, cause any anomalies?”
HI jerry,
To add more information, the sensor is using MIPI to transmit 4K60 RGB888 and it uses two 4-lane gang modes. The sensor splits into two 4-lane output sources. In this case, how should I configure the DTS mode with num-lanes and bus-width? Are there any considerations for DTS configuration in this mode? Also, when transmitting at lower resolutions such as 4K30 or 1080p, the sensor only uses 4 lanes to transmit data, while the other group of 1 clk+4 lanes is used in copy mode to output the same data. Should I disable the other 4 lanes output?
hello TrevinN,
for your use-case, you should have device tree settings to configure as 8-lane settings.
once device tree configure as 8-lane, two CSI bricks (assume it’s CSI-A/B + CSI-C/D) will be used to capture buffers. but… there’s logic (in VI driver) to fall back as single CSI brick (only 4-lane being used) since it’s 4K30 image capture. that’s tricky by setting num-lanes=<4> in device tree, (assume it’s using CSI-C/D). it may works as long as there’s correct MIPI signaling on CSI-C/D.
hence,
you should have the unused 4-lanes keep connecting, disable copy mode and only output 4-lane signals to CSI brick.
hi jerry,
“Okay, I’ll go turn off copymode for testing. Also, how can I differentiate between 4K30 and 4K60 in the modeX list while in this mode, as 4K30 uses ‘num-lanes=<4>’ and 4K60 uses ‘num-lanes=<8>’? When operating on the /dev/video0 node, my understanding is that VIDIOC_S_FMT can be used to switch to the desired resolution using W and H parameters, but there are no request parameters for frame rate, and since different frame rates may require a different num-lanes, how do I configure the modex accurately for the same resolution but different frame rates? Or can I use ‘num-lanes=<8>’ for 4K30 even if it uses a 4-lane transmission?”
please refer to developer guide, Device Properties.
you may toggle use_sensor_mode_id=1 and uses the TEGRA_CAMERA_CID_SENSOR_MODE_ID control to select a specific sensor mode.
hi，jerry
“Do I need to add TEGRA_CAMERA_CID_SENSOR_MODE_ID to the ctrl_cid_list in the sensor driver structure?
static struct tegracam_ctrl_ops imx219_ctrl_ops = {
.numctrls = ARRAY_SIZE(ctrl_cid_list),
.ctrl_cid_list = ctrl_cid_list,
.set_gain = imx219_set_gain,
.set_exposure = imx219_set_exposure,
.set_frame_rate = imx219_set_frame_rate,
.set_group_hold = imx219_set_group_hold,
};
Also, does ID=0 refer to mode0 in the DTS modeX or is it related to the array index of the static const struct camera_common_frmfmt lt6911uxc_frmfmt?”
{{1920, 1080}, lt6911uxc_60fps, 1, 0, 0},
{{3840, 2160}, lt6911uxc_60fps, 1, 0, 1},
{{1920, 1080}, lt6911uxc_60fps, 1, 0, 2},
hello TrevinN,
yes, you should have TEGRA_CAMERA_CID_SENSOR_MODE_ID added in the driver layer.
please also check IMX185 sensor drivers for reference,
static const u32 ctrl_cid_list[] = {
...
TEGRA_CAMERA_CID_SENSOR_MODE_ID,
};
besides,
by checking reference driver, it uses enum{} to define sensor modes, which is start from 0.
for example,
enum {
IMX185_MODE_1920X1080_CROP_30FPS,
IMX185_MODE_1920X1080_CROP_10BIT_30FPS,
IMX185_MODE_1920X1080_CROP_60FPS,
...
hi jerry,
For 4K30, the MIPI CLK output from the LT6911 front-end chip to NVIDIA is calculated as MipiClock = 4 * Byteclk in Byteclk, where PixelBytes is the number of bytes used for each pixel, for example, RGB888/YUV44 will have PixelBytes = 3, and RGB565/YUV422 will have PixelBytes = 2. For a 4K30Hz HDMI input and single-port 4-lane CSI YUV422 output, for example, Pixclk = 297MHz, and Byteclk = (297*2/4)MHz+5MHz. The Byteclk value is generally added with 5~10 to the theoretical value for a horizontal screen and 10~20 for a vertical screen. The theoretical value added is 5~10M, and it may affect the CSI receiving data of NVIDIA?
you may refer to developer guide, Sensor Pixel Clock section for several ways to obtain the correct sensor pixel clock rate.
HI jerry,
I am currently configured to receive sensor RGB24 data with 4 lanes and have tested the following resolutions:
3840x1080@30 received correctly
3840x1200@30 received abnormally
3840x1184@30 received abnormally
1920x2160@30 received correctly
1920x2160@60 received correctly
3840x2160@30 received abnormally
I found that when W = 3840, if H > 1080, it seems to time out the reception. Is this related to the sensor MIPI output or the NVIDIA CSI configuration?
Here is the DTS configuration. Different resolutions use mode1 with only active_w/h changed, while everything else remains the same.
mode1 { // E2832_3840x2160
mclk_khz = “24000”;
num_lanes = “4”;
tegra_sinterface = “serial_a”;
phy_mode = “DPHY”;
discontinuous_clk = “yes”;
dpcm_enable = “false”;
cil_settletime = “0”;
active_w = "3808 ";
active_h = “1184”;
mode_type = “rgb”;
pixel_phase = “rgb888”;
csi_pixel_bit_depth = “24”;
readout_orientation = “0”;
line_length = “3840”;
inherent_gain = “1”;
mclk_multiplier = “24”;
pix_clk_hz = “576000000”;
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "16"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "60000000"; /* 60.0 fps */
step_framerate = "1";
default_framerate = "60000000"; /* 60.0 fps */
min_exp_time = "30"; /* us */
max_exp_time = "683709"; /* us */
step_exp_time = "1";
default_exp_time = "16667"; /* us */
};
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
e2832_out0: endpoint {
port-index = <0>;
bus-width = <4>;
remote-endpoint = <&e2832_csi_in0>;
};
};
Hi jerry，
Adding to that, even with the DTS configuration, when using 3808x1184@30, it still times out during reception.
DTS line_length = “3840”; this fix to 3808 ,the problem still persists
hello TrevinN,
quick question.
are those three sensor mode of W=3840 using the same pixel clock, pix_clk_hz ?
The variation in pixel clock difference is due to the difference in H value multiplied by 30.
HI. jerry
Would the pixel clock difference affect CSI reception? I always restart the program for testing. In addition, the reception of 3840x540*30 is also normal.