I have been able to capture successfully at 4 lanes from CD and with overlay on HDMI.
My big mistake was the FPS mismatch between device tree (sensor modes) and device driver mode tables.
Actually I set the CSI for 594 Mbps x 4 lanes with expected FPS 25, so in the device tree sensor modes I wrote a maximum frame rate of 25 fps for the mode0 full-frame.
Unfortunately in the device driver mode tables I leaved the nominal frame rate of 60p, instead of adding 25p to the list of suitable frameintervals for such mode.
congrats! i know the feeling of relief seeing it work for first time :p
Congrats, tessadriandrea,
I’m also developing cmos sensor driver(imx347) on Jetson Nano, and I can get the stream with v4l2-ctl, but not working with nvarguscamera/argus_camera, which report the error TEGRA_VI_CSI_ERROR_STATUS 0x00000001, I noticed that you have also the active_w/active_h information in device tree, could you kindly share how you calculated it?
By following the offical L4T documents below I’m getting errors, Thanks ahead.
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fcamera_sensor_prog.html%23wwpID0E0KCB0HA
Hi Yawei Yang,
active_w and active_h comes from the sensor datasheet and are the width and height of the visible part of the sensor, so litterally the real pixels.
Please pay attention to have sensor modes on device tree properly matched with frmfmt array elements in the .h of your driver, otherwise gstreamer will get confused.
If you are able to capture successfully with v4l2-ctl that means your video pipeline sensor → CSI → VI is correct.
It can be an issue related to frame rates or frame formats.
Have you followed the guide for a driver version1 or version2 ?
Check that your v4l2_subdev_pad_ops contains pointers to camera_common functions:
static struct v4l2_subdev_pad_ops imx347_subdev_pad_ops = {
...
.enum_mbus_code = camera_common_enum_mbus_code,
.enum_frame_size = camera_common_enum_framesizes,
.enum_frame_interval = camera_common_enum_frameintervals,
};
If you don’t use camera_common functions your device tree settings won’t become visible to tegra_channel, so arguscamera won’t know how your sensor is configured.
Thanks tessadriandrea,
For the quick reply, I’m following version2 in the guide for driver development, where the camera common control is as below:
static struct camera_common_sensor_ops imx347_common_ops = {
.numfrmfmts = ARRAY_SIZE(imx347_frmfmt),
.frmfmt_table = imx347_frmfmt,
.power_on = imx347_power_on,
.power_off = imx347_power_off,
.write_reg = imx347_write_reg,
.read_reg = imx347_read_reg,
.parse_dt = imx347_parse_dt,
.power_get = imx347_power_get,
.power_put = imx347_power_put,
.set_mode = imx347_set_mode,
.start_streaming = imx347_start_streaming,
.stop_streaming = imx347_stop_streaming,
};
Please put some debug messages on your imx347_set_mode function to check which is the mode gstreamer is asking for.
Maybe you expect a particular mode to become active, but gstreamer will ask for another one.
Hello tessadriandrea,
After modifying the active_w and active_h (with 1920 x 1082), I’m able to get the v4l2 ctl with no error, I belive that the Jetson nano has rules of 4 charactors aligning, so 2712 * 1538 is not OK.
But still when I try to run with nvarguscamerasrc, I got no response after serveral print-out.
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=1920, height=1082, framerate=30/1, format=NV12' ! nvvidconv flip-method=0 ! 'video/x-raw,width=960, height=616' ! nvvidconv ! nvegltransform ! nveglglessink -e
Could you kindly help to check the log for the possible reason?
[ 50.472224] imx347 6-001a: [imx347]: imx347_power_on: power on
[ 50.472231] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio low
[ 50.472235] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio high
[ 50.495463] imx347 6-001a: [imx347]: imx347_power_off: power off
[ 50.496699] imx347 6-001a: [imx347]: imx347_power_on: power on
[ 50.496706] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio low
[ 50.496711] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio high
[ 50.519949] imx347 6-001a: [imx347]: imx347_power_off: power off
[ 50.520159] imx347 6-001a: [imx347]: imx347_power_on: power on
[ 50.520167] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio low
[ 50.520173] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio high
[ 50.543602] imx347 6-001a: [imx347]: imx347_power_off: power off
[ 50.788056] imx347 6-001a: [imx347]: imx347_power_on: power on
[ 50.788064] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio low
[ 50.788068] imx347 6-001a: [imx347]: imx347_power_on: power reset gpio high
[ 50.840974] imx347 6-001a: [imx347]: imx347_set_mode: set sensor mode: 1
[ 50.840990] imx347 6-001a: [imx347]: imx347_start_streaming: sensor working mode: 0x0
[ 50.840994] imx347 6-001a: [imx347]: imx347_start_streaming: sensor lane mode: 0x1
[ 50.840997] imx347 6-001a: [imx347]: start streaming
[ 52.567021] fence timeout on [ffffffc0eef28d80] after 1500ms
[ 52.567071] fence timeout on [ffffffc0eef28e40] after 1500ms
[ 52.567125] name=[nvhost_sync:21], current value=0 waiting value=1
[ 52.567147] name=[nvhost_sync:20], current value=0 waiting value=1
[ 52.567186] ---- mlocks ----
[ 52.567195] ---- mlocks ----
[ 52.567263] ---- syncpts ----
[ 52.567273] ---- syncpts ----
[ 52.567345] id 5 (disp1_a) min 347 max 347 refs 1 (previous client : )
[ 52.567393] id 6 (disp1_b) min 2 max 2 refs 1 (previous client : )
[ 52.567436] id 7 (disp1_c) min 2 max 2 refs 1 (previous client : )
[ 52.567485] id 9 (gm20b_507) min 16012 max 16012 refs 1 (previous client : )
[ 52.567531] id 11 (gm20b_506) min 26 max 26 refs 1 (previous client : )
[ 52.567578] id 12 (gm20b_505) min 1794 max 1794 refs 1 (previous client : gm20b_505)
[ 52.567645] id 20 (54680000.isp_0) min 0 max 3 refs 4 (previous client : )
[ 52.567688] id 21 (54680000.isp_1) min 0 max 3 refs 4 (previous client : )
[ 52.567730] id 22 (54680000.isp_2) min 7 max 15 refs 10 (previous client : )
[ 52.567772] id 23 (54680000.isp_3) min 0 max 3 refs 4 (previous client : )
[ 52.567815] id 24 (gm20b_504) min 6 max 6 refs 1 (previous client : )
[ 52.567861] id 25 (gm20b_503) min 686 max 686 refs 1 (previous client : )
[ 52.567908] id 27 (vblank1) min 3036 max -4 refs 1 (previous client : )
[ 52.567950] id 28 (gm20b_502) min 2 max 2 refs 1 (previous client : )
[ 52.567990] id 29 (gm20b_501) min 2 max 2 refs 1 (previous client : )
[ 52.568031] id 30 (gm20b_500) min 2 max 2 refs 1 (previous client : )
[ 52.568072] id 31 (gm20b_499) min 4 max 4 refs 1 (previous client : )
[ 52.568125] id 33 (54080000.vi_1) min 0 max 3 refs 4 (previous client : 54080000.vi_1)
[ 52.568170] id 34 (54080000.vi_2) min 2 max 7 refs 7 (previous client : 54080000.vi_2)
[ 52.568219] id 37 (54080000.vi_4) min 0 max 3 refs 4 (previous client : 54080000.vi_4)
[ 52.568268] id 40 (gm20b_498) min 6 max 6 refs 1 (previous client : )
[ 52.568310] id 41 (gm20b_497) min 8 max 8 refs 1 (previous client : )
[ 52.568352] id 42 (gm20b_496) min 6 max 6 refs 1 (previous client : )
[ 52.568394] id 43 (gm20b_495) min 6 max 6 refs 1 (previous client : )
[ 52.568435] id 44 (gm20b_494) min 6 max 6 refs 1 (previous client : )
[ 52.568475] id 45 (gm20b_493) min 6 max 6 refs 1 (previous client : )
[ 52.568978] ---- channels ----
[ 52.569049]
channel 0 - 54600000.isp
[ 52.569095] 0-54600000.isp (1):
[ 52.569125] inactive
[ 52.569174]
channel 1 - 54680000.isp
[ 52.569215] 1-54680000.isp (18):
[ 52.569264] active class 01, offset 0050, val 00000014
[ 52.569304] DMAPUT 00000198, DMAGET 00000110, DMACTL 00000000
[ 52.569338] CBREAD 00000014, CBSTAT 00010050
[ 52.569403]
ffffffc0b6f71800: JOB, syncpt_id=20, syncpt_val=1, first_get=00000090, timeout=10000, num_slots=3, num_handles=1
[ 52.569500] GATHER at ffc30000+4c54, 8 words
[ 52.569530] 20000001
[ 52.569561] 00000414
[ 52.569587] 20000001
[ 52.569615] 00000515
[ 52.569636] 20000001
[ 52.569648] 00000617
[ 52.569659] 200c0001
[ 52.569669] 00000005
[ 52.569695] GATHER at ffc30000+4c74, 2 words
[ 52.569704] 20000001
[ 52.569714] 00000016
[ 52.569747]
channel 2 - 54080000.vi
[ 52.569765] 2-54080000.vi (13):
[ 52.569779] active class 01, offset 0050, val 00000021
[ 52.569792] DMAPUT 00000100, DMAGET 00000100, DMACTL 00000000
[ 52.569803] CBREAD 00000021, CBSTAT 00010050
[ 52.569827]
ffffffc0b6f73800: JOB, syncpt_id=33, syncpt_val=1, first_get=00000018, timeout=10000, num_slots=7, num_handles=1
[ 52.569856] GATHER at ffb98000+0000, 35 words
[ 52.569865] 10620001
[ 52.569875] 0000001f
[ 52.569884] 10630002
[ 52.569894] 00000000
[ 52.569903] 00000000
[ 52.569913] 103a0001
[ 52.569937] 10100010
[ 52.569947] 103c0001
[ 52.569956] 10101010
[ 52.569965] 10420001
[ 52.569975] 00000001
[ 52.569985] 10430006
[ 52.569995] 00000000
[ 52.570006] 001c984c
[ 52.570016] 00000000
[ 52.570026] 00000000
[ 52.570036] 00000000
[ 52.570047] 00000000
[ 52.570057] 10590001
[ 52.570068] 00000003
[ 52.570079] 120e0006
[ 52.570090] 007f0015
[ 52.570101] 080301f0
[ 52.570112] 00000000
[ 52.570123] 00140000
[ 52.570135] 0000f005
[ 52.570161] 00000000
[ 52.570190] 12420001
[ 52.570211] 00000001
[ 52.570234] 124b0003
[ 52.570257] 00000000
[ 52.570282] 00000000
[ 52.570454] 00000000
[ 52.570470] 128b0001
[ 52.570499] 00000000
[ 52.570563] GATHER at ffb98000+008c, 3 words
[ 52.570587] 304e0005
[ 52.570615] 00000007
[ 52.570639] 00000016
[ 52.570695] GATHER at ffb98000+0098, 17 words
[ 52.570719] 10430006
[ 52.570744] 00200004
[ 52.570759] 001c984c
[ 52.570773] 00000000
[ 52.570792] 043a0780
[ 52.570812] 00000960
[ 52.570824] 0000002b
[ 52.570837] 124b0003
[ 52.570853] 00000000
[ 52.570865] 00000000
[ 52.570876] 00000009
[ 52.570887] 20000001
[ 52.570898] 00001d21
[ 52.570910] 20000001
[ 52.570922] 00000525
[ 52.570933] 20410001
[ 52.570951] 00000001
[ 52.570978] GATHER at ffb98000+00dc, 2 words
[ 52.570990] 20000001
[ 52.571002] 00000022
[ 52.571056]
---- host general irq ----
[ 52.571068] sync_hintmask_ext = 0xc0000000
[ 52.571080] sync_hintmask = 0x80000000
[ 52.571092] sync_intc0mask = 0x00000001
[ 52.571104] sync_intmask = 0x00000011
[ 52.571117]
---- host syncpt irq mask ----
[ 52.571130] syncpt_thresh_int_mask(0) = 0x00000001
[ 52.571143] syncpt_thresh_int_mask(1) = 0x00005500
[ 52.571156] syncpt_thresh_int_mask(2) = 0x00000414
[ 52.571169] syncpt_thresh_int_mask(3) = 0x00000000
[ 52.571183] syncpt_thresh_int_mask(4) = 0x00000000
[ 52.571196] syncpt_thresh_int_mask(5) = 0x00000000
[ 52.571210] syncpt_thresh_int_mask(6) = 0x00000000
[ 52.571224] syncpt_thresh_int_mask(7) = 0x00000000
[ 52.571239] syncpt_thresh_int_mask(8) = 0x00000000
[ 52.571253] syncpt_thresh_int_mask(9) = 0x00000000
[ 52.571268] syncpt_thresh_int_mask(10) = 0x00000000
[ 52.571282] syncpt_thresh_int_mask(11) = 0x00000000
[ 52.571297]
---- host syncpt irq status ----
[ 52.571311] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[ 52.571326] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[ 52.571341] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[ 52.571355] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[ 52.571370] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[ 52.571385] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[ 52.571400]
---- host syncpt thresh ----
[ 52.571417] syncpt_int_thresh_thresh_0(0) = 1
[ 52.571447] syncpt_int_thresh_thresh_0(20) = 1
[ 52.571463] syncpt_int_thresh_thresh_0(21) = 1
[ 52.571505] syncpt_int_thresh_thresh_0(22) = 8
[ 52.571522] syncpt_int_thresh_thresh_0(23) = 1
[ 52.571545] syncpt_int_thresh_thresh_0(33) = 1
[ 52.571562] syncpt_int_thresh_thresh_0(34) = 3
[ 52.571579] syncpt_int_thresh_thresh_0(37) = 1
[ 52.571637] id 5 (disp1_a) min 347 max 347 refs 1 (previous client : )
[ 52.571669] id 6 (disp1_b) min 2 max 2 refs 1 (previous client : )
[ 52.571696] id 7 (disp1_c) min 2 max 2 refs 1 (previous client : )
[ 52.571722] id 9 (gm20b_507) min 16012 max 16012 refs 1 (previous client : )
[ 52.571747] id 11 (gm20b_506) min 26 max 26 refs 1 (previous client : )
[ 52.571767] id 12 (gm20b_505) min 1794 max 1794 refs 1 (previous client : gm20b_505)
[ 52.571792] id 20 (54680000.isp_0) min 0 max 3 refs 4 (previous client : )
[ 52.571806] id 21 (54680000.isp_1) min 0 max 3 refs 4 (previous client : )
[ 52.571820] id 22 (54680000.isp_2) min 7 max 15 refs 10 (previous client : )
[ 52.571835] id 23 (54680000.isp_3) min 0 max 3 refs 4 (previous client : )
[ 52.571850] id 24 (gm20b_504) min 6 max 6 refs 1 (previous client : )
[ 52.571865] id 25 (gm20b_503) min 686 max 686 refs 1 (previous client : )
[ 52.571882] id 27 (vblank1) min 3037 max -4 refs 1 (previous client : )
[ 52.571896] id 28 (gm20b_502) min 2 max 2 refs 1 (previous client : )
[ 52.571910] id 29 (gm20b_501) min 2 max 2 refs 1 (previous client : )
[ 52.571934] id 30 (gm20b_500) min 2 max 2 refs 1 (previous client : )
[ 52.571948] id 31 (gm20b_499) min 4 max 4 refs 1 (previous client : )
[ 52.571964] id 33 (54080000.vi_1) min 0 max 3 refs 4 (previous client : 54080000.vi_1)
[ 52.571980] id 34 (54080000.vi_2) min 2 max 7 refs 7 (previous client : 54080000.vi_2)
[ 52.574426] id 37 (54080000.vi_4) min 0 max 3 refs 4 (previous client : 54080000.vi_4)
[ 52.574441] id 40 (gm20b_498) min 6 max 6 refs 1 (previous client : )
[ 52.574452] id 41 (gm20b_497) min 8 max 8 refs 1 (previous client : )
[ 52.574462] id 42 (gm20b_496) min 6 max 6 refs 1 (previous client : )
[ 52.574472] id 43 (gm20b_495) min 6 max 6 refs 1 (previous client : )
[ 52.574481] id 44 (gm20b_494) min 6 max 6 refs 1 (previous client : )
[ 52.574490] id 45 (gm20b_493) min 6 max 6 refs 1 (previous client : )
Hi Yawei Yang,
it happened to me also to have this fence timeout on [ffffffc0eef28d80] after 1500ms.
Please post your frmfmt array (written in the driver) and your modes on DTS so that we can cross check if there is a mismatch between frame rates and pixel clock.
The main issue here is that gstreamer with libargus selects a capture mode that is different by the one you set with registers. So, CSI + VI are set to run at pixel clock and frame rate different from what the sensor is programmed for.
Looking forward
Andrea
Hello tessadriandrea,
The mode I added in DTS:
mode0 { /* imx347_MODE_1920x1082_30FPS */
mclk_khz = "24000";
num_lanes = "2";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "yes";
dpcm_enable = "false";
cil_settletime = "0";
active_w = "1920";
active_h = "1082";
mode_type = "bayer";
pixel_phase = "rggb";
csi_pixel_bit_depth = "10";
readout_orientation = "0";
line_length = "2712";
inherent_gain = "1";
mclk_multiplier = "7.5";
pix_clk_hz = "178200000";
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "16"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "30000000"; /* 30.0 fps */
step_framerate = "1";
default_framerate = "30000000"; /* 30.0 fps */
min_exp_time = "13"; /* us */
max_exp_time = "683709"; /* us */
step_exp_time = "1";
default_exp_time = "2495"; /* us */
embedded_metadata_height = "1";
};
mode1 { /* IMX347_MODE_2704X1538_CROP_30FPS */
mclk_khz = "24000";
num_lanes = "2";
tegra_sinterface = "serial_a";
phy_mode = "DPHY";
discontinuous_clk = "no";
dpcm_enable = "false";
cil_settletime = "0";
active_w = "2704";
active_h = "1538";
mode_type = "bayer";
pixel_phase = "rggb";
csi_pixel_bit_depth = "10";
readout_orientation = "0";
line_length = "2720";
inherent_gain = "1";
mclk_multiplier = "7.5";
pix_clk_hz = "178200000";
gain_factor = "16";
framerate_factor = "1000000";
exposure_factor = "1000000";
min_gain_val = "16"; /* 1.00x */
max_gain_val = "170"; /* 10.66x */
step_gain_val = "1";
default_gain = "16"; /* 1.00x */
min_hdr_ratio = "1";
max_hdr_ratio = "1";
min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "30000000"; /* 30.0 fps */
step_framerate = "1";
default_framerate = "30000000"; /* 30.0 fps */
min_exp_time = "13"; /* us */
max_exp_time = "683709"; /* us */
step_exp_time = "1";
default_exp_time = "2495"; /* us */
embedded_metadata_height = "1";
};
and the frmfmt array:
static const int imx347_30fps[] = {
30,
};
/*
* WARNING: frmfmt ordering need to match mode definition in
* device tree!
*/
static const struct camera_common_frmfmt imx347_frmfmt[] = {
{{1920, 1082}, imx347_30fps, 1, 0,
IMX347_MODE_1920X1082_CROP_30FPS},
{{2704, 1538}, imx347_30fps, 1, 0,
IMX347_MODE_2704X1538_CROP_30FPS},
};
And the previous log I post was testing 1920x1082 mode. Please kindly help take a cross check. Thanks!
Hi Yawei Yang,
in your previous log it seams you were working with sensor mode 1, was the desired mode you expected ?
[ 50.840974] imx347 6-001a: [imx347]: imx347_set_mode: set sensor mode: 1
[ 50.840990] imx347 6-001a: [imx347]: imx347_start_streaming: sensor working mode: 0x0
[ 50.840994] imx347 6-001a: [imx347]: imx347_start_streaming: sensor lane mode: 0x1
There is a very important difference between mode 0 and mode 1 that is discontinuous_clk: please try for each mode discontinuous_clk = “yes” and discontinuous_clk = “no”.
How about line_length ?
For mode 0 is 2712, for mode 1 is 2720: have you checked these values with the sensor datasheet ?
Pixel clock for both modes is the same: is it correct?
I suppose that if the frame size is different between the 2 modes and the pixel clock is the same you will for sure expects different frame rates.
Hello tessadriandrea,
For the mode index, previous I faced with the issue of active_w and active_h, I re-ordered the mode index, and currently I’m working with mode0, which do not have the issue “TEGRA_VI_CSI_ERROR_STATUS 0x00000001”, and only the argus_camera not work.
For the discontinuous_clk, I have tried both “yes” and “no”, and neither of them worked. I’m wondering the meaning of this setting, and which item in sensor datasheet it can be referred to?
For the line_length, as imx347 is not for consumer industry, after consulted with Sony, I used 2712 in mode0: 1920x1082
For pixel clock, as the posting from NVIDIA, there 4 ways to calculate, I’m also feel confused, here I used the sensor datarate to cacluate: 891Mbps * 2 /10, as I have 2 lane CSI and used 10 bits/pixel.
Thanks for your help.
Please post output messages from gstreamer when you get the error.
I am really wondering if gstreamer was trying to capture at a frame rate different from the one you expect.
Try also to force only one mode in your device tree and driver, so that gstreamer won’t have any choice except the one you want to use.
Try also to force a fix frame rate: do you want 30 fps ? so in device tree use min frame rate = 29000000 and max = 30000000 (don’t let gstreamer to make compromises …).
Hello tessadriandrea,
I posted the log got from dmesg before, any specific optional need to be set to enable the gstreamer log?
By the way, I have tried the set the min frame rate = 29000000 and max = 30000000 in device tree, by getting the the same result.
Thanks!
At the end of the statement that you use to start gstreamer, please add -ev in order to increase verbosity.
You should have a detailed description of the freme rate selected by gstreamer and, maybe, a reason for the error.
Thanks tessadriandrea,
The issue is fixed, the sensor I’m using is output continuous clk, previously I also tried but not found the root cause, as it coupled with the “4-characters-aligning rule” issue.
Thanks for your support.