I encounter the port binding issue during camera porting by using the Main Platform Device Tree File. To verify dts setting, I just use the tegra234-p3737-0000-camera-imx185-a00.dtsi in the released code, the port parsing issue still there.
I remoe other camera sensor support in the camera module dts, and change the status to ‘okay’, and use FDT to point new built dtb file then check kernel boot log and found the port parsing issue.
Please help me how to use main platform dts file to enable camera sensor.
Thanks!
tegra234-p3737-camera-modules.txt (13.0 KB)
tegra234-camera-imx185-a00.txt (14.2 KB)
kernel_log.txt (74.9 KB)
The Dev kit was Jetson AGX Orin Develop Kit 64G, and the Jetpak version was Jetpack 5.1.2 [L4T 35.4.1].
Some fail log:
[ 4.685866] tegra-camrtc-capture-vi tegra-capture-vi: tegra_channel_csi_init:Fail to parse port info
[ 4.695982] tegra-camrtc-capture-vi tegra-capture-vi: channel init failed
[ 4.703684] tegra-camrtc-capture-vi tegra-capture-vi: all channel init failed
[ 4.711745] tegra-camrtc-capture-vi tegra-capture-vi: Init channel failed
[ 4.719501] tegra-camrtc-capture-vi tegra-capture-vi: tegra_vi_media_controller_init_int: failed
[ 4.729252] tegra-camrtc-capture-vi tegra-capture-vi: media controller init failed
Hi,
For the camera basic functionality first needs to check the device and driver configuration.
You can reference to below program guide for the detailed information of device tree and driver implementation.
Please refer to Applications Using V4L2 IOCTL Directly by using V4L2 IOCTL to verify basic camera functionality.
Once confirm the configure and still failed below link help to get log and some information and some tips for debug.
Thanks!
hello xianchao.zhang,
may I know what’s the actual CSI port you’re used?
please see-also Release Notes (r35.6.1) of [4. Implementation Details] to create a device tree overlay (DTB overlay or .dtbo) to register the camera module.
Thank carolyuu and JerryChang.
I workrond with dtbo method, with jetson-io tool, the generated dtb works.
However I encounter new error, the recived frame rate was too small, maybe only 1 frame, I had got the trace log and found there was intr_stat_pd_wc_short_err_vc0.
CPU 1 buffer started
vi-output, imx1-3912 [001] … 739.806127: tegra_channel_capture_frame: sof:769.572499424
vi-output, imx1-3912 [001] … 739.806130: tegra_channel_capture_frame: eof:769.588493824
vi-output, imx1-3912 [001] … 739.831145: tegra_channel_capture_frame: sof:769.589160224
vi-output, imx1-3912 [001] … 739.831148: tegra_channel_capture_frame: eof:769.605154624
vi-output, imx1-3912 [001] … 742.953707: tegra_channel_capture_setup: vnc_id 0 W 3840 H 2160 fmt 13
vi-output, imx1-3912 [001] … 743.288399: tegra_channel_capture_frame: sof:773.54607168
vi-output, imx1-3912 [001] … 743.288401: tegra_channel_capture_frame: eof:773.70601600
vi-output, imx1-3912 [001] … 743.796243: tegra_channel_capture_frame: sof:773.537770432
vi-output, imx1-3912 [001] … 743.796245: tegra_channel_capture_frame: eof:773.553764864
CPU 0 buffer started
kworker/0:10-232 [000] .... 744.118982: rtcpu_nvcsi_intr: tstamp:24182956847 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
kworker/0:10-232 [000] .... 744.118983: rtcpu_nvcsi_intr: tstamp:24182956847 class:CORRECTABLE_ERR type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
kworker/0:10-232 [000] .... 744.118983: rtcpu_nvcsi_intr: tstamp:24182957407 class:GLOBAL type:STREAM_VC phy:0 cil:0 st:0 vc:0 status:0x00000004
By the way, this time I use a hdmi-csi bridge to output the mipi csi signal, and the setting was 8bit uyvy@4K 60fps, could you please give me some tips for debugging such issue?
I have also upload the trace log csi-trace.txt (91.8 MB) and kernel log
kernel_log.txt (89.5 KB)
.
Thanks!
hello xianchao.zhang,
just double check, is it YUV420 or YUV422?
did you update VI driver to extend the support formats?
Hi JerryChange,
It’s YUV422(UYVY 4:2:2). Following was the key dts setting:
csi_pixel_bit_depth = "16";
mode_type = "yuv";
pixel_phase = "uyvy";
active_w = "3840";
active_h = "2160";
readout_orientation = "0";
line_length = "4400";
inherent_gain = "1";
/*mclk_multiplier = "24"; Deprecated*/
pix_clk_hz = "594000000";
please see-also Topic 307355.
Hi JerryChange,
Try the cmd, the issue still there:
$ v4l2-ctl --device /dev/video0 --set-fmt-video=width=3840,height=2160,pixelformat=NV16 --set-ctrl bypass_mode=0 --stream-mmap --stream-to=frame.yuv --stream-count=100
<<<<< 0.00 fps
<< 0.00 fps
<< 0.00 fps
<< 0.00 fps
<< 0.00 fps
<< 0.00 fps
<< 0.00 fps
<< 0.04 fps
<< 0.00 fps, dropped buffers: 4294967293
…
<< 0.00 fps
<
Here was the kernel key kernel log:
[ 2730.274407] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[ 2730.274413] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 45 bpl 3840 dt 30
[ 2730.292398] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 45 bpl 3840 dt 30
[ 2730.300866] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 45 bpl 3840 dt 30
[ 2730.309306] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 45 bpl 3840 dt 30
[ 2730.341955] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: flags 2, err_data 0
[ 2730.368705] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 45 bpl 3840 dt 30
[ 2730.412569] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 45 bpl 3840 dt 30
[ 2732.994532] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[ 2733.003683] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[ 2733.023840] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 2733.031564] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[ 2733.042236] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_set_config: mipi_clock_rate=1188412, csi_clk_freq=102000000
[ 2733.053969] (NULL device *): vi_capture_control_message: NULL VI channel received
[ 2733.061681] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
[ 2733.073433] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
whether this caused by mipi timing? could you please give me some clues?
hello xianchao.zhang,
could you please check.. $ v4l2-ctl --device /dev/video0 --list-formats-ext
BTW, there’s device tree setting, embedded_metadata_height, please configure it as zero if your sensor did not output embedded metadata.
Hi JerryChang:
$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'UYVY' (UYVY 4:2:2)
Size: Discrete 3840x2160
Interval: Discrete 0.017s (60.000 fps)
[1]: 'NV16' (Y/CbCr 4:2:2)
Size: Discrete 3840x2160
Interval: Discrete 0.017s (60.000 fps)
[2]: 'UYVY' (UYVY 4:2:2)
Size: Discrete 3840x2160
Interval: Discrete 0.017s (60.000 fps)
For metadata, the bridge had no metadata, and I set the height to 0 in dts file:
default_exp_time = "33334";/* us */
embedded_metadata_height = "0";
};
hello xianchao.zhang,
is it a bridge driver? is it possible to double check it’s output the stream actually?
those error logs per your v4l test pipeline, it meant VI cannot receive a frame within 2500ms.
you may give it a try to boost the VI/CSI/EMC clocks for testing.
for instance,
sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
Thanks JerryChang.
The bridge chip firmware refresh freq was 60hz, if the input of the bridge was 4k30fps, kernel would not timeout, the recived frame looks fine, but the trace log still arise status:0x00000004, once the input of the bridge was 4k60fps, kernel will arise timeout, the trace log also arise status:0x00000004, the recived frame rate was less then 1fps.
I have also boost the clock, but there was no improvement.
Here was the bridge uart output;
[DEBUG] ulMipiTxDataRateIncre:120000
[DEBUG] ulHalfPixClk:148495 calc ulByteclk:163495, ulPhyClk:1307960, ucBpp:16, ucTxPortNum:0x01, ucTxLaneNum:0x04
[DEBUG] prediv_set N1:0x01, freq_set N2:0x01, tbcr set M1:0x02, ucDivSet M2:0x1b
[DEBUG] LMTXPLL calibration bandout: 0x01 0x01
[INFO ] MIPI TX[H]: Htotal:4400,Hactive:3840,Hsw:88,Hfp:176,Hbp:296,Hspol:1
[INFO ] MIPI TX[V]: Vtotal:2250,Vactive:2160,Vsw:10,Vfp:8,Vbp:72,Vspol:1
Is there any param we could adapt the mipi timing of the bridge?
Thanks!
xianchao
please double check Sensor Pixel Clock section to review your device tree settings.
Thanks JerryChang.
I have try several way as following:
- 4400*2250*60 = 594000000 using frame size & frame rate
- 312104*8*4/16*1000 = 624208000 using csi output date rate
- 624208000*1.15 = 717834600
But there is no improvment.
For testing, I have change the bridge and kernel driver/dts down to 1080@60P, it works well.
I suspect whether is it the mipi bandwith limitation? Do we need add deskew calibration for 4k@60p?
Or is there anything else I can try?
Thanks!
hello xianchao.zhang,
may I know the actual output data-rate of 4K@60p?
note, skew calibration is required if sensor or deserializer is using DPHY, and the output data rate is > 1.5Gbps.
Hi JerryChang,
The mipi clk was 2496816k from the uart log, and the data rate should > 1.5Gbps.
I have tried set deskew_initial_enable to ‘true’, but there was kernel NULL pointer. So if we should enable deskew featrue, how to do it?
Thanks!
xianchao
hello xianchao.zhang,
there’re some camera software changes to update deskew algorithm, and also stability fixes.
is it possible for moving forward to latest JetPack 5.1.5, or even JP-6 to include all the changes for verification?
for instance,
Topic 268833: JP-5.1.2 camera firmware to update deskew algorithm, and also stability fixes
Topic 268519: fix memory corruption within libnvargus.
Topic 309780: RCE: General error queue is out of sync with frame queue.
..etc.
Hi JerryChang
I have update the camera firmware Debug_camera-rtcpu-t234-rce.img
and set deskew_initial_enable = “true”; in the dts file, but still encounter Null pointer.
[ 163.066053] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 19 bpl 7680 dt 30
[ 163.069886] bwmgr API not supported
[ 163.074597] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 19 bpl 7680 dt 30
[ 163.083455] [RCE] calibration status1 21284293 status2 1f29ce71
[ 163.085770] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_set_config: mipi_clock_rate=2304000, csi_clk_freq=102000000
[ 163.086325] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008
[ 163.086327] Mem abort info:
[ 163.086328] ESR = 0x96000004
[ 163.086330] EC = 0x25: DABT (current EL), IL = 32 bits
[ 163.086331] SET = 0, FnV = 0
[ 163.086332] EA = 0, S1PTW = 0
[ 163.086333] Data abort info:
[ 163.086334] ISV = 0, ISS = 0x00000004
[ 163.086335] CM = 0, WnR = 0
[ 163.086337] user pgtable: 4k pages, 48-bit VAs, pgdp=000000010bb3a000
[ 163.086338] [0000000000000008] pgd=0000000000000000, p4d=0000000000000000
[ 163.086344] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 163.086347] Modules linked in: lt6911uxe(E) xt_conntrack(E) xt_MASQUERADE(E) nf_conntrack_netlink(E) nfnetlink(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) xt_addrtype(E) iptable_filter(E) br_netfilter(E) nvidia_modeset(OE) fuse(E) lzo_rle(E) lzo_compress(E) zram(E) overlay(E) ramoops(E) reed_solomon(E) bnep(E) snd_soc_tegra210_ope(E) snd_soc_tegra210_dmic(E) snd_soc_tegra210_mvc(E) input_leds(E) snd_soc_tegra210_iqc(E) snd_soc_tegra210_admaif(E) snd_soc_tegra186_asrc(E) snd_soc_tegra186_dspk(E) rtl8822ce(E) snd_soc_tegra186_arad(E) snd_soc_tegra210_amx(E) snd_soc_tegra210_afc(E) snd_soc_tegra210_adx(E) rtk_btusb(E) snd_soc_tegra210_mixer(E) snd_soc_tegra_pcm(E) snd_soc_tegra210_sfc(E) snd_soc_tegra210_i2s(E) aes_ce_blk(E) crypto_simd(E) btusb(E) btrtl(E) cryptd(E) cam_cdi_tsc(E) btbcm(E) snd_soc_tegra210_adsp(E) aes_ce_cipher(E) ghash_ce(E) btintel(E) snd_hda_codec_hdmi(E) snd_hda_tegra(E) sha2_ce(E) snd_hda_codec(E) nv_hawk_owl(E)
[ 163.086407] snd_soc_tegra_machine_driver(E) ucsi_ccg(E) snd_soc_tegra_utils(E) sha256_arm64(E) sha1_ce(E) nvadsp(E) snd_soc_simple_card_utils(E) typec_ucsi(E) snd_hda_core(E) snd_soc_tegra210_ahub(E) max96712(E) snd_soc_rt5640(E) snd_soc_spdif_tx(E) nct1008(E) cfg80211(E) typec(E) i2c_nvvrs11(E) snd_soc_rl6231(E) tegra210_adma(E) tegra_bpmp_thermal(E) nvidia(OE) userspace_alert(E) spi_tegra114(E) loop(E) binfmt_misc(E) ina3221(E) pwm_fan(E) nvgpu(E) nvmap(E) ip_tables(E) x_tables(E) [last unloaded: nv_imx185]
[ 163.086443] CPU: 2 PID: 3650 Comm: v4l2src0:src Tainted: G OE 5.10.120-tegra #13
[ 163.086444] Hardware name: Unknown Jetson AGX Orin Developer Kit/Jetson AGX Orin Developer Kit, BIOS 4.1-33958178 08/01/2023
[ 163.086447] pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=–)
[ 163.086458] pc : nvcsi_deskew_setup+0x290/0x4f8
[ 163.086460] lr : nvcsi_deskew_setup+0xc4/0x4f8
[ 163.086461] sp : ffff8000153eb920
[ 163.086462] x29: ffff8000153eb920 x28: 0000000000000004
[ 163.086465] x27: ffffaaa9dbf62908 x26: 000000000000000c
[ 163.086466] x25: 0000000000000000 x24: 0000000000000016
[ 163.086468] x23: 000000000000000f x22: ffff24bfc1fac280
[ 163.086470] x21: 0000000000000000 x20: 000000000000000f
[ 163.086472] x19: 0000000000000000 x18: 0000000000000001
[ 163.086474] x17: 0000000000000000 x16: ffffaaa9da03d608
[ 163.086476] x15: ffff24b12967ecb0 x14: 0000000000000000
[ 163.086478] x13: 0000000000000000 x12: 0000000000000000
[ 163.086480] x11: 0000000000000000 x10: 0000000000000000
[ 163.086482] x9 : 0000000000000000 x8 : 0000000000000000
[ 163.086484] x7 : ffff000000000000 x6 : 0000000000000001
[ 163.086486] x5 : 0000000000000001 x4 : 0000000000000004
[ 163.086488] x3 : ffff24b12967e740 x2 : 0000000000000000
[ 163.086489] x1 : 0000000000000000 x0 : 0000000000000000
[ 163.086492] Call trace:
[ 163.086494] nvcsi_deskew_setup+0x290/0x4f8
[ 163.086498] tegra_csi_s_stream+0x22c/0x2f8
[ 163.086501] tegra_channel_set_stream+0x538/0x570
[ 163.086503] vi5_channel_start_streaming+0x280/0x3f8
[ 163.086505] tegra_channel_start_streaming+0x50/0x78
[ 163.086508] vb2_start_streaming+0x6c/0x150
[ 163.086510] vb2_core_streamon+0x98/0x198
[ 163.086512] vb2_streamon+0x30/0x78
[ 163.086514] vb2_ioctl_streamon+0x54/0x60
[ 163.086517] v4l_streamon+0x3c/0x50
[ 163.086518] __video_do_ioctl+0x180/0x3e8
[ 163.086520] video_usercopy+0x27c/0x788
[ 163.086522] video_ioctl2+0x3c/0x178
[ 163.086524] v4l2_ioctl+0x64/0x90
[ 163.086528] __arm64_sys_ioctl+0xa8/0xe8
[ 163.086532] el0_svc_common.constprop.0+0x7c/0x1c0
[ 163.086534] do_el0_svc+0x34/0xa0
[ 163.086538] el0_svc+0x1c/0x28
[ 163.086539] el0_sync_handler+0xa8/0xb0
[ 163.086541] el0_sync+0x16c/0x180
[ 163.086545] Code: 17ffffa2 f9404b60 b9400b62 b9400361 (f9400400)
[ 163.086555] —[ end trace 499854635eb24d5b ]—
[ 163.087526] tegra-camrtc-capture-vi tegra-capture-vi: h 2160 w 3840 fmt 19 bpl 7680 dt 30
[ 163.091717] Kernel panic - not syncing: Oops: Fatal exception
[ 163.091722] SMP: stopping secondary CPUs
[ 163.093619] Kernel Offset: 0x2aa9c9f50000 from 0xffff800010000000
[ 163.093620] PHYS_OFFSET: 0xffffdb5040000000
[ 163.093622] CPU features: 0x08040006,4a80aa38
[ 163.093623] Memory Limit: none
[ 163.556525] —[ end Kernel panic - not syncing: Oops: Fatal exception ]—
hello xianchao.zhang,
that’s due to NULL pointer dereference, please dig into your device tree settings for checking.
BTW,
you may see-also AGX Orin’s reference driver for cross-check.
for instance,
$public_sources/r35.6.0/Linux_for_Tegra/source/public/kernel_src/hardware/nvidia/platform/t23x/common/kernel-dts/t234-common-modules/tegra234-camera-imx185-a00.dtsi
Thanks JerryChang!
I found a workaround way that streaming 4k@60p.
First streaming upon lower input pixel clock 296992000, then boost the input pixel clock
upto 593980000 online, tthis way the 4k@60p works fine.
It looks like mipi timing issue, is it right? Could you please give some tips for such issue?
Thanks!
xianchao