Adv7282 PAL to MIPI CSI Linux Driver Development

Hi,

I have few doutbs

  1. for command
    v4l2-ctl --list-formats-ext

the response i see is as below ,only thing is i could not find Size parameter output in it, anyways info all command throw the set resolution anyway as posted previously.

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2

  1. For format of 720x572 i was not seeing short frame error or long frame error but was seeing CHANSEL_FAULT, which mainly occurs due to mismatch between input resolution in DTB and driver(understanding from other forum topics)
    why is it happening here i am not able to trace exactly in my case, where i am hardcoding the resolution 720x572

trace log attached again
trace.log (25.7 KB)

  1. has it got anything to do with stride alignment as raised in some other questions on forum ?

This log show the pixel run away.
PIXEL_RUNAWAY means VI receives more lines than expected (frame too high)
You may need to trace the driver for the resolution report.

CHANSEL_FAULT data bit field
    bit 16:31  LINE_NUMBER
    bit 15	DTYPE_MISMATCH
    bit 14	EMBED_INFRINGE
    bit 13	EMBED_LONG_LINE
    bit 12	EMBED_SPURIOUS
    bit 11	EMBED_RUNAWAY
    bit 10	EMBED_MISSING_LE
    bit 9	PIXEL_SHORT_LINE
    bit 8	PIXEL_LONG_LINE
    bit 7	PIXEL_SPURIOUS
    bit 6	PIXEL_RUNAWAY
    bit 5	PIXEL_MISSING_LE
    bit 4	PIXEL_LINE_TIMER
    bit 3	EMBED_EOF
    bit 2	EMBED_SOF
    bit 1	PIXEL_EOF
    bit 0	PIXEL_SOF

Correct, if the v4l2 driver and architecture has a difference in line width the PIXEL RUNAWAY issue may occur, as per my understanding the issue in the most forums is resolved by changing device tree width and height

In my approach there is not width and height options in device tree

the driver is a functional release after resolution with Jetpack 4.2 in below thread.
ADV7280m configuration - Jetson & Embedded Systems / Jetson TX2 - NVIDIA Developer Forums

Jetpack 4.2 and 4.3 does they have any wide difference when it comes to v4l2 implementations ??

Suppose J4.2 and J4.3 shouldn’t have huge different. I would suggest to trace the driver to knowing more for it.

I am verifying step by step, first i have modified the device tree for single channel single lane mipi csi for which i dont have detection problems, but one clarity i am missing is the tegra camera platform parameters, and patch requirement

  1. can you please clarify is there any mistake in below parameters for resolutions 720x576 for frame rate of 24fps

     num_csi_lanes = <1>;
     max_lane_speed = <1500000>;
     min_bits_per_pixel = <10>;
     vi_peak_byte_per_pixel = <2>;
     vi_bw_margin_pct = <25>;
     max_pixel_rate = <240000>;
     isp_peak_byte_per_pixel = <5>;
     isp_bw_margin_pct = <25>;
    
  2. in other forum you have suggested a patch , is it required ?
    ADV7280m configuration - Jetson & Embedded Systems / Jetson TX2 - NVIDIA Developer Forums

  1. No any mistake.
  2. Those patch only for TX2. Similar patch for NX in below link.
    J4.5/J4.6
    IMX219 Raw8 mode on the XavierNX: Disabling CRC - #9 by ShaneCCC
    J4.3
    Problem with capturing images by Video4Linux - #13 by ShaneCCC

Irrespective of your patch i could not avoid the short frame error which is occuring

after apply the patch i am seeing a segmentation fault and trace cut from dmesg is below, i assume i applied it with out any errors, may be if the segfault is removed i may over come the issue, kindly help

[ 12.808819] ------------[ cut here ]------------
[ 12.809040] WARNING: CPU: 1 PID: 6226 at /dvs/git/dirty/git-master_linux/kernel/kernel-4.9/net/wireless/nl80211.c:14141 cfg80211_ch_switch_notify+0x150/0x170 [cfg80211]
[ 12.809286] Modules linked in: zram overlay bnep rtk_btusb btusb btrtl btbcm btintel rtl8822ce cfg80211 spidev userspace_alert nvgpu bluedroid_pm ip_tables x_tables

[ 12.809366] CPU: 1 PID: 6226 Comm: RTW_CMD_THREAD Not tainted 4.9.201-tegra #56
[ 12.809372] Hardware name: NVIDIA Jetson Xavier NX Developer Kit (DT)
[ 12.809378] task: ffffffc1cff19c00 task.stack: ffffffc1cfd98000
[ 12.809461] PC is at cfg80211_ch_switch_notify+0x150/0x170 [cfg80211]
[ 12.810093] LR is at rtw_cfg80211_ch_switch_notify+0x94/0x98 [rtl8822ce]
[ 12.810100] pc : [<ffffff80011be080>] lr : [<ffffff8001304ac8>] pstate: 60c00045
[ 12.810104] sp : ffffffc1cfd9bc20
[ 12.810110] x29: ffffffc1cfd9bc20 x28: ffffff8001295760
[ 12.810123] x27: ffffff800be331bc x26: ffffff800145a578
[ 12.810135] x25: ffffff80014de13c x24: ffffffc1e2afa000
[ 12.810148] x23: ffffffc1d5b72000 x22: ffffffc1d5b83800
[ 12.810160] x21: 0000000000000000 x20: ffffffc1cfd9bca8
[ 12.810172] x19: ffffffc1de07bc00 x18: 0000000000000030
[ 12.810185] x17: 0000007f7c4e4d40 x16: ffffff80082753c8
[ 12.810196] x15: 0000000000000000 x14: 0000000000000000
[ 12.810208] x13: 000000000000ffff x12: 13192d1a2d000100
[ 12.810220] x11: 0000000000000004 x10: 0000000000000002
[ 12.810232] x9 : 0000000000000004 x8 : ffffffc1cfd9bd3a
[ 12.810244] x7 : 0000000000000000 x6 : 0000000000000001
[ 12.810256] x5 : 0000000000000040 x4 : 0000000000000000
[ 12.810268] x3 : 0000096c00000001 x2 : ffffffc1d5b84848
[ 12.810280] x1 : 0000096c00000001 x0 : 0000000000000000

[ 12.810297] ---[ end trace 1b49f253cf19e93f ]---
[ 12.810412] Call trace:
[ 12.810497] [<ffffff80011be080>] cfg80211_ch_switch_notify+0x150/0x170 [cfg80211]
[ 12.811094] [<ffffff8001304ac8>] rtw_cfg80211_ch_switch_notify+0x94/0x98 [rtl8822ce]
[ 12.811663] [<ffffff80012956c4>] rtw_chk_start_clnt_join+0x43c/0x4d8 [rtl8822ce]
[ 12.812285] [<ffffff8001295998>] join_cmd_hdl+0x238/0x330 [rtl8822ce]
[ 12.812872] [<ffffff8001252420>] rtw_cmd_thread+0x308/0x608 [rtl8822ce]
[ 12.812886] [<ffffff80080db07c>] kthread+0xec/0xf0
[ 12.812896] [<ffffff80080838a0>] ret_from_fork+0x10/0x30

I have no experience with cameras, but am hoping to clear up some concepts related to the topic which might speed things up for you. This means possibly answering about ideas instead of concrete examples, and not a direct answer, and often not even directly related to your question (I’m trying to be thorough, and really need a summary of what current questions are for any hope of a useful answer…this is a long thread).

Regarding the kernel and modules, understand that some features are invasive and were never designed to work as a loadable module. An example is virtual memory for user space programs…swapping out for cases of insufficient RAM. One cannot simply add or remove that feature since it affects just about everything, and so this has no option to be built as a module. Building as a module does have extra requirements, and so typically something like a driver for a specific device can be built as a module, but not always.

When a module or kernel driver does load which is for some optional hardware (such as a camera sensor), then the driver needs to find the device before it can be used. For plug-n-play devices (including USB and many PCI devices) this is not a problem since the device itself can self-report. For devices which cannot self-report the device tree becomes the primary method of giving such details to the driver. For example, the device tree will mention which driver(s) the device can be used with, along with a physical address to find the device (in the kernel physical addresses are used, not virtual addresses). Other details also can be added if the driver has an option for those arguments, e.g., I imagine that if the camera has more than one resolution, then probably a default starting resolution would be a candidate for the device tree.

If you want to see the current device tree simply extract it and browse the file (it is human readable). On a running Jetson you can extract its current device tree like this (if you don’t have “dtc”, then add this with “sudo apt-get install device-tree-compiler”):
dtc -I fs -O dts -o extracted.dts /proc/device-tree

Regarding how modules load, examine the output you see from “uname -r”. The answer will usually be something similar to “4.9.140-tegra”. In that output the “4.9.140” is the source code release of the kernel, and the suffix “-tegra” is from the CONFIG_LOCALVERSION (someone set this kernel config feature to “-tegra” before compiling). Now look at this file location:
/lib/modules/$(uname -r)/kernel
…you will see all modules which were built for this kernel in the directory tree at that location.

If a kernel has a module loaded which was built against this combination of kernel release version and the kernel was configured the same as your running system, then the module can simply be copied to the correct subdirectory of “/lib/modules/$(uname -r)/kernel”. Modules in any other location will fail to be found. Modules compiled against a significantly different kernel configuration will be found but will fail to load or else will produce a kernel error upon load.

Typically, if you build a kernel which starts with the existing kernel’s configuration and “CONFIG_LOCALVERSION”, and you merely add another feature as a module, then you won’t need to build a new kernel (you could build just the module). As soon as something is added to the kernel configuration which is not a module (especially something not able to be built as a module), then you risk the new module (and perhaps older modules) failing to work with the kernel. If the configuration changes enough to risk this, and if you replace the kernel itself (the “/boot/Image”), then you would be advised to use a new “uname -r” (which means a new “CONFIG_LOCALVERSION” other than “-tegra”, e.g., “-test”), followed by installing all new modules and the Image. Unless you use some new feature which is invasive and not available as a module this shouldn’t be needed.

Note that the device tree is likely consulted as you load such modules for items which are not plug-n-play. This implies that the device tree is specific to kernel features, but the tree itself is just an argument passed to the features/drivers. The tree can be built in the kernel, but it isn’t because the tree is part of the kernel. The tree is actually able to be reverse compiled, edited, recompiled, and added back in as a simple file without ever touching the kernel source. The device tree exists because without such a method of supplying arguments you would need a different kernel driver for every possible configuration of that driver, e.g., if one manufacturer were to put the camera at one physical address, but another at a different physical address, then you’d need a driver for every possible variation. The device tree makes it possible to have one driver and then each manufacturer tweaks the device tree details for their hardware. Note that physical address and many other details of the device tree are specific to a specific carrier board. A Jetson on one carrier board will require a tree which is different than that of another manufacturer’s carrier board.

So as you build a kernel you device what device tree nodes are required. As an option that fragments of source to a device tree can be glued together to form a full tree source based on the features you’ve enabled. Then the tree can be built by the kernel via building the “dtbs” target. Even that tree will need edits for anything which is non-default in your carrier board and/or end device.

You can edit any tree, and for example, the extracted.dts file mentioned from above can be edited with any text editor. Then it could be recompiled:
dtc -I dts -O dtb -o edited.dtb extracted.dts

To safely try another kernel and/or tree, consider the following (related to installing the new tree without overwriting the old content)…

Examine file “/boot/extlinux/extlinux.conf”. This is just from a TX2, so it isn’t an exact match for the NX, but a boot entry looks like this:

...
DEFAULT primary
...

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

If you boot with serial console attached, then there is a short moment when you can pick a kernel. If you don’t pick a kernel, then it will boot the DEFAULT (there are some bugs which some releases also require the DEFAULT to be the first entry). The example looks for LABELprimary” when nothing is chosen.

Assuming you have serial console and can interrupt boot and pick a kernel, then you could safely do the following and keep the defaults, but pick your test versions without risk by adding content as follows:

  • Add a new extlinux.conf entry, e.g., something like this:
LABEL test
      MENU LABEL test kernel
      LINUX /boot/Image-test
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

Since you are going to add a device tree based on a file (there are also ways to do this as a partition), the entry would be further modified with a DTB entry:

LABEL test
      MENU LABEL test kernel
      FDT /boot/dtb/edited.dtb
      LINUX /boot/Image-test
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

Now, if you copied “edited.dtb” to “/boot/dtb/edited.dtb”, and you’ve picked the “test kernel” boot entry, expect your tree to be used without touching the original boot content. If it fails, then you simply pick the “primary kernel” entry. If that entry is still set as the default, then you don’t have to do anything to get the original boot.

Let’s extend this to also adding a very different kernel due to non-module changes which are invasive. Your extlinux.conf entry was already edited for the new kernel with name “Image-test” for the file, but you will also now need to install a complete set of modules built against that configuration at this location (just to emphasize, this is an example which assumes “uname -r” responds with “4.9.140-test”):
/lib/modules/4.9.140-test/kernel

I don’t know anything about the camera, and have not worked on cameras before, but if you were to find an example entry for your carrier board, then it might just be a direct addition of that entry into your tree. If not, then perhaps you can build the kernel source “dtbs” target to get an example, and then edit the content which shows up for your camera (but tailored to your carrier board).

As a caution note on kernel building, even if you are only building a module, you must take steps to propagate a proper configuration throughout the source before building. If you start by building the kernel itself, the “Image” target, then this will all be taken care of and you won’t need to propagate configuration. If you don’t build Image first, then to build a module you must propagate the configuration with the “make modules_prepare” step (if you are building in a temp location, then you’d have to include options to that location, and if cross compiling, then you’d also have to have the cross compile options set).

I don’t know what your current situation is since there is a lot going on in this thread, and I am the wrong guy to ask about camera drivers, but if you summarize the current steps you want elaboration on, then I can probably give more exact answers (the above is purposely generalized since I don’t know details). Do note though that if you have an error, such as a kernel OOPS, then it is best to provide a copy of the full serial console boot log since hardware setup can occur as early as bootloader stages and the error itself is not very useful without knowing what lead up to the error.

I have switched to Jeston Nano now,
As like in xavier i am not seeing any segmentation falult

but in the dmesg i am seeing following error when clock is not discontinous in device tree

[ 382.706704] adv7282 7-0021: adv7180_mbus_fmt: w=720, h=576
[ 382.706708] adv7282 7-0021: adv7180_mbus_fmt: w=720, h=576
[ 382.827556] vi 54080000.vi: cil_settingtime was autocalculated
[ 382.827566] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[ 382.829220] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 0
[ 383.041201] video4linux video0: frame start syncpt timeout!0
[ 383.046948] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
[ 383.046953] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
[ 383.046957] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
[ 383.046997] vi 54080000.vi: cil_settingtime was autocalculated
[ 383.047001] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[ 383.055122] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 2
[ 383.075031] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 3

For a discontinous clock the following is the error

[ 99.187358] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[ 99.390078] video4linux video0: frame start syncpt timeout!0
[ 99.395759] video4linux video0: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
[ 99.395766] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
[ 99.395770] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000010
[ 99.395774] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00040041

Please expalin what is clock being discontinous ? i dont think my device supports such feature.

Kinldy help.

This is one I can’t answer since I don’t work with cameras.

actually, i have marked to @ShaneCCC.

Have a check the MIPI spec for the continuous clocks.

From datasheet of ADV7282 i can clearly see it wont support the feature of discontinous clock


so there is no point enabling discontinous clock in DT

so coming back again to the error posted above, i am seeing the registers throwing no error on jetson nano, but still i am seeing the channel error 4000

[ 382.706704] adv7282 7-0021: adv7180_mbus_fmt: w=720, h=576
[ 382.706708] adv7282 7-0021: adv7180_mbus_fmt: w=720, h=576
[ 382.827556] vi 54080000.vi: cil_settingtime was autocalculated
[ 382.827566] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[ 382.829220] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 0
[ 383.041201] video4linux video0: frame start syncpt timeout!0
[ 383.046948] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
[ 383.046953] vi 54080000.vi: TEGRA_CSI_CIL_STATUS 0x00000000
[ 383.046957] vi 54080000.vi: TEGRA_CSI_CILX_STATUS 0x00000000
[ 383.046997] vi 54080000.vi: cil_settingtime was autocalculated
[ 383.047001] vi 54080000.vi: csi clock settle time: 13, cil settle time: 10
[ 383.055122] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 2
[ 383.075031] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 3

kindly help

The discontinous in device tree only configure the host(Nano) you need to configure the device too.
It could be CSI didn’t receive any thing to report the error state.

you meant to say that the IC did not throw any kind of data ? over MIPI Lines ?
i am very much sure the mipi lines are throwing data, any way i shall come back with the probe shots

I mean It could be the output signal have problem get validate data from CSI.

One thing i observed is my MIPI Clock itself is not availble, i asked analog if i did any mistake with the driver, they responded as below

*As part of the MIPI specification the MIPI receiver needs to terminate the signals correctly. The termination required changes depending on the MIPI mode (e.g. high speed, low power mode etc). The receiver needs to detect the mode of operation and dynamically set its termination accordingly.*

does the MIPI Rx of Nvidia wait for transition from HS to LS Modes ?

in which part of the driver i can check if anything is going wrong ?

That’s should be the logic behavior instead of configure by any REG.

Data after launching video capture / cheese


Clock after launching video capture / cheese

clock is kept trigger and very primary sequence of pulses seens over clock and data just after launching video capture command(cheese), is it as per spec ?

green is data, clock is yellow

Hi, from hardware side, is the scope bandwidth enough for MIPI test? Have you checked the signals based on MIPI spec?