Adv7282 PAL to MIPI CSI Linux Driver Development

changing PAL format size did not prove fruitful.
can you sense any mistakes from the v4l2 info ?

Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, adv7282 10-0021
	Bus info      :
	Driver version: 4.9.201
	Capabilities  : 0x84200001
		Video Capture
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 2: ok)
DV timings:
	Active width: 0
	Active height: 0
	Total width: 0
	Total height: 0
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 0 Hz
	Horizontal frontporch: 0
	Horizontal sync: 0
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 0
	Vertical backporch: 0
Format Video Capture:
	Width/Height      : 720/576
	Pixel Format      : 'UYVY'
	Field             : Alternating
	Bytes per Line    : 1440
	Size Image        : 829440
	Colorspace        : SMPTE 170M
	Transfer Function : Default (maps to Rec. 709)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Limited Range)
	Flags             : 

User Controls

                     brightness 0x00980900 (int)    : min=-128 max=127 step=1 default=0 value=0 flags=slider
                       contrast 0x00980901 (int)    : min=0 max=255 step=1 default=128 value=128 flags=slider
                     saturation 0x00980902 (int)    : min=0 max=255 step=1 default=128 value=128 flags=slider
                            hue 0x00980903 (int)    : min=-127 max=128 step=1 default=0 value=0 flags=slider
                 fast_switching 0x00981970 (bool)   : default=0 value=0

Camera Controls

           sensor_configuration 0x009a2032 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
         sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
      sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=30 flags=read-only

Modify your driver to report less lines one by one and check the log until it disappear. like 574, 572, …

Width/Height      : 720/576

Tried going up to 570, even tried 640x 480
Attached are the logs of 570, can you let me know which portion of the log gives you the hint that the resolution is creating issue ?
trace_log720570.gz (242.8 KB)

The tag “CHANSEL_SHORT_FRAME” tell the short frame.
And when you report the width as 640 get additional “CHANSEL_FAULT” tell receive more pixel than 640.

Have a check the bring up wiki.

this particular log attached is not 640x480 its just 720x570 only, i assume the log is a bit misleading from actual issue.

in the device tree parameters we have the sensor size, can it cause any issue here ?

	/* Physical dimensions of sensor */
                physical_w = "4.713";
                physical_h = "3.494";

Doesn’t matter with this. It about the sensor configure.

I double went through the code , device tree and the trace log, i dont find much problmatic any where,
why do you thing this short frame error keeps coming, kindly help me in this regard

When NVCSI/VI receive the pixel or lines less as driver report will show this message.

how can i know what is expected/how much resolution is expected by NVCSI/VI ? i am tweaking the resolution left and right by hardcoding, still no resolution,

The resolution that the sensor driver report.

Like below.

        Index       : 0
        Type        : Video Capture
        Pixel Format: 'BG10'
        Name        : 10-bit Bayer BGBG/GRGR
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1458
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.008s (120.000 fps)


I have few doutbs

  1. for command
    v4l2-ctl --list-formats-ext

the response i see is as below ,only thing is i could not find Size parameter output in it, anyways info all command throw the set resolution anyway as posted previously.

Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2

  1. For format of 720x572 i was not seeing short frame error or long frame error but was seeing CHANSEL_FAULT, which mainly occurs due to mismatch between input resolution in DTB and driver(understanding from other forum topics)
    why is it happening here i am not able to trace exactly in my case, where i am hardcoding the resolution 720x572

trace log attached again
trace.log (25.7 KB)

  1. has it got anything to do with stride alignment as raised in some other questions on forum ?

This log show the pixel run away.
PIXEL_RUNAWAY means VI receives more lines than expected (frame too high)
You may need to trace the driver for the resolution report.

CHANSEL_FAULT data bit field
    bit 16:31  LINE_NUMBER
    bit 13	EMBED_LONG_LINE
    bit 11	EMBED_RUNAWAY
    bit 3	EMBED_EOF
    bit 2	EMBED_SOF
    bit 1	PIXEL_EOF
    bit 0	PIXEL_SOF

Correct, if the v4l2 driver and architecture has a difference in line width the PIXEL RUNAWAY issue may occur, as per my understanding the issue in the most forums is resolved by changing device tree width and height

In my approach there is not width and height options in device tree

the driver is a functional release after resolution with Jetpack 4.2 in below thread.
ADV7280m configuration - Jetson & Embedded Systems / Jetson TX2 - NVIDIA Developer Forums

Jetpack 4.2 and 4.3 does they have any wide difference when it comes to v4l2 implementations ??

Suppose J4.2 and J4.3 shouldn’t have huge different. I would suggest to trace the driver to knowing more for it.

I am verifying step by step, first i have modified the device tree for single channel single lane mipi csi for which i dont have detection problems, but one clarity i am missing is the tegra camera platform parameters, and patch requirement

  1. can you please clarify is there any mistake in below parameters for resolutions 720x576 for frame rate of 24fps

     num_csi_lanes = <1>;
     max_lane_speed = <1500000>;
     min_bits_per_pixel = <10>;
     vi_peak_byte_per_pixel = <2>;
     vi_bw_margin_pct = <25>;
     max_pixel_rate = <240000>;
     isp_peak_byte_per_pixel = <5>;
     isp_bw_margin_pct = <25>;
  2. in other forum you have suggested a patch , is it required ?
    ADV7280m configuration - Jetson & Embedded Systems / Jetson TX2 - NVIDIA Developer Forums

  1. No any mistake.
  2. Those patch only for TX2. Similar patch for NX in below link.
    IMX219 Raw8 mode on the XavierNX: Disabling CRC - #9 by ShaneCCC
    Problem with capturing images by Video4Linux - #13 by ShaneCCC

Irrespective of your patch i could not avoid the short frame error which is occuring

after apply the patch i am seeing a segmentation fault and trace cut from dmesg is below, i assume i applied it with out any errors, may be if the segfault is removed i may over come the issue, kindly help

[ 12.808819] ------------[ cut here ]------------
[ 12.809040] WARNING: CPU: 1 PID: 6226 at /dvs/git/dirty/git-master_linux/kernel/kernel-4.9/net/wireless/nl80211.c:14141 cfg80211_ch_switch_notify+0x150/0x170 [cfg80211]
[ 12.809286] Modules linked in: zram overlay bnep rtk_btusb btusb btrtl btbcm btintel rtl8822ce cfg80211 spidev userspace_alert nvgpu bluedroid_pm ip_tables x_tables

[ 12.809366] CPU: 1 PID: 6226 Comm: RTW_CMD_THREAD Not tainted 4.9.201-tegra #56
[ 12.809372] Hardware name: NVIDIA Jetson Xavier NX Developer Kit (DT)
[ 12.809378] task: ffffffc1cff19c00 task.stack: ffffffc1cfd98000
[ 12.809461] PC is at cfg80211_ch_switch_notify+0x150/0x170 [cfg80211]
[ 12.810093] LR is at rtw_cfg80211_ch_switch_notify+0x94/0x98 [rtl8822ce]
[ 12.810100] pc : [<ffffff80011be080>] lr : [<ffffff8001304ac8>] pstate: 60c00045
[ 12.810104] sp : ffffffc1cfd9bc20
[ 12.810110] x29: ffffffc1cfd9bc20 x28: ffffff8001295760
[ 12.810123] x27: ffffff800be331bc x26: ffffff800145a578
[ 12.810135] x25: ffffff80014de13c x24: ffffffc1e2afa000
[ 12.810148] x23: ffffffc1d5b72000 x22: ffffffc1d5b83800
[ 12.810160] x21: 0000000000000000 x20: ffffffc1cfd9bca8
[ 12.810172] x19: ffffffc1de07bc00 x18: 0000000000000030
[ 12.810185] x17: 0000007f7c4e4d40 x16: ffffff80082753c8
[ 12.810196] x15: 0000000000000000 x14: 0000000000000000
[ 12.810208] x13: 000000000000ffff x12: 13192d1a2d000100
[ 12.810220] x11: 0000000000000004 x10: 0000000000000002
[ 12.810232] x9 : 0000000000000004 x8 : ffffffc1cfd9bd3a
[ 12.810244] x7 : 0000000000000000 x6 : 0000000000000001
[ 12.810256] x5 : 0000000000000040 x4 : 0000000000000000
[ 12.810268] x3 : 0000096c00000001 x2 : ffffffc1d5b84848
[ 12.810280] x1 : 0000096c00000001 x0 : 0000000000000000

[ 12.810297] ---[ end trace 1b49f253cf19e93f ]---
[ 12.810412] Call trace:
[ 12.810497] [<ffffff80011be080>] cfg80211_ch_switch_notify+0x150/0x170 [cfg80211]
[ 12.811094] [<ffffff8001304ac8>] rtw_cfg80211_ch_switch_notify+0x94/0x98 [rtl8822ce]
[ 12.811663] [<ffffff80012956c4>] rtw_chk_start_clnt_join+0x43c/0x4d8 [rtl8822ce]
[ 12.812285] [<ffffff8001295998>] join_cmd_hdl+0x238/0x330 [rtl8822ce]
[ 12.812872] [<ffffff8001252420>] rtw_cmd_thread+0x308/0x608 [rtl8822ce]
[ 12.812886] [<ffffff80080db07c>] kthread+0xec/0xf0
[ 12.812896] [<ffffff80080838a0>] ret_from_fork+0x10/0x30

I have no experience with cameras, but am hoping to clear up some concepts related to the topic which might speed things up for you. This means possibly answering about ideas instead of concrete examples, and not a direct answer, and often not even directly related to your question (I’m trying to be thorough, and really need a summary of what current questions are for any hope of a useful answer…this is a long thread).

Regarding the kernel and modules, understand that some features are invasive and were never designed to work as a loadable module. An example is virtual memory for user space programs…swapping out for cases of insufficient RAM. One cannot simply add or remove that feature since it affects just about everything, and so this has no option to be built as a module. Building as a module does have extra requirements, and so typically something like a driver for a specific device can be built as a module, but not always.

When a module or kernel driver does load which is for some optional hardware (such as a camera sensor), then the driver needs to find the device before it can be used. For plug-n-play devices (including USB and many PCI devices) this is not a problem since the device itself can self-report. For devices which cannot self-report the device tree becomes the primary method of giving such details to the driver. For example, the device tree will mention which driver(s) the device can be used with, along with a physical address to find the device (in the kernel physical addresses are used, not virtual addresses). Other details also can be added if the driver has an option for those arguments, e.g., I imagine that if the camera has more than one resolution, then probably a default starting resolution would be a candidate for the device tree.

If you want to see the current device tree simply extract it and browse the file (it is human readable). On a running Jetson you can extract its current device tree like this (if you don’t have “dtc”, then add this with “sudo apt-get install device-tree-compiler”):
dtc -I fs -O dts -o extracted.dts /proc/device-tree

Regarding how modules load, examine the output you see from “uname -r”. The answer will usually be something similar to “4.9.140-tegra”. In that output the “4.9.140” is the source code release of the kernel, and the suffix “-tegra” is from the CONFIG_LOCALVERSION (someone set this kernel config feature to “-tegra” before compiling). Now look at this file location:
/lib/modules/$(uname -r)/kernel
…you will see all modules which were built for this kernel in the directory tree at that location.

If a kernel has a module loaded which was built against this combination of kernel release version and the kernel was configured the same as your running system, then the module can simply be copied to the correct subdirectory of “/lib/modules/$(uname -r)/kernel”. Modules in any other location will fail to be found. Modules compiled against a significantly different kernel configuration will be found but will fail to load or else will produce a kernel error upon load.

Typically, if you build a kernel which starts with the existing kernel’s configuration and “CONFIG_LOCALVERSION”, and you merely add another feature as a module, then you won’t need to build a new kernel (you could build just the module). As soon as something is added to the kernel configuration which is not a module (especially something not able to be built as a module), then you risk the new module (and perhaps older modules) failing to work with the kernel. If the configuration changes enough to risk this, and if you replace the kernel itself (the “/boot/Image”), then you would be advised to use a new “uname -r” (which means a new “CONFIG_LOCALVERSION” other than “-tegra”, e.g., “-test”), followed by installing all new modules and the Image. Unless you use some new feature which is invasive and not available as a module this shouldn’t be needed.

Note that the device tree is likely consulted as you load such modules for items which are not plug-n-play. This implies that the device tree is specific to kernel features, but the tree itself is just an argument passed to the features/drivers. The tree can be built in the kernel, but it isn’t because the tree is part of the kernel. The tree is actually able to be reverse compiled, edited, recompiled, and added back in as a simple file without ever touching the kernel source. The device tree exists because without such a method of supplying arguments you would need a different kernel driver for every possible configuration of that driver, e.g., if one manufacturer were to put the camera at one physical address, but another at a different physical address, then you’d need a driver for every possible variation. The device tree makes it possible to have one driver and then each manufacturer tweaks the device tree details for their hardware. Note that physical address and many other details of the device tree are specific to a specific carrier board. A Jetson on one carrier board will require a tree which is different than that of another manufacturer’s carrier board.

So as you build a kernel you device what device tree nodes are required. As an option that fragments of source to a device tree can be glued together to form a full tree source based on the features you’ve enabled. Then the tree can be built by the kernel via building the “dtbs” target. Even that tree will need edits for anything which is non-default in your carrier board and/or end device.

You can edit any tree, and for example, the extracted.dts file mentioned from above can be edited with any text editor. Then it could be recompiled:
dtc -I dts -O dtb -o edited.dtb extracted.dts

To safely try another kernel and/or tree, consider the following (related to installing the new tree without overwriting the old content)…

Examine file “/boot/extlinux/extlinux.conf”. This is just from a TX2, so it isn’t an exact match for the NX, but a boot entry looks like this:

DEFAULT primary

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

If you boot with serial console attached, then there is a short moment when you can pick a kernel. If you don’t pick a kernel, then it will boot the DEFAULT (there are some bugs which some releases also require the DEFAULT to be the first entry). The example looks for LABELprimary” when nothing is chosen.

Assuming you have serial console and can interrupt boot and pick a kernel, then you could safely do the following and keep the defaults, but pick your test versions without risk by adding content as follows:

  • Add a new extlinux.conf entry, e.g., something like this:
LABEL test
      MENU LABEL test kernel
      LINUX /boot/Image-test
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

Since you are going to add a device tree based on a file (there are also ways to do this as a partition), the entry would be further modified with a DTB entry:

LABEL test
      MENU LABEL test kernel
      FDT /boot/dtb/edited.dtb
      LINUX /boot/Image-test
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4

Now, if you copied “edited.dtb” to “/boot/dtb/edited.dtb”, and you’ve picked the “test kernel” boot entry, expect your tree to be used without touching the original boot content. If it fails, then you simply pick the “primary kernel” entry. If that entry is still set as the default, then you don’t have to do anything to get the original boot.

Let’s extend this to also adding a very different kernel due to non-module changes which are invasive. Your extlinux.conf entry was already edited for the new kernel with name “Image-test” for the file, but you will also now need to install a complete set of modules built against that configuration at this location (just to emphasize, this is an example which assumes “uname -r” responds with “4.9.140-test”):

I don’t know anything about the camera, and have not worked on cameras before, but if you were to find an example entry for your carrier board, then it might just be a direct addition of that entry into your tree. If not, then perhaps you can build the kernel source “dtbs” target to get an example, and then edit the content which shows up for your camera (but tailored to your carrier board).

As a caution note on kernel building, even if you are only building a module, you must take steps to propagate a proper configuration throughout the source before building. If you start by building the kernel itself, the “Image” target, then this will all be taken care of and you won’t need to propagate configuration. If you don’t build Image first, then to build a module you must propagate the configuration with the “make modules_prepare” step (if you are building in a temp location, then you’d have to include options to that location, and if cross compiling, then you’d also have to have the cross compile options set).

I don’t know what your current situation is since there is a lot going on in this thread, and I am the wrong guy to ask about camera drivers, but if you summarize the current steps you want elaboration on, then I can probably give more exact answers (the above is purposely generalized since I don’t know details). Do note though that if you have an error, such as a kernel OOPS, then it is best to provide a copy of the full serial console boot log since hardware setup can occur as early as bootloader stages and the error itself is not very useful without knowing what lead up to the error.

I have switched to Jeston Nano now,
As like in xavier i am not seeing any segmentation falult

but in the dmesg i am seeing following error when clock is not discontinous in device tree

[ 382.706704] adv7282 7-0021: adv7180_mbus_fmt: w=720, h=576
[ 382.706708] adv7282 7-0021: adv7180_mbus_fmt: w=720, h=576
[ 382.827556] vi cil_settingtime was autocalculated
[ 382.827566] vi csi clock settle time: 13, cil settle time: 10
[ 382.829220] vi tegra_channel_error_status:error 4000 frame 0
[ 383.041201] video4linux video0: frame start syncpt timeout!0
[ 383.046948] vi TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
[ 383.046953] vi TEGRA_CSI_CIL_STATUS 0x00000000
[ 383.046957] vi TEGRA_CSI_CILX_STATUS 0x00000000
[ 383.046997] vi cil_settingtime was autocalculated
[ 383.047001] vi csi clock settle time: 13, cil settle time: 10
[ 383.055122] vi tegra_channel_error_status:error 4000 frame 2
[ 383.075031] vi tegra_channel_error_status:error 4000 frame 3

For a discontinous clock the following is the error

[ 99.187358] vi csi clock settle time: 13, cil settle time: 10
[ 99.390078] video4linux video0: frame start syncpt timeout!0
[ 99.395759] video4linux video0: TEGRA_VI_CSI_ERROR_STATUS 0x00000000
[ 99.395766] vi TEGRA_CSI_PIXEL_PARSER_STATUS 0x00000000
[ 99.395770] vi TEGRA_CSI_CIL_STATUS 0x00000010
[ 99.395774] vi TEGRA_CSI_CILX_STATUS 0x00040041

Please expalin what is clock being discontinous ? i dont think my device supports such feature.

Kinldy help.

This is one I can’t answer since I don’t work with cameras.