Can someone clarify what is the recommended approach for adding additional camera support to TX2 NX besides the few IMX sensors that are built in the kernel? Specifically, should I use LKM, kernel overlay with or without jetson-io tool, full kernel build from source, patch files, etc.? I see many different approaches on the forums and documentation but no tutorials from beginning to end and I am missing something. This seems like a topic that would benefit many people as cameras are constantly changing and computer vision is fundamental to the reason the Jetson platform is so popular.
Carrier board: Seeed A203v2 Module: Jetson TX2 NX Jetpack: 4.6 (32.6.1) Host pc: Linux 18.04 MIPI CSI Camera: e-consystems e-CAM131_CUNX with AR1335 sensor or Arducam IMX477
For reference, I have successfully flashed the board (with custom .dtb kernel file from Seeed) and tested an IMX219 camera without issue. I built a custom .dts file for the AR1335 and I thought I added it to the device tree but the camera was not recognized after following the Kernel Customization and Camera Development documentation. Apparently I missed a step. So can someone explain what to do after I create the .dtsi file that looks like the example " IMX185 V4L2 sensor driver" in the āSensor Software Driver Programmingā documentation?
@ShaneCCC, first of all, I appreciate your prompt response but admittedly, I hoped for some more insight. I did follow the documentation as far as I can tellā¦ but there are a lot of details missing and it didnāt work for me. Maybe some more details on my methods would help uncover the issue. At the very least, I just want to know the ābest practiceā for adding a new camera on a TX2 NX.
These are the steps I followed directly from the āSensor Software Driver Programmingā documentation:
Create a tegra-camera-platform device tree node in the kernel source tree (.dtsi file)
Verify port binding with sudo media-ctl -p -d /dev/media0 (what? it isnāt flashed yetā¦not sure how to do this step hereā¦)
Loadable Kernel Module (LKM). Oh this sounds fantastic! But how do I do it? Docs say by placing CONFIG_VIDEO =m in kernel configuration file (I assume this is source/kernel/kernel-4.9/arch/arm/configs/tegra_defconfig). Okay did thatā¦ now what?
Rebuild the kernel and the sensor driver (againā¦ how? build the entire kernel from source?). I built the kernel source how the other documentation says but ar1335.ko is nowhere to be foundā¦ that is too bad. This method sounded promising for debugging.
Device registration. Should I use plugin manager, Jetson-IO tool, or Main Platform Device Tree File for TX2 NX? I have read that Jetson-IO tool only works with certain platforms and plugin manager is not the preferred method anymore. I tried to create a .dtbo file but my include statements gave errors. That left me with Main Platform Device Tree File method. So I changed the .dtsi file to include my camera .dtsi file and comment out camera-plugin-manager.dtsi
Thatās it for instruction in the docs. So I assume I need to build the kernel and flash the jetson for any of these .dtsi and config file changes to take effect? I tried but the jetson image and device tree appears unchanged.
I have since tried using JP4.6.2 (32.7.2) with no success either. This is how I built the kernel and flashed:
Downloaded and untarred public_sources.tbz2 (also ran alternative method of ./source_sync.sh -k tegra-l4t-r32.7.1 to get sources folder)
Unpacked kernel_src.tbz2 to get hardware/ and kernel/
Made modifications as follows (I also did the same mods by creating a patch file if you want me to share that)
added tegra186-camera-ar1335.dtsi and tegra186-camera-lanai-ar1335.dtsi to sources/hardware/nvidia/platform/t18x/lanai/kernel-dts/common
I did not originally add ar1335.c to Kconfig. After adding this, I successfully ran these commands: make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT tegra_defconfig make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT menuconfig
which allowed me to select the ar1335 under Device Drivers -><*> Multimedia support ->NVIDIA overlay Encoders, decoders, sensors and other helper chips -><*M> AR1335 camera sensor support. For the sake of documentation, this was followed by these 4 commands: make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j8 Image make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j8 dtbs make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j8 modules make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT modules_install INSTALL_MOD_PATH=$KERNEL_MODULES_OUT
I used the main platform device tree method and modified these files under source/hardware/nvidia/platform/t18x/lanai/kernel-dts/
tegra186-p3636-0002-p3509-0000-a01.dts and tegra186-p3636-0001-p3509-0000-a01.dts
I did replace dtb and Image. I verified based on the timestamp as well as copying the file and running dtc -I dtb -O dts -o modified.dts tegra186-p3636-0001-p3509-0000-a01.dtb to check the source code.
Another error on my part was using the ā-rā option during flash. This was reusing the old image. After using this command: sudo ./flash.sh -k kernel-dtb -d kernel/dtb/tegra186-p3636-0001-p3509-0000-a01.dtb jetson-xavier-nx-devkit-tx2-nx nvme0n1p1, I was able to properly flash the Jetson TX2 NX. I verified by logging in to the Jetson and running dtc -I fs -O dts -o extracted.dts /proc/device-tree and searching for āar1335ā in the extracted.dts file and it looks correct. As recommended, I also created a LABEL backup in /boot/extlinux/extlinux.conf to boot from the backup image in case something goes wrong with my custom kernel.
I still canāt see /dev/video0 on the device but at least I am able to modify drivers and flash so it is probably an error in my .dtsi or driver files which I will explore next. At least I can debug now :)
Thanks again for taking the time to help me troubleshoot this.
I checked dmesg and unfortunately ar1335 wasnāt found. I noticed on my host pc that <top>/kernel/kernel-4.9/arch/arm64/configs/tegra_defconfig was missing CONFIG_VIDEO_AR1335=m so i added that and tried again. The /lib/modules/4.9.253/kernel/drivers/media/i2c/ folder does not contain ar1335.ko and no ar1335.o anywhere after CMAKE but it does have a few others, including imx477.ko. Is there a Makefile that needs to be manually edited somewhere?
Since I have the imx477, I tried to imx477.ko to /lib/modules/4.9.253-tegra/kernel/drivers/mdeia/i2c on the jetson but i get an error when running sudo insmod imx477.ko to do the LKM method. The error is: ācould not insert module imx477.ko: Invalid module formatā. Some of the other modules load just fine so must be an issue with that oneā¦ I was hoping to get lucky.
I get FATAL: Module ar1335 not found in directory /lib/modules/4.9.253-tegra when I run sudo modprobe ar1335.
I donāt get any errors when I run sudo modprobe imx477 but it still has the same error when I sudo insmod imx477.ko (I think that specific driver needs to be modified on my endā¦ I just grabbed the existing file to try)
I copied the /lib/modules/4.9.253 folder over from host to Jetson and renamed it to 4.9.253-tegra after renaming the original to 4.9.253-tegra-backup. Hopefully that is okay.
I still get the same modprobe error that module ar1335 is not found in the directory. I suspect the Makefile or something else is missing because it isnāt building the ar1335.ko or ar1335.o files.
I actually had to add obj-$(CONFIG_VIDEO_AR1335) += ar1335.o to the Makefile, not Kconfig in that folder but that definitely fixed some of my issues. I finally got some ar1335.c and .h file errors (meaning it was building the driver) after adding that to the Makefile and running make -C kernel/kernel-4.9/ ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=${CROSS_COMPILE} -j8 modules. I followed this with the rest of the build, flash, and moved .dtb, Image, and lib/modules over to the Jetson.
Now when I run dmesg | grep -i ar1335 on the jetson, I get the error āar1335: disagrees about version of symbol module_layoutā and sudo modprobe ar1335 gives me āERROR: could not insert āar1335ā: Exec format errorā. I suspect I need to start over and run make clean and do a fresh build on my folders in case some older kernel dependencies / files are still lingeringā¦ I will work on that over the weekend and report back. Thanks for your continued guidance @ShaneCCC.
Okay, I think this is mostly solved at this point. I still donāt have a working camera but it is recognized and loading the driver. My previous errors āExec format errorā were due to a JetPack version mismatch. I saw this error in several places with dmesg. I originally flashed JP4.6 (32.6.1) but my last kernel build from source was JP4.6.2 (32.7.2). After creating a clean folder on host PC and rebuilding from source using 32.6.1 files, I was able to recognize the cameras with my custom kernel. Now my error is related to driver issues, which I expected because I copied a lot from drivers for other jetpack/carrier-board combinations.
After digging deep into the driver source code, my previous error is related to the driver. The āpwdn_gpiosā which are named in the driver did not exist in my .dtb file and I donāt have enough info from the vendor to fix that. I found that the ar1335 driver utilizes the tca6424 chip which I havenāt been able to confirm exists on the Seeed A203v2 carrier board. Without more documentation from either e-consystems or Seeed, I am not going to pursue the a1335 camera on this TX2 NX ($160 thrown away + labor). It is a shame because it worked well on my Jetson Nano.
I moved on to the imx477 which has imx477.h and imx477.c files provided by Nvidia and the driver is much simpler. However after building, modifying, flashing several ways with several tweaks, I can not get the i2c to detect the camera. Running i2cdetect -y -r 2 shows only 0x40 is detected. The errors when using modprobe, dmesg, and insmod indicate the device is busy or already loaded. I suspect the gpio pins to turn on the device are not configured correctly but after several days of messing around, I have to move on. I have reverted back to the poor quality imx219 that at least gives me an image so I can work on my computer vision application and revisit this mess another dayā¦ I have to say I am quite disappointed.
I would use jetson-io but this is a non-dev carrier board which requires a custom overlay. There is no CSI Connector config option by default. Maybe that is what you mean; I need to create the overlay file? Is there an example for a camera overlay anywhere? All the existing overlays are for gpio, audio, etc., not a camera. I attempted to quickly create an imx477.dtbo overlay myself and apply it but the jetson-io tool gave me an error āFailed to overlay /mnt/APP/boot/dtb/kernel_tegra186-p3636-0001-p3509-0000-a01.dtb with /boot/imx477.dtbo!ā I have never made an overlay file so I am sure there are syntax errors in my overlay file.
There isnāt native support that I can find for the imx477 on TX2 NX (tegra186-p3636). I was able to find some camera examples such as ātegra194-p3668-all-p3509-0000-camera-imx477-imx219.dtboā in the /boot directory so I am going to use that and try to build one for the TX2 NX board.