Can't install modified camera driver

I’m trying to modify a camera driver (IMX477 in this case), I have modified sources and procedure for a different camera and that works fine (in JP4.4)
So I took kernel sources (JP4.6), found all the relevant files for IMX477, made changes I needed, followed the procedure …and nothing.
Tried a few different things and as of right now I can say that I can remove IMX477 drivers from available but the modified one (IMX478 in this case, it’s still477 but renamed through find-replace all) just never appears, loads or anything. Dmesg is empty of any camera drivers and all after I removed IMX219 and IMX477 from .config.

Here’s the procedure:
procedure.txt (1.4 KB)
Driver source:
JetsonNanoB01_IMX478_src.zip (35.2 KB)

I just don’t know what’s wrong with it… Help, suggestions on what to do or read are appreciated.

Just some comments, not necessarily a solution…

The procedure is fortunately backing up the original Image file. Unfortunately though, I don’t see anything adding an alternate boot entry for that backup copy. Do you have serial console available? You need this to pick among alternate boot entries. Since it still boots with a new Image you should add an entry in “/boot/extlinux/extlinux.conf” for the backup. Then boot to the backup and find the “uname -r”. In the new kernel it will likely respond with something similar to “4.9.140”, but booting to the original kernel would likely respond something like “4.9.140-tegra” (actual release is unimportant, but the “-tegra” is very important).

The default entry in extlinux.conf is likely something like this (use your own extlinux.conf for actual values):

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

If you change “LABEL” and “MENU LABEL”, then you can pick among these via serial console. Sometimes there is a bug as to whether the “DEFAULT” at the top of the file works, so I’ll just set it to use the original Image by changing this to these two entries:

# Notice this now names "Image.org":
LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image.org
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

# Notice this changes labels and names "Image":
LABEL new
      MENU LABEL new kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

The importance is that modules are searched for here:
/lib/modules/$(uname -r)/kernel

The output of “uname -r” is built into the Image file at compile time. Unless you set CONFIG_LOCALVERSION only the 4.9.140 part will participate in “uname -r”. I don’t think you should be installing the Image file itself and should stick to the original Image file and original kernel modules. To do this (if you choose to use the new Image) you would edit the “.config” like this (there are other ways to do this, but the script did not use any of those):
CONFIG_LOCALVERSION="-tegra"

If you have this in your build of a 4.9.140 source, then the “uname -r” will respond with “4.9.140-tegra”, and the only module you will need to install is the new camera module. The second boot entry will allow you to pick and choose though via serial console during boot if you interrupt with a keystroke at the right time.

I’m not sure about some of the other content, e.g., the copy of the .dtb files. You could save all of the originals before overwriting something, but unless there is an “FDT” key/value entry in your extlinux.conf, then this won’t do anything (instead the device tree in the partition would be used). What is in your “extlinux.conf”? The step where it uses flash.sh to install the DTB partition would change this, but it is better to have a copy of the original device tree files to use with the backup image.

This is worth fixing because you need to know which kernel modules are used, and you need to know which location contains those modules. The module won’t be able to load if this is not correct.

It would be best to post this content:

  • The extlinux.conf (after adding the alternate entry).
  • The “uname -r” output for each Image, both new and original.
  • The exact name of the file which is your kernel camera module (something “.ko”).

Since I already cooked that installation I re flashed the card and started from scratch. Same procedure but in config imx478 is set to compile as a module. I also copied original Image as Image.backup

extlinux.conf after adding alternative entry. (omitting the commented out backup part of the file)

TIMEOUT 30
DEFAULT primary

MENU TITLE L4T boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      INITRD /boot/initrd
      APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0



LABEL stock
      MENU LABEL stock kernel
      LINUX /boot/Image.backup
      INITRD /boot/initrd
      APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

For original image “ uname -r ” outputs 4.9.253-tegra
For the new image “ uname -r ” outputs 4.9.253

Module name is imx478.ko as confirmed by

nvidia@nvidia-desktop:/lib/modules/4.9.253/kernel/drivers/media/i2c$ ls
imx185.ko  imx185_v1.ko  imx274.ko  imx478.ko  ov5693.ko  tc358840.ko

[I was trying out stuff to see if anything can be changed]
This attempt didn’t yeld a working anything but I should probably explain why I thought the original procedure should work.

I got a driver source with the procedure just like the ones I attached in the original post. It was for JP4.4 and I figured this procedure with minor modifications would work for JP4.6. I downloaded kernel sources through wget as per original procedure but with a modified link. I can’t find these sources on embedded downloads but they seem to work just fine so far.
And that procedure was sufficient to get the driver working in under a couple of hours. But it doesn’t seem to be the case for JP4.6. And I have no prior experience of building drivers, modifying kernel or anything… I suppose I should have some to fully understand the above reply but oh well… (yes, just dumbly copy pasting commands from the file. At uni {software engineering course} we study how to make chess in MS Excel spreadsheets…)

The most recent change I did is I also flashed .dtbo's and that gave some results that may mean that I’m close to get it working but what do I know.
This time I can choose 478 as an option in Jetson IO

myfirstgimpimage

But it cannot be applies due to failed overlay

second gimp image

I suspect that it’s because I left imx219 and imx477 built in to the kernel and since my 478 is technically 477 with “find - replace all 477->478” they cannot be applied at the same time. I’m not sure if that’s how it works but I can try again, like I did probably 10 or more times before.

If anything there gave any clue what’s wrong I’d be happy to hear. for now I’m probably gonna compile all this stuff again with 219 and 477 removed out of the kernel and 478 as a module and will update if anything comes out of it.

In config I set IMX478 to compile as module and everything else to not compile.(dts's still left in, so they were compiled anyway) The result is following: no IMX219 or IMX477 drivers loaded, no traces of that or any other camera driver in dmesg.
Picture in Jetson IO is the same as the last time and it also can’t overlay like the last time. Which makes very little sense to me because IMX477 Dual does overlay without any problems though it’s identical to the IMX478 Dual one except every imx477 is replaced with imx478 in the latter.
I’m kinda lost here.

There is a lot to sort, so this isn’t necessarily in any particular order…

The extlinux.conf looks good. Hopefully you have tested the ability to interrupt boot with serial console and pick either the default/first/modified entry, and the second entry.

Because you’ve only added a module you should not actually need a second entry. Your new entry will also require 100% new module content, which is just adding work since the base Image is the same (other than your new Image no longer being able to find modules…uname -r changed).

Is there a reason why in your new Image (which is not needed) that you did not set “CONFIG_LOCALVERSION” to “-tegra”? Admittedly this makes the new and old Image exact matches, so a new Image is not needed, but if you were to want two separate Image files, then you might want to share the module location anyway (you are only adding one module to the original configuration).

I have noticed that a large number of IMX camera drivers are built in to the Image by default:

# zcat /proc/config.gz | egrep -i '(imx)'
# CONFIG_TOUCHSCREEN_IMX6UL_TSC is not set
CONFIG_VIDEO_IMX185=y
CONFIG_VIDEO_IMX219=y
CONFIG_VIDEO_IMX268=y
CONFIG_VIDEO_IMX274=y
CONFIG_VIDEO_IMX318=y
CONFIG_VIDEO_IMX390=y
CONFIG_VIDEO_IMX204=y
# CONFIG_SND_SOC_IMX_AUDMUX is not set

Note that if your driver is already in place (as integrated “=y”), then you would never see a module, nor would you ever load the module (the driver would always be there since it is integrated). The above is from a bit older Jetson, so there might be other added drivers available in your release. I have to emphasize that I do not know the difference between an IMX478 or any other IMX driver (I don’t work on cameras). However, what do you mean that no driver loaded? I ask because finding a module loaded has nothing to do with whether the driver loads if the driver is not in the format of a module.

Another point to emphasize: Which interface are you using for the camera? An interface which is “plug-n-play” (one which can self-report its hardware capabilities) will automatically load when detected, but an interface which simply depends on passing the bus address to the driver during driver load requires a device tree entry (which is that information to be passed to the driver at boot).

I will suggest that you are closer, but on both the original Image and new Image, if you alternate and boot to each, run this command and compare the two:
zcat /proc/config.gz | egrep 'IMX'

When the Jetson is running the “config.gz” is a reflection of the running kernel’s configuration during compile. If the only difference is from modules, then you don’t need to install a new Image. You could simply copy the module to the module location for the original Image and it would be done (not all features can be built as a module, but likely your camera has no issue with being a module). Double check that your module file (and all original modules) exist at:
/lib/modules/$(uname -r)/kernel

Note that if you were to compile the driver as “integrated” (rather than as a module), then this would be a valid reason to install a new Image file, but I wish to emphasize that if CONFIG_LOCALVERSION had remained as “-tegra” on both old and new, then you would not have had to install any modules other than the single module you just built. I will advise doing this in the case of only adding new modules (versus altering integrated content with a significant change).

I admit I didn’t, mostly because I don’t have USB to serial converters handy. But in case I’d need I’ll pull an Arduino apart and use it as one.

I disabled other IMX drivers to make sure that if anything works, then it’s most definitely mine. So Image's are not identical

By default, on boot IMX219 (or IMX477 if it’s set) tries to probe for the sensor (can be seen in dmesg). If it didn’t even try, that should mean that it’s not active/loaded. At least I think so.

I’m using CSI connector on the board. As far as I know, it can’t self-report its hardware. .dts and .dtsi files in the driver source I attached are renamed and modified. I’m not sure if my modifications are sufficient because Jetson IO won’t overlay them. I tried modifying IMX477 driver and its device tree files without renaming anything but nothing changed (I’m trying to bump max available framerate
but it always reported 1080p@60 is maximum). However, I’m not sure if I ever reapplied the device tree overlay on that one.

Old:

nvidia@nvidia-desktop:~$ zcat /proc/config.gz | egrep 'IMX'
# CONFIG_TOUCHSCREEN_IMX6UL_TSC is not set
CONFIG_VIDEO_IMX185=m
CONFIG_VIDEO_IMX477=y
CONFIG_VIDEO_IMX219=y
CONFIG_VIDEO_IMX268=y
CONFIG_VIDEO_IMX274=m
CONFIG_VIDEO_IMX318=y
CONFIG_VIDEO_IMX390=y
CONFIG_VIDEO_IMX204=y
# CONFIG_SND_SOC_IMX_AUDMUX is not set

New:

nvidia@nvidia-desktop:~$ zcat /proc/config.gz | egrep 'IMX'
# CONFIG_TOUCHSCREEN_IMX6UL_TSC is not set
# CONFIG_VIDEO_IMX185 is not set
CONFIG_VIDEO_IMX478=m
# CONFIG_VIDEO_IMX219 is not set
# CONFIG_VIDEO_IMX268 is not set
# CONFIG_VIDEO_IMX274 is not set
# CONFIG_VIDEO_IMX318 is not set
# CONFIG_VIDEO_IMX390 is not set
# CONFIG_VIDEO_IMX204 is not set
# CONFIG_SND_SOC_IMX_AUDMUX is not set

About what I would expect.

Honestly, for now, it’s unclear to me what should I do next.

I may try again just the modified IMX477 without renaming and see if that works.

In the case of removing a driver which was previously integrated into the kernel, then you do need to install the actual Image file. You would also want a different “uname -r” (the “CONFIG_LOCALVERSION” should differ from “-tegra”) and full module install into the separate module location. Having no “CONFIG_LOCALVERSION” does qualify, though I’d probably use something like “-noimx”. Just make sure you’ve built and installed all modules to the new “/lib/modules/$(uname -r)/kernel” load location.

You are correct that CSI will use device tree content to find the camera. That’s one I can’t help with. Not sure what changes would be needed. There is a bring-up guide somewhere which would probably help.

Not necessarily useful, but I see this as the diff to the two config.gz IMX items:

  • Old has IMX477=y, but new uses “IMX478=m”. Note that the number changed from 477 to 478. I have no knowledge of their differences, but in this case you could indeed load/unload the IMX478 since it is now a module.
  • All other drivers are not enabled, and likely what you want.

Did you really want to change from “IMX477” to “IMX478”? If not, perhaps this is a typographic error. If it is, then perhaps you wanted the new name for whatever reason, but any device tree would need to know the correct driver.

Just to make sure that it’s my driver but eh… it doesn’t have to be renamed.

I poked around kernel source a bit more and found that I forgot to include my new .dts .dtsi files if a few places and since I left original IMX477 device tree files in place, it never error’ed out but also never included IMX478

This time I only made changes in mode table files,setting max allowable framerate to 120 and changed the same thing in .dtsi's

min_framerate = "2000000"; /* 2.0 fps */
max_framerate = "120000000"; /* 120.0 fps */

Image compiled, modules installed, dtb flashed… just in case

And it seems to work…
Except nvarguscamerasrc says

GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 2160 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 22,250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59,999999 fps Duration = 16666667 ; Analog Gain range min 1,000000, max 22,250000; Exposure Range min 13000, max 683709000;

and refuses to work at anything above 60fps (e.g. 61) like nothing has ever changed (similar to my previous attempts at changing the absolute minimum)
But v4l2-ctl reports that something has changed

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB
		Size: Discrete 3840x2160
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1920x1080
			Interval: Discrete 0.008s (120.000 fps)

Which may suggest that I forgot to change something somewhere else…

I should probably go with 478 source again, but leave 477 in place and being extra thorough with including my source where needed.

Other than that I don’t know what to do because all I wanted to do is to change a few values to let me run sensor at higher frame rates (I compared sensor mode settings with the sensor that does run at 1080p 120 and see no reason why it can’t). But it looks like it’s not as simple as that.

Before the question is asked:
“If you have a sensor that does run with the resolution and framerate you want and you have working drivers for it, why wouldn’t you use it?”
It has an optical defect that can’t be fixed because it’s a feature.

Just comments on the kernel driver side which now works…

If there is a rename of a driver, or a copy with a new name, then I’d be suspicious of the new name (or rename) not being correct in the kernel source. Configuration via Kconfig is perhaps not simple at times (you would have to carefully see what is being built and where the files go in the kernel source during a build to verify, and then verify that as a module the file actually loads with that name.

The .dts and .dtsi files only matter if you use the device tree from the kernel build. But yes, you would need new content under the new driver name (possibly it would just be a copy of the old tree node for that driver, but if the name is wrong, then the content will be missing).

And yes, so long as there is no other conflict, then you should in theory be able to have both the 477 driver and the 478 driver available. If you want to name the camera as available under both drivers (but this does not necessarily pick which one loads), then the “compatible” entry of content for the camera would name both drivers. At that point it would be possible to load the camera with either one. If the compatible entry names just one, then only that driver will load for the camera. Imagine if your extlinux.conf has boot entries which differ only by which device tree loads, and one states only the 477 driver as compatible, the other boot choice dtb picks only the 478 driver as compatible, and perhaps some default entry in extlinux.conf could use another dtb which says both drivers are compatible. It makes for easier debugging and development.


There isn’t much I can say about nvarguscamerasrc since I don’t work with this. Note that the IOCTL is performed by the driver, and if the driver accepts a change, then you are at least part way there. If nvarguscamerasrc still fails, the perhaps it is a limitation of the device tree, or perhaps the driver still needs further modification. I don’t know, and don’t really have a way to speculate further on it.

So far as extended frame rates go I don’t know, I don’t really work much with cameras. One possibility would be some other hardware (e.g., a CODEC) has a limitation. Someone from NVIDIA could probably comment on what hardware is limiting in the use of the nvarguscamerasrc code.

1 Like

I redid all the changes today twice and got the same thing. Wasn’t too down because it never worked before but since I knew I changed everything I started looking for anything else. It never occurred that the DTB that the overlay is applied to was never replaced with newly compiled (Where I included my dts/dtsi).
I replaced it and it applied right away.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.