Raspberry Pi HQ Camera in Jetson Xavier NX

Hi everyone,

We are glad to announce the early-version of the IMX477 Raspberry Pi HQ Camera Driver for Jetson NX available in our Github repo: https://github.com/RidgeRun/NVIDIA-Jetson-IMX477-RPIV3

You can find documentation about how to build, install and use the driver.

The first version of the ISP config file created by Leopard Imaging was included, you can follow the instructions to use it in your setup.

Feel free to share your progress with the driver and if you have questions.

Note1: The initial conversation about this driver can be found here

2 Likes

Unfortunateley I was not able to get the driver to run . I have a Nvidia Xaveier NX developerr kit. I flashed it with the latest JetPack_4.4_DP_Linux_DP_JETSON_XAVIER_NX_DEVKIT. So quite a clean starting point to patch the kernel.

I was able to follow the complete patch and installation process without any visible error message besides the last intruction about moving the kernel modules from tmp to /lib/modules/ seems to be not exactly correct. The original image contains already a folder called 4.9.140-tegra . note tegra with lower case t. if i follow exactly your mv command sudo mv /tmp/4.9.140/* /lib/modules/4.9.140-Tegra/ I get 
 no directory error message. if i remove the ‘*’ the command works and it results in two folders in /lib/modules/ first ‘4.9.140-Tegra’ and second ‘4.9.140-tegra’. But boot hangs at some point and display blinks. There is quit at the begiining of boot process a ’ failed’ error message telling something similar to ‘kernel modules could not be started’. I also tried to remove the original kernel modul folder and moved the new one into a folder named exactly like the folder from original image. All variants I tried seems to be not able to start kernel modules. If I replace the patched image with the original one I can boot again (at least if the modules are in the folder 4.9.140-tegra with lower case t). Unfortunately I do not have any idear to bring it to work at moment. I hope someone can give me a hint in the right direction.

I’m quite sure the problem is with the kernel modules but do not find the right way to do it. Pherhaps someone who was successfull can copy paste me the exact path of the resulting folder in /lib/modules/. Should it be 4.9.140-tegra with lower case t like the original image or should it be 4.9.140-Tegra like in the instructions with upper case T?

Hi,

Like @m.kirsch, wasn’t successful in getting the driver to run, I have the same problem with the kernel modules.
I already contact the RidgeRun team by email to try to solve the problem.

Best Regards

Hello,

I have got this running, the module directory should be named “4.9.140”, in my case.

Not sure where I found it but the directory should have the same name as “uname -r” on the target.

/Fredrik

1 Like

Seems like that is the answer!

My colleague @tiago.f got it running after renaming the folder.

Thank you @Fredde

1 Like

How is the image quality ?

Today, my team found a situation copying the kernel modules since the uname -r command returns 4.9.140-tegra before updating the kernel files.

But after rebooting, the uname -r command returns 4.9.140. So we renamed the folder to the correct one (without the tegra prefix) and now everything is working.

So, updating the instruction to sudo mv /tmp/4.9.140/* /lib/modules/$(uname -r)/ would not be enough.

We had to copy the files, not moving, since the kernel/ folder was not empty.

Hi everybody.

Thank you for all your valuable feedback about the IMX477 driver guide. We really appreciate it.
It seems the solution found by @Fredde on the modules copy works as expected.

I have updated the guide to ease the driver installation for future interested users.

Stay safe,

JC

I noticed that the Jetpack in the instructions is the Developer Preview (DP). Has anyone been successful at patching the GA or released Jetpack 4.4. I get errors running “quilt”. Some files are patched and others seem to be missing.

can’t find file to patch at input line 3077
1 out of 1 hunk ignored
can’t find file to patch at input line 3092
16 out of 16 hunks ignored

Hi @rolinster.

We are currently modifying the repository structure in order to place the sources of Xavier NX and Nano in different directories. We will let you know once the changes are done on today afternoon.

The JetPack version shouldn’t be a problem since the patch are applied on the 32.4.3 sources required for both JP 4.4 DP and JP 4.4.

Regards,

JC

Hi, Firstly thanks for giving this a go for Jetson Nano and NX. I first had Nano then when I found out you actually support nx went and bought that however now I think the patch process is broken for both :(

There seem to be some files missing. Here is what I get when I try quilt push to patch:
Applying patch patches/4.4.3_l4t32.4.3_rbpv3_imx477_support.patch
patching file hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-camera-jakku-rbpcv3-imx477.dtsi
patching file hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-camera-rbpcv3-imx477.dtsi
patching file hardware/nvidia/platform/t19x/jakku/kernel-dts/common/tegra194-p3509-0000-a00.dtsi
patching file kernel/nvidia/drivers/media/i2c/Kconfig
patching file kernel/nvidia/drivers/media/i2c/Makefile
patching file kernel/nvidia/drivers/media/i2c/imx477.c
patching file kernel/nvidia/drivers/media/i2c/imx477_mode_tbls.h
patching file kernel/nvidia/include/media/imx477.h
patching file hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms/tegra210-camera-rbpcv3-dual-imx477.dtsi
patching file hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms/tegra210-camera-rbpcv3-imx477.dtsi
patching file hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms/tegra210-porg-camera-rbpcv3-dual-imx477.dtsi
patching file hardware/nvidia/platform/t210/porg/kernel-dts/porg-platforms/tegra210-porg-camera-rbpcv3-imx477.dtsi
can’t find file to patch at input line 3077
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:

|Index: sources/hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-porg-p3448-common.dtsi
|===================================================================
|— sources.orig/hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-porg-p3448-common.dtsi

+++ sources/hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-porg-p3448-common.dtsi
No file to patch. Skipping patch.
1 out of 1 hunk ignored
can’t find file to patch at input line 3092
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:

|Index: sources/hardware/nvidia/platform/t210/porg/kernel-dts/porg-plugin-manager/tegra210-porg-plugin-manager.dtsi
|===================================================================
|— sources.orig/hardware/nvidia/platform/t210/porg/kernel-dts/porg-plugin-manager/tegra210-porg-plugin-manager.dtsi

+++ sources/hardware/nvidia/platform/t210/porg/kernel-dts/porg-plugin-manager/tegra210-porg-plugin-manager.dtsi
No file to patch. Skipping patch.
16 out of 16 hunks ignored
Patch patches/4.4.3_l4t32.4.3_rbpv3_imx477_support.patch does not apply (enforce with -f)

Hi Sam, you can refer to my github issue here:
https://github.com/RidgeRun/NVIDIA-Jetson-IMX477-RPIV3/issues/6

Just be aware that I think there is a issue with the nano drivers still, have a look at this issue here before you start:
https://github.com/RidgeRun/NVIDIA-Jetson-IMX477-RPIV3/issues/5

But I think the Ridgerun guys are aware of it and working on it.
Cheers,
Peter.

Hi All,

We updated the instructions and the repository in order to have one folder for nano and one for NX to make the process easier and avoid some of the issues reported when applying the patch.

-David

Hi all,

We have uploaded a new version of the ISP configuration file. Please, check it out and let us know how it works for you.

Regards,

-JC

The ‘Copy the generated kernel and dtb’ part of this instruction is specific to Jetson NX. If you have a Jetson Nano you should copy a different dtb file.

Hi all.

We have updated the driver install instructions in order to provide an easiest way to add the IMX477 support.
From now on you are able to install the driver via debian packages with only a few commands.

This new instructions are only available for the Xavier NX, but we will update the guide with similar steps for the Jetson Nano as soon as possible.

Please, check the new section and let us know how it works for you.

Driver installation via Debian packages

Regards,

JC

2 Likes

Hi,

Is it just me having a flipped image using the driver?

/Fredrik

Hello JC,

First I’d like to thank you for your effort.

After removing R8 from the camera board and installing the driver packages, I can confirm that the HQ cam is working.

Problem is, that when using the JPEG conversion from gstreamer, the colors look quite awful - green tint, flat:

Also applying different white balance modes does not really help:

For comparison, this is how it looks, when the camera is attached to a Raspberry Pi and the image is saved with raspistill:

Will there be any option for better postprocessing?

1 Like

Hi @helge,

Did you follow the instructions to change the ISP configuration file? Instructions can be found here: GitHub - RidgeRun/NVIDIA-Jetson-IMX477-RPIV3: NVIDIA Jetson IMX477 HQ RPI V3 camera driver

After remove the resistor you were able to use it on RPi4?