I am currently trying to get both the IMX477 and IMX219 based Raspberry Pi cameras working with the Auvidea J20 board. (Note that I am not trying to get the IMX477 and IMX219 working simultaneously, but rather I am trying to get separate drivers for the IMX477 and IMX219 working.) Ridgerun has developed drivers for the J20 + IMX477 on AGX Orin, however, I cannot get these drivers to work properly. Without the ISP file in place, these drivers work to produce images of poor quality. However, as soon as I use the ISP file, something breaks and I cannot get any images from the camera. Attached is the gstreamer error I see when trying to open the camera with the ISP file in place. I have also attached the results of running V4l2-compliance with the ISP file in place, and with the ISP file removed.
Interestingly, the drivers produced by Auvidea for the IMX219 + J20 fail similarly. For the IMX219, the drivers fail with the same gstreamer timeout error that the IMX477 fails with. However, the IMX219 also fails without the ISP file.
I am hoping that someone has some insight as to what could be causing this failure, and could point me in the right direction to solve it and get the cameras working. It is very important that I can get either the IMX219 or IMX477 (ideally both) working with the J20 board on AGX Orin. If you require more information, I am more than happy to provide it!
first of all, rbpcv2-imx219 and rbpcv3-imx477 are not supported on AGX Orin by default.
you may port the sensor device tree from Orin NX series, or, it’s done by vendor’s driver package.
please note that, there’s lane_polarity configuration, which only necessary for Orin NX series, you may setting them as 0 when bring-up those on AGX Orin.
if the basic camera functionality works, please setup a terminal to enable Argus daemon LOGs,
please gather the details when trying to open the camera with the ISP file in place.
for example,
$ sudo su
# pkill nvargus-daemon
# export enableCamScfLogs=5
# nvargus-daemon
or.. # nvargus-daemon 2>&1 | tee argus-output.log
The drivers for the IMX477 on the J20 board have been developed by RidgeRun and are publicly available from this Github repo. The driver works without the included ISP file in place, but fails as soon as the ISP file is used.
I have attached three log files from the nvargus-daemon to this post; one without using the ISP file, one with the ISP file, and one with the ISP file and nvargus-deamon launched with the enableCamInfiniteTimeout=1 flag. I see various errors and warnings in these log files, but I lack experience debugging ISP/driver issues, and thus I don’t fully understand the errors. To my untrained eye, it looks like the ISP is having issues mapping the module to an ISP config string, causing the V4l2 device to become unavailable, and the camera to be unopenable. If you could take a look at the logs and provide me with your thoughts, I would greatly appreciate it. If you require any more information, I am more than happy to provide it. Thanks!
the ISP file contain the tuning parameter settings to fine-tune the image quality.
according to error logs, argus-log-IMX477-with-ISP.log
it shows errors related to Sensor could not be opened, and then follow-by timeout errors which caused camera app malfunction.
there’s an unique identifier to distinguish the ISP files.
could you please double check the sensor device tree settings for the badge property.
or… you may review the ISP files, to modify the string of Part# for testing.
anyways,
where did you get the ISP file, who shared this ISP file. is this ever validated?
note, you cannot apply the same ISP to different platforms, or different BSPs.
Thank you for your insight on this issue. Looking at the device tree settings, I see that the badge property for the three cameras are set to imx477_bottomleft_477IMX, imx477_centerleft_477IMX, and imx477_topleft_477IMX. As there are only three cameras in use, should these names be changed to imx477_bottom_477IMX, imx477_top_477IMX, and imx477_center_477IMX? Could these names being incorrect be the reason for my issue, or are they just convention?
As for the camera_overrides.isp file, can you tell me what I should be changing the Part# string to? Looking at working ISP files I have for other cameras isn’t providing much insight as they all have an empty string for the part number.
I obtained the ISP file from the same repository as the driver patch file, and is available here. I am not sure if the ISP file has been validated, but I assume it would have been if it is being released with the driver. Is there any way to tell if the ISP file I am using has been compiled for, or is compatible with the jetson board I am using (Jetson AGX Orin, jetpack 5.0.2)?
there’re some camera software stack bugs, you may see-also Release Notes (r35.3.1) .
is it possible for moving to latest release version (i.e. l4t-r35.3.1) for confirmation?
I don’t have drivers for the latest release version (r35.3.1), so moving to that release could be a rather time consuming venture. I would have to port the drivers myself, and troubleshoot that port while knowing that the driver may very well be faulty. What are the odds that moving to the latest release solves the problems I am seeing? If you think it will likely solve the problem, I will work to port the driver to the latest release version. Are there any other tests we can do to determine what the issue may be?
can you root-cause the issue is happened only with ISP override file applied?
once it’s without ISP override file, can you confirm the sensor functionality works normally?
hence…
let’s have some tests before you porting drivers to JP-5.1.1/ l4t-r35.3.1
for example,
(1) please check you’ve clean all the cache files before sensor apply ISP.
i.e. /var/nvidia/nvcam/settings/*.bin
(2) where did you locate the ISP files. the path in comment messages looks incorrect.
please put the files under… /var/nvidia/nvcam/settings/camera_overrides.isp
(3) I’ve went through github, camera_overrides.isp very quickly.
you may revise ISP as following for some furthre testing…
(3.1) disable LSC… lensShading.correction_type.enableLSC = FALSE;
(3.2) hacking the control points as 1… lensShading.ctrlPointsCount = 1;
(3.3) please double check your sensor device tree settings, the active dimension should be the same as… lensShading.imageHeight = 3040;, and lensShading.imageWidth = 4056;
moreover,
it’s suggest to moving forward to the latest Jetpack release. you may also ping vendors for the latest driver, or patches to adapt the drivers with r35.3.1.
there’re developer guides for your reference,
for example, Camera Driver Porting, and Sensor Software Driver Programming.
Disabling LSC and setting lensShading.ctrlPointsCount = 1 has done the trick! I am now able to see a clear image from the camera, and with the proper colors. Thank you so much for your help!
I now need to add a couple other resolution modes to the sensor driver, but I should be able to pull from other IMX477 drivers to help make this happen.
