I’ve spent the past few months working with IMX477 sensor modules across the Jetson and Raspberry Pi platforms. Even for a 5 year old sensor, it’s very capable, and all module variations (Arducam, Leopard Imaging, Raspberry Pi HQ, etc.) seem to perform similarly in regards to their color output (to be expected). However, when used on the Raspberry Pi platform, the color accuracy and subjective color science quality is noticeable better compared to the Jetson platform. I’ve tried every camera_overrides.isp file across the internet, and while some are better than others, none of them get close to the output from the Raspberry Pi. Some of this discrepancy is due to the tuning process taking into account the actual lens being used. These lens shading parameters are only valid for that one particular lens, and may create unwanted color shifts across the frame. Yet even when removing the shading parameters, the output is still far off.
So, here I am, without any documentation of the Jeston ISP provided by Nvidia, asking folks to figure out a way to blindly work with the camera_overrides.isp file. It’s certainly not an ideal situation, and is also a very tall order. But I figured I’d toss a post on the forums to see if anyone had ideas on how we could tackle this. I realize most folks don’t need accurate nor subjectively “good looking” color science as they’re using the Jetson platform for computer vision, AI, machine learning, etc. But there are some people (and likely small companies) that are unable to pay the expensive $5k-$15k for ISP tuning. Those people, including myself, are simply looking to achieve better color science from the Jetson ISP, and ideally the tools to effectively tune it themselves.
Quick update with some rough comparison photos with an IMX477 on both a Jetson Nano and Raspberry Pi focused on color noise reduction (but you can also easily make some subjective judgements on color as well). I will try to take some more over the next few days, including outdoor shots focused on color reproduction.
These photos were created with the following commands:
Note that these were captured with different lenses and the placement of the cameras weren’t perfect. The photos were also cropped ~1.5x to highlight regions that were easy to compare. I also explicitly enabled tnr-mode=1 for the Nano but disabled noise reduction on the Pi.
I understand the current Nvidia policy is to refer to a camera partner, but is it okay for the community (ie. not Nvidia) to discuss ways to work with the camera_overrides.isp file on this forum?
In my opinion, Nvidia should strongly reconsider its stance on providing documentation / tools for working with the ISP. I believe it’s possible for camera partners to still provide a value-add while also allowing developers work with the ISP themselves. Not everyone is going to be able to afford contracting a camera partner, particularly during an evaluation phase when they’re testing multiple sensors, lenses, etc. I think there’s room for a balance - allow developers to roughly tune on their own, particularly during the prototyping/eval phases, and then they can work with camera partners to finalize/perfect the tuning.
