Disabling Xavier NX ISP Oversharpening

[alexisguiter]

Hi everyone,

I’m currently testing out a colour IMX296 sensor with Gstreamer and Libargus. When reviewing footage, I noticed that there is oversharpening happening in the frames. Looking through the gstreamer/libargus preferences and settings, I could not find any way of disabling oversharpening. Is there a way to disable or minimize oversharpening with Libargus? For reference, here is the gstreamer pipeline that I used:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 timeout=30 ispdigitalgainrange='1 1' gainrange='1 1' aeantibanding=0 wbmode=0 ee-mode=0 tnr-mode=0 ! 'video/x-raw(memory:NVMM),width=1440,height=1080,framerate=60/1,format=NV12' ! omxh265enc control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0 temporal-tradeoff=0 preset_level=3 ! matroskamux ! filesink location=test.mkv

Furthermore, here is an example of oversharpening in a frame. Notice the white halo surrounding the wooden poll.

(This is also linked with a 3 year old thread about turning off corrections done by the ISP)

[JerryChang]

hello alexisguiter,

please narrow down the issue, you may dump the IMX296 raw images to check capture frames before processed by ISP.
for example,
you may review the sensor pixel formats, $ v4l2-ctl -d /dev/video0 --list-formats-ext
please using the supported resolution and pixel formats to dump the raw files,
i.e. $ v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --stream-mmap --stream-count=1 -d /dev/video0 --stream-to=test.raw

there’re some 3rdparty tools to analysis the raw files, such as 7yuv, IrfanView…etc.
thanks

[JerryChang]

hello alexisguiter,

BTW,
are you having other Jetson platforms (such as Jetson-TX2) to reproduce the issue?
had you also contact with your sensor vendor for the tuning supports.
thanks

[alexisguiter]

The sensor pixel format is RG10 for my colour camera and Y10 for my monochrome camera, and when dumping and reviewing RAW videos (I used ffmpeg to encode them) the oversharpening is not present, the image looks much cleaner, more useful for my algorithms and better on the eye. I still believe the issue is due to post-processing done on the Nvidia ISP.

I have a Jetson Nano I could test this on, but not a TX2 or TX2 NX.

[JerryChang]

hello alexisguiter,

thanks for sharing the info, yes, this should related to the post-processing within ISP.
since the software were different for each Jetson series (i.e. Jetson Xavier NX v.s. Jetson Nano),
please also have confirmation on Jetson Nano platform, this may helps to narrow down the issue.

[alexisguiter]

Hi JerryChang,

I will test this out on the Jetson Nano and get back to you on this thread as soon as I can.

[prapor]

I can confirm this issue on Jetson Nano 2GB with default IMX477 driver.

[JerryChang]

hi all,

please also check you still capture oversharp images with Argus sample applications,
for example, Argus/public/samples/oneShot,
thanks

[player500]

Here is a still image captured with argus_camera (denoising, EE, digital ISP gain disabled). HW setup: Jetson NX devkit, imx477 camera, Edmund Optics 4mm lens.

[JerryChang]

hello player500,

I don’t see white halo surroundings but lots of noise according to your capture results.
I’ll also arrange resources to check this from our side, thanks

[carolyuu]

Hi player500,

Share our image captured with argus_camera on r32.5.1/Nano-2GB/imx477 result for you reference.

[prapor]

Hello carolyuu,

here is the problem in your picture.

[alexisguiter]

Hi,

I have tried argus oneshot and obtained the following images (With a Xavier NX and IMX296 sensor). Note how the outdoor image still shows oversharpening (trees and the roof have a white halo).

[JerryChang]

quick questions,
are you working with sensor vendors to have ISP tuning parameter applied? could you please contact with them to figure out what’s the settings being used.
thanks

[JerryChang]

hi all,

I’ve upload a pre-built binary based-on r32.5.1 to disable ISP sharpness function. Topic175200_May27_libnvscf.zip (2.7 MB)
could you please have a try to replace /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so with the attachment.
please also share the results for reference,
thanks

[alexisguiter]

Hi @JerryChang thanks for the update.

I logged in into my NX, and replaced the file you mentioned with your attachement.

However after a reboot, I get the following error:

Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:557 No cameras available

The pipeline I used was:

gst-launch-1.0 nvarguscamerasrc ispdigitalgainrange=“1 1” gainrange=“1 1” aeantibanding=0 wbmode=0 ee-mode=0 tnr-mode=0 ! ‘video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1440, height=(int)1080’ ! nvvidconv ! ximagesink -e

BTW I am currently using Jetpack 4.4

P.S concerning your previous question, no I am not currently working with sensor vendors for ISP tuning.

[JerryChang]

@alexisguiter

BTW I am currently using Jetpack 4.4

I see, the pre-built binary I’ve just attach is based-on JetPack-4.5.1 / l4t-r32.5.1
here’s another binary based-on JetPack-4.4 / l4t-r32.4.3 for your verification. Topic175200_JP44_libnvscf.zip (2.7 MB)

note,
it just a workaround to disable ISP sharpness function.
please do have verification and share us the results, so we could have investigation this further.
thanks

[alexisguiter]

Hello @JerryChang

This time, I was able to capture footage (See attached screenshots). However I do not see any visual changes regarding the oversharpening. In the screenshots the trees and the roof both show white halos.

To capture this footage I used:

gst-launch-1.0 nvarguscamerasrc ispdigitalgainrange=“1 1” gainrange=“1 1” aeantibanding=0 wbmode=0 ee-mode=0 tnr-mode=0 ! “video/x-raw(memory:NVMM),width=1440,height=1080,framerate=60/1,format=NV12” ! omxh265enc control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0 temporal-tradeoff=0 preset_level=3 ! matroskamux ! filesink location=test.mkv

Please let me know if there is anything else I could try/test on my side.

[JerryChang]

hello alexisguiter,

could you please try again with the attach pre-built binary,
Topic175200_May31_libnvscf.zip (2.7 MB)
thanks

[prapor]

Hello @JerryChang
Can you please make the same file (to disable ISP sharpness) for Jetson Nano 2Gb (Jetpack 4.5.1)?