VPI Stereo Disparity background (sky) noise

Hi,

I’m currently running vpi 2.1 stereo disparity example from vpi_sample_02_stereo_disparity on Jetson (Jetpack 5.0.2).
The results have a lot of noise in the background – mainly in the sky.
The sky in this example image (input) is clear.

Input:


Output:
(CUDA backend, disp num 128, fixed blocksize, default confidence threshold and quality)

Expected:

The parameters I’ve tried in combination of:
[backend] CUDA, PVA-NVENC-VIC
[max disparity] 128, 256
[block size] 3, 5, 7
[confidence threshold] 32767, 49150
[quality] 4, 6, 8

All of the disparity results had the noise in the sky.

Tried adding smoothing filters like gaussian filter to try to minimize the noise, but has little affect.

Including the image produced from opencv stereo algorithm using the same sample set:

The left and right images are both rectified and captured with global shutter

Any idea what’s causing the issue here?

Hi,

Do you apply the filter to the input stereo images or the disparity map?

Thanks.

Hi,

No filter applied to input stereo images or the disparity map
Thank you

Hi,

Please try to apply some smoothing filter to the input image to see if it helps.

Thanks.

Hi, I have already tried in various sizes of filters, including aggressively smoothing but it could never get rid of the noise in the sky.

Hi,

Below is what we got by feeding your image pair to the default 02-stereo_disparity sample.

It looks like the confidence of the sky is much lower.
You can try to update the disparity value if the confidence value is not high enough in that region.

Thanks.

Hi,

Thanks for giving it a try.
As mentioned above, I’ve tried different combinations of disparity values block sizes, confidence threshold, and quality to see if I can get a better result.
I was not able to get rid of the sky noise.

Thanks!

Hi,

Have you tried this with other libraries like OpenCV?

If not, please give it a try.
This will tell us whether this is a limitation of the algorithm or something we can tune to get a better result.

Thanks.

Hi,

Yes, I have! Please see the last image on the post.

It’s looking like a limitation of the algorithm. Can you please try tuning it?

Hi @AastaLLL ,

Following up to see if you were able to make any progress on this.
Please let me know if you need me to try anything!

Thanks