NvBufSurfTransform Interpolation Method Issue

[eslam.ahmed]

• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 7.1
• TensorRT Version: 10.3.0.26
• NVIDIA GPU Driver Version (valid for GPU only): 565.57.01
• Issue Type( questions, new requirements, bugs ) : Issue/Bug
• How to reproduce the issue ? Resize a small face image (50x50) to 256x256 while using NvBufSurfTransform using nearest and bilinear interpolation.

Good day,

I am not sure if this by design or a bug but nvinfer, working in secondary mode, is resizing the boxes from the primary detector using NvBufSurfTransform while using the default interpolation mode NvBufSurfTransformInter_Default which applies Nearest Interpolation for both GPU and VIC. Nearest interpolation introduced blocky/pixelated when upscaling the crop/bbox to network dimensions of 256x256 causing major instability in the inference results.

Furthermore, changing the interpolation mode to NvBufSurfTransformInter_Bilinear produces the same exact blocky/pixelated crop which should not be the case. I am wondering if Bilinear interpolation should produce different results than the Nearest interpolation.

Only the Cubic interpolation NvBufSurfTransformInter_Algo1 that was able to produce a smooth image.

Find the origin image and the upscaled variants attached below.

Origin Image

Nearest

Bilinear

Cubic

Please advise,
Thanks!

[Fiona.Chen]

The Bilinear interpolation algorithm and Bicubic interpolation algorithm themselves can show you the reason.

[eslam.ahmed]

Yes, that’s expected of Cubic interpolation since it considers the closest 16 pixel values in the input image to compute the new pixel value. but Why is the Bilinear Interpolation, which considers the nearest 4 pixels, is producing blocky results like the nearest interpolation?

Bilinear interpolation should result in soft, blurry image and not as blocky as the nearest interpolation.

[Fiona.Chen]

You can try other implementations of bilinear algorithm to check whether there is blocks.
Or you can discuss in other algorithm related forums.

[eslam.ahmed]

Yes, OpenCV is indeed producing different results. For reference, here’s the bilinear interpolation for the same image

It is a good idea to give nvinfer the ability to configure the interpolation mode used in resizing for the preprocessing instead of forcing the nearest interpolation by default.

Closing this ticket,
Thanks!

[Fiona.Chen]

If you need openCV aligned preprocessing implementation, there is a sample for it: deepstream_tao_apps/apps/tao_others/deepstream_custom_preprocessing_app at master · NVIDIA-AI-IOT/deepstream_tao_apps