VPI image cut much more after undistortion than OpenCV

[ps031a]

Enviornment: Jetpack 5.0.2, Ubuntu 20.04, vpi 2.1.6
VPI code:

    ctx->map = {};
    ctx->map.grid.numHorizRegions = 1;
    ctx->map.grid.numVertRegions = 1;
    ctx->map.grid.regionWidth[0] = ctx->cam_w;
    ctx->map.grid.regionHeight[0] = ctx->cam_h;
    ctx->map.grid.horizInterval[0] = 1;
    ctx->map.grid.vertInterval[0] = 1;
    CHECK_STATUS(vpiWarpMapAllocData(&ctx->map));
    // Extrinsic should be an identical matrix.
    ctx->extrinsic = {};
    ctx->extrinsic[0][0] = ctx->extrinsic[1][1] = ctx->extrinsic[2][2] = 1;

    loadIntrinsic(ctx->intrinsic_path, ctx->intrinsic,
        ctx->distortionModel.lens);

    vpiWarpMapGenerateFromPolynomialLensDistortionModel(
        ctx->intrinsic, ctx->extrinsic, ctx->intrinsic, &ctx->distortionModel.lens, &ctx->map);
    

    CHECK_STATUS(vpiSubmitRemap(ctx->stream, VPI_BACKEND_CUDA, ctx->remap, ctx->vimg,
                                            ctx->tmpIn, VPI_INTERP_CATMULL_ROM, VPI_BORDER_ZERO, 0));

OpenCV code (python):

newCameraMatrix, validPixROI = cv.getOptimalNewCameraMatrix(intrinsic, distortion, (1920, 1080), 0)

R = np.array([[1,0,0],[0,1,0],[0,0,1]])

map_x, map_y = cv.initUndistortRectifyMap(intrinsic, distortion, R, newCameraMatrix, (1920, 1080), cv.CV_32FC1)
warped_img = cv.remap(orig_img, map_x, map_y, cv.INTER_CUBIC)

original image:

undistorted by cv:

undistorted by vpi:

Obvoiusly the keyboard of this laptop is clipped after undistort by vpi.

[AastaLLL]

Hi,

It looks like there is still some distortion in the OpenCV results.
So the view angle will be larger.

Thanks.

[ps031a]

So the warpmap of opencv and vpi is not the same even if they use the same camera matrix and distortion coefficients using the above code?

[ps031a]

The above pics is not the camera’s orignial intrinsics. I got another group of pics of the camera with the orignal intrinsics:
original:

vpi:

opencv:

[AastaLLL]

Hi,

Could you share how you apply the same distortion coefficients to the OpenCV and VPI?
Thanks.

[ps031a]

Here is the code and original image.
undistort_test.tar.gz (1.2 MB)

[AastaLLL]

Hi,

Thanks for sharing.
Will update more info with you.

[AastaLLL]

Hi,

Thanks for your patience.

It looks like the maps used for warping are different.
Could you try to use the same warping map to see if the output is the same?
(Maybe set the undist_map with the value of map_x and map_y?)

This helps to figure out whether the issue comes from warping or the way of generating the map.

Thanks.

[ps031a]

I tested again with C++ and finally found the difference.
It’s caused by newCameraMatrix calculated by cv2.getOptimalNewCameraMatrix() . If I change the kout param of polynomial_correction to newCameraMatrix, the output image is the same as opencv result.

[AastaLLL]

Hi,

It sounds like the difference comes from the algorithm.
Do the NewCameraMatrix results meet your requirements?

Thanks.

[ps031a]

yes. that’s the result I want.