How to use PolynomialLensDistortion Model?

[happy886]

Distortion correction model using vpi interface on agx equipment；
version：
nvidia@localhost:~/xxw/df_gtest/df_src_test$ head -n 1 /etc/nv_tegra_release

R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020

vpi 0.4.4

There is a demo for the fisheye model, but no demo for the polynomial model. The customer wants to compare these two models.

The polynomial model requires k1,k2,k3,k4,k5,k6 and p1,p2, which can be obtained by using opencv’s CALIB_RATIONAL_MODEL, but it seems that the relevant parameters obtained by this method are brought into the polynomial model interface of vpi, and the image obtained is not correct.
How should the polynomial model of vpi interface be used correctly?

[happy886]

Why after the rectified image was polynomial model get stereo?

[AastaLLL]

Hi,

Have you checked below VPI doc?

https://docs.nvidia.com/vpi/algo_ldc.html

Thanks.

[happy886]

yes,There is only the fisheye model and only 4 distortion coefficients.
How should the polynomial model of vpi interface be used correctly?
How to obtain the distortion coefficient of polynomial model?

[happy886]

4 distortion coefficients

[happy886]

Polynomial model needs 8 distortion coefficients. How to obtain them?

[AastaLLL]

Hi,

Sorry that we currently do have a sample for the polynomial model.

Could you share the sample and images with us?
We can give it a check and share how to modify it.

Thanks.

[happy886]

Thank you, this problem has been solved. It is caused by the large error of 8 parameters calculated by opencv because there are only 4 calibration pictures. The number of calibration images increased to 31, and the parameters calculated by opencv were imported into the polynomial model with correct distortion correction.
However, when the fisheye model uses the data of the camera, the calibration checkerboard is 8x8,the data after distortion correction is not correct. I wonder why?The same program uses the official own pictures and self-collected camera pictures have a very big difference. Official Checkerboard 10x7; The company uses a checkerboard 8x8

chessboard_1.zip (6.0 MB)

[happy886]

I downloaded the checkerboard, 10x7, from opencv’s official website and tested the images with vpi_sample_11_fisheye demo, also unsuccessful

https://docs.opencv.org/2.4/_downloads/pattern.png

Pictures1.zip (6.7 MB)

[AastaLLL]

Hi,

Thanks.

We need to check this further.
Will share more information with you later.

[AastaLLL]

Hi,

Could you also share the source you used with us?
Thanks.

[AastaLLL]

Hi,

Could you also try to collect the dataset as follows:

• Fix camera position
• Move the pattern around (the checkerboard needs to be on the same plane)
• Cover the field of view as much as possible.

Thanks.

[happy886]

Use your official demo;

The following can be added to the flag parameter of the official demo to solve this problem

    // VPI currently doesn't support skew parameter on camera matrix, make sure
    // calibration process fixes it to 0.
    int flags = cv::fisheye::CALIB_FIX_SKEW;   //official par
    //add flags  par

** int flags=0;**
** flags |=cv::fisheye::CALIB_RECOMPUTE_EXTRINSIC;**
** flags |=cv::fisheye::CALIB_CHECK_COND;**
** flags |=cv::fisheye::CALIB_FIX_SKEW;**

    // Run calibration
    {
        cv::Mat rvecs, tvecs; // stores rotation and translation for each camera, not needed now.
        double rms = cv::fisheye::calibrate(corners3D, corners2D, imgSize, camMatrix, coeffs, rvecs, tvecs, flags);
        printf("rms error: %lf\n", rms);
    }

[AastaLLL]

Hi,

Do you get the expected result after applying the OpenCV calibration?
Thanks.

[happy886]

Yes, opencv correction is also correct after adding the relevant flag parameters