VPI Lens Correction without Cropping

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
, ,
1

Set up info:

Jetson AGX ORIN
Deepstream version : 6.3
Jetpack version: 5.1.2
TRT: 8.5.2

Issue: I am trying to calibrate my camera using ‘remap’ in VPI with CUDA back-end. I already have done camera calibration(without crop) using Opencv which I am trying to replicate using VPI. However, image is calibrated only with cropping. Could you let me know if there’ s any way to resolve it?

Currently, I am using the below code using opencv calibration method which gives me complete image without cropping:

new_K = cv2.fisheye.estimateNewCameraMatrixForUndistortRectify(scaled_K, D, dim2, np.eye(3), balance=balance)
map1, map2 = cv2.fisheye.initUndistortRectifyMap(scaled_K, D, np.eye(3), new_K, dim3, cv2.CV_16SC2)
undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)

The below code gives me an error if I try to change grid size to the size I need. Otherwise, it works fine.

grid = vpi.WarpGrid((960, 540))
undist_map = vpi.WarpMap.fisheye_correction(grid,
                                            K=K[0:2,:], X=np.eye(3,4), coeffs=D.flatten(),
                                            mapping=vpi.FisheyeMapping.EQUIDISTANT)
with vpi.Backend.CUDA:
   undistort_img = vpi.asimage(img).convert(vpi.Format.NV12_ER).remap(undist_map, interp=vpi.Interp.CATMULL_ROM, border=vpi.Border.ZERO).convert(vpi.Format.RGB8)

und_image = undistort_img.cpu()
3

Hi,

Please note that VPI uses backward warping.

The default the size between output and input is equal.
Have you tried to set a bigger output size to reduce the cropped region?

Thanks.

4

Hi AastaLLL,

Yes, I gave the required output size to create the WarpGrid. But, it seems to only accept original input size.

Thanks.

5

Hi,

If you want to preserve the view angle in the same resolution, scaling down is required.
Could you give it a try?
Generate a new undistortion matrix with a scaling matrix.

Thanks.

6

Hi AastaLL,

Thanks for your response.

The given solution might work. We’ ll give it a try. Thank you.