Performance issues while locking and unlocking VPI image

I am using VPI for rescaling purpose on Nvidia Xavier with JetPack 5.0 and VPI 2.0

Dummy code is as follows.

cv::Mat img[CIRC_BUFF_COUNT]; //cv::Mat declared for input
VPImage inp_res; //VPI Image for wrapping inp created.

vpiImageCreateWrapperOpenCVMat(img, 0, inp_res); //Created a wrapper for a cv::Mat container

Read images in img in a circular buffer
for(i=0; i<CIRC_BUFF_COUNT; i++)
{
img[i] = cv::imread(path);
}

vpiImageLock(img);
img.at(1000,1000) = 23; //One pixel value of input updated
vpiImageUnlock(img);

Looks like VPI is copying the complete buffer data between locking and unlocking which is taking large amount of time ~20ms.
Please share the details of it?