NVIDIA VPI - Access raw data of VPI_IMAGE_FORMAT_NV12_ER_BL or any BL image

[alex_aje]

I’m trying to implement Nvidia VPI Convert Image Format algorithm. When my output image is in a PitchLinear format, I’m not having any issue, I am able to access raw data of image correctly, either by using .buffer.pitch or by exporting to a cvMat and working with cvMat.data (when viable conversion).

On the other hand, when using some BlockLinear output, I have not found a way to access raw data in images (either to export it to a cv mat or to store it in another place)(the “common” and “most seen” way is locking image using vpiImageLockData, but this is only compatible with Pitch Linear bufferTypes, not BL), I took a look at documentation, and I did not really find anything useful that may help me with this.

My simplified code to convert from BGR to NV12_ER_BL looks like:

void BGRToNV12_ER_BL(const cv::Mat& cvIn, cv::Mat& cvOut)
{

VPIContext context;
vpiContextCreate(0, &context);

vpiContextSetCurrent(context);

VPIStream stream;
vpiStreamCreate(0, &stream);

VPIImage input;
vpiImageCreateWrapperOpenCVMat(cvIn, 0, &input);

VPIImage middle;
vpiImageCreate(w, h, VPI_IMAGE_FORMAT_BGRA8, 0, &middle);

VPIImage output;
vpiImageCreate(w, h, VPI_IMAGE_FORMAT_NV12_ER_BL, 0, &output); //also this could be with VPI_RESTRICT_MEM_USAGE  instead of 0 flag to restrict locking image

vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, input, middle, NULL);
vpiSubmitConvertImageFormat(stream, VPI_BACKEND_VIC, middle, output, NULL);
vpiStreamSync(stream);

/* 
Here, I don't know how to lock output image to retrieve its data from VIC memory, 
considering it is NV12_ER_BL, that does not use pitch
and considering that vpiImageLockData does not support
VPI_IMAGE_BUFFER_CUDA_ARRAY, only VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR 
and VPI_IMAGE_BUFFER_CUDA_PITCH_LINEAR, which are not
suitable here, since I want Block Linear
*/
VPIImageData outData;

}

Any ideas?

[kayccc]

Which Jetson platfor and JetPack SW you’re using?

[alex_aje]

Jetson Orin Nano. VPI 3.2. Jetpack 6

[kayccc]

Hi,
Moving this issue into Orin Nano forum.

Please cconvert the frame data to pitch linear. It is not supported to access data in block linear:

VPI - Vision Programming Interface: Image

[alex_aje]

Ok, I’ve tried converting it to pitch linear
To my previous code, I’ve added:
VPIImageData outData;
vpiImageLockData(output, VPI_LOCK_READ, VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR, &outData);
vpiImageDataExportOpenCVMat(outData, &cvOut);

//acording to documentation in vpiImageDataExportOpenCVMat, I should be able to use this:
cv::cvtColor(cvOut, cvOut, cv::COLOR_YUV2RGB_NV12);
cv::imwrite(“thisShouldBeEqualsToMyInput.jpg”, cvOut);

However, I don’t get exactly the same image as my input.
My input is:

However, what I get as output after cv::imwrite(“thisShouldBeEqualsToMyInput.jpg”, cvOut);

Is it right what I am doing? Does that have to do with the fact that I use VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR as buffer type when my image is BLOCK_LINEAR?

[alex_aje]

Any other recommendation?

[DaneLLL]

Hi,
Do you compare it with OpenCV? Or which software stack? Please test on Jetpack 6.2 and see if you observe the issue. If yes, share a test sample so that we can check further.

It looks similar to this:

Jetson/L4T/VPI Interoperability Example - eLinux.org

Should be the software stack does not handle extension range correctly or does not perform gamma correction.