How to Decode & rescale with HW decoder + VPI rescale

[vkakula]

Hi, I am using Jetson Xavier AGX, in which trying to decode the image with multimedia API as below

jpegdec->decodeToFd(fd, in_buffer, file_size, pixfmt, width, height);

And after that how can I convert the fd buffer to VPIImage for doing further processing like rescaling etc?

[DaneLLL]

Hi,
Do you use Jetpack 4.6.2 or 5.0.2? The interface is different between the two releases.

[vkakula]

Jetpack 5.0.2

[DaneLLL]

Hi,
You can call the functions to get EglImage:

NvBufSurfaceFromFd();
NvBufSurfaceMapEglImage();

And refer to this patch to wrap EglImage to VPI Image:
Using VPI in GStreamer - #21 by AastaLLL

If you only need to rescale the resolution, can also call NvBufSurf::NvTransform()

[vkakula]

I have tried using the functions you have mentioned, but getting a segmentation fault, my code is below

ret = jpegdec->decodeToFd(fd, in_buffer, file_size, pixfmt, width, height);
NvBufSurface pSurf = NULL;
ret = NvBufSurfaceFromFd(fd,(void*)(&pSurf));
ret = NvBufSurfaceMapEglImage(pSurf, 0);

In the last line, I am getting a segmentation fault.

[DaneLLL]

Hi,
Please create other NvBufSurface and copy the decoded data to the buffers, for further processing. In 12_camera_v4l2_cuda sample, there is similar code for MJPEG decoding. Please refer to the part.

[vkakula]

Can you suggest to me which library should I link as I am getting the below error?

Error:
undefined reference to NvBufSurf::NvAllocate(NvBufSurf::NvCommonAllocateParams*, unsigned int, int*) undefined reference to NvBufSurf::NvTransform(NvBufSurf::NvCommonTransformParams*, int, int)

[DaneLLL]

Hi,
There is wrapper in

/usr/src/jetson_multimedia_api/samples/common/classes/NvBufSurface.cpp

Please build the cpp file.

[vkakula]

Yes, now the application got compiled, but I am getting below error while creating VPI wrapper from egl image

NvBufSurface pSurf = NULL;
NvBufSurfaceFromFd(fd,(void*)(&pSurf));

NvBufSurfaceMapEglImage(pSurf, 0);
EGLImageKHR egl_image;
egl_image = pSurf->surfaceList[0].mappedAddr.eglImage;

VPIImageData egl_img_data,new_vpiImgData;
memset(&egl_img_data, 0, sizeof(egl_img_data));
egl_img_data.bufferType = VPI_IMAGE_BUFFER_EGLIMAGE;
egl_img_data.buffer.egl = egl_image;

VPIImage new_im;
CHECK_STATUS(vpiImageCreateWrapper(&egl_img_data, nullptr, 0, &new_im));

Error
VPI_ERROR_INVALID_OPERATION: PVA is not available and may be oversubscribed in the system: PvaError_DeviceUnavailable

But here I hvae not given any flag to use PVA device

[vkakula]

Hi, after restarting the device the error was gone. But now i am getting a green image as output, not actual image

[vkakula]

Even though I have tried the below code without the functions you mentioned above, I got the output image and saved it. But the output image is a little brighter, not the same as the input image. Can you please tell me what I am doing wrong here?

Code:
jpegdec->decodeToFd(fd, in_buffer, file_size, pixfmt, width, height);
VPIImageData imgdata;
memset(&imgdata, 0, sizeof(imgdata));
imgdata.buffer.fd = fd;
imgdata.bufferType = VPI_IMAGE_BUFFER_NVBUFFER;
VPIImageWrapperParams vpi_params;
vpi_params.colorSpec = VPI_COLOR_SPEC_BT601;
VPIImage im, im2;
VPIImageData vpiImgData;
vpiImageCreateWrapper(&imgdata,&vpi_params,0,&im);
vpiImageCreate(width, height, VPI_IMAGE_FORMAT_RGBA8, 0, &im2);
vpiSubmitConvertImageFormat(stream, VPI_BACKEND_VIC, im, im2, NULL);
vpiStreamSync(stream);
vpiImageLockData(im2,VPI_LOCK_READ_WRITE,VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR,&vpiImgData)
cv::Mat Output_img;
vpiImageDataExportOpenCVMat(vpiImgData, &Output_img);
cv::imwrite(“out.png”, Output_img);

This is the code, below are the input and output images i am using
Input

Output

You can see that output image is little brighter than input.

[vkakula]

could you check the code in the above comment and suggest some input for solving the issue?

[DaneLLL]

Hi,
It may be an issue in VPI. Please try to do conversion through NvBufSurfTransform() and check if the color is correct.

[vkakula]

Okay will try that