How do I get image from cudaBayerDemosaic and connect to VPI?

Hi,

I try to modify cudaBayerDemosaic example to get images to VPI, but something is wrong and need your help:

For example, after :
iCaptureSession->capture(request.get(), TIMEOUT_INFINITE, &status);

I add these codes:

int dmabuf_fd;
EGLImageKHR eglImage;
CUgraphicsResource cudaResource; 
CUeglFrame cudaEGLFrame;
VPIImageData tempImagedata;
CUresult cudaStatus;

NvBufferCreate(&dmabuf_fd, 
               iEGLStreamSettings->getResolution().width(), 
               iEGLStreamSettings->getResolution().height(), 
               NvBufferLayout_Pitch, 
               NvBufferColorFormat_ABGR32);            

eglImage = NvEGLImageFromFd(iEGLOutputStream->getEGLDisplay(), dmabuf_fd);
**cudaStatus = cuGraphicsEGLRegisterImage(&cudaResource, eglImage, 0);**

But it fails and cudaStatus hows “CUDA_ERROR_INVALED_CONTEXT”.

What happens?

Thanks!

Hi,

VPI has an image wrapper for NvBuffer.
Would you mind checking if it can meet your requirement?

https://docs.nvidia.com/vpi/1.2/group__VPI__NvBufferInterop.html#ga6c392012aedd3f4be73cbf4b57ad19ba

Thanks.

Hi,

I tried, modify code as follows:

int dmabuf_fd;
VPIImage vpiImage;
EGLImageKHR eglImage;
CUgraphicsResource cudaResource; 
CUeglFrame cudaEGLFrame;
CUresult cudaStatus;
    
vpiImageCreate(
    iEGLStreamSettings->getResolution().width(),
    iEGLStreamSettings->getResolution().height(),
    VPI_IMAGE_FORMAT_RGBA8,
    0, 
    &vpiImage);

vpiImageCreateNvBufferWrapper(dmabuf_fd, NULL, 0, &vpiImage);
eglImage = NvEGLImageFromFd(iEGLOutputStream->getEGLDisplay(), dmabuf_fd);
cudaStatus = cuGraphicsEGLRegisterImage(&cudaResource, eglImage, 0);

But, cudaStatus shows: “CUDA_ERROR_UNKNOWN”

Hi,

How do I get RAW image data from Argus → cudaBayerDemosaic → VPI ?

Thanks!

Hi,

Do you want to get the raw image from Argus only?
Is there any image processing or algorithm in VPI that needs to be applied?

If not, you don’t need to wrap it into the VPI image.

Thanks.

Hi,

What I want to do is like that:

VPI Remap producing green output - Jetson & Embedded Systems / Jetson AGX Xavier - NVIDIA Developer Forums

Thanks~

Hi,

So you want to apply the remap algorithm from VPI to the Argus image, is that correct?
Thanks.

Hi,

Yes, it’s correct.
I want apply remap algorithm from VPI to the Argus image(From RAW Image → cudaBayerDemosaic → RGB image).

Thanks!

Thanks for the confirmation.

We do have an example to run VPI remap algorithm with the CSI camera.
It should find it on the Jetson Desktop or the below path:

/opt/nvidia/vpi1/bin/vpi_demo_remap

Thanks.

Hi,

That’s a good example but could you share source code to me?

If couldn’t,

  1. How do I modify cudaBayerDemosaic example to apply this remap algorithm?

  2. Or how do I warp image from “iEGLOutputStream->getEGLDisplay()” to VPI image?
    ==
    e.g. (after line 117~136 from main.cpp in cudaBayerDemosaic example)
    // Create the CUDA Bayer consumer and connect it to the RAW16 output stream.
    CudaBayerDemosaicConsumer cudaConsumer(iEGLOutputStream->getEGLDisplay(),
    iEGLOutputStream->getEGLStream(),
    iEGLStreamSettings->getResolution(),
    options.frameCount());
    PROPAGATE_ERROR(cudaConsumer.initialize());
    PROPAGATE_ERROR(cudaConsumer.waitRunning());

    // Submit the batch of capture requests.
    for (unsigned int frame = 0; frame < options.frameCount(); ++frame)
    {
    Argus::Status status;
    uint32_t result = iCaptureSession->capture(request.get(), TIMEOUT_INFINITE, &status);
    if (result == 0)
    {
    ORIGINATE_ERROR(“Failed to submit capture request (status %x)”, status);
    }
    }

    // Wait until all captures have completed.
    iCaptureSession->waitForIdle();

==
Thanks!

Hi,

Have you checked vpiImageCreateEGLImageWrapper(.)?
https://docs.nvidia.com/vpi/1.2/group__VPI__EGLInterop.html#ga06c12116ce8eda9dbf96b75883d08853

It can create a VPI image from a given EGLImage.
Please check if this can meet your requirement.

Thanks.

Hi,

I use vpiImageCreateEGLImageWrapper as follows:
(I add vpiSubmitConvertImageFormat is just for test)

But imageTempData.planes[0].data is still 0…

Hi,

Thanks for the update.

Would you mind sharing a simple reproducible source with us?
So we can check why the data is not valid directly.

Thanks.

Hi,

I use this code to modify:
/usr/src/jetson_multimedia_api/argus/samples/cudaBayerDemosaic

Unzip password : 12345

As attahcment:

cudaBayerDemosaic.zip (50.3 KB)

Thanks~~~