Converting an EGLImageKHR to a VPIImage throws an error

[vladimir32]

I have an EGLImageKHR which I’m getting from a camera using libargus and an egl stream. I’m running on a jetson orin on jetpack 6.0.

I’m trying to convert the egl image to a VPIImage with the following code:

VPIImageData image_data;
image_data.bufferType = VPI_IMAGE_BUFFER_EGLIMAGE;
image_data.buffer.egl = egl_image;
VPIImage vpi_image = NULL;
VPIStatus status = vpiImageCreateWrapper(&image_data, NULL, 0, &vpi_image);

I’m getting VPI_ERROR_INVALID_ARGUMENT for the status.

I’ve also tried passing flags for other backends, like CPU, CUDA, VPI, and TEGRA.

Is what I’m trying to do supported? Or is VPI_IMAGE_BUFFER_EGLIMAGE a deprecated constant?

[AastaLLL]

Hi,

Based on the document, here are the possible reasons for returning VPI_ERROR_INVALID_ARGUMENT.

• img is NULL.
• data is NULL or contains invalid/unsupported values
• Buffer type in data isn’t supported.
• EGLImage handle is NULL or invalid (e.g. EGL_NO_IMAGE).
• NvBuffer file descriptor is invalid.
• No backend were given and VPI_REQUIRE_BACKENDS is set.

Please check if you have hit any of the above issues.

Thanks.

[vladimir32]

img is not null from my code.
data is also not null from my code.

I have a separate way of getting the data out by copying the data out to an nv buffer, then using NvBufSurfaceFromFd and NvBufSurfTransform, then making a vpi image with type VPI_IMAGE_BUFFER_NVBUFFER. That other way works (I can see the image), so the EGL image must be not null and valid.

“NvBuffer file descriptor is invalid.” doesn’t apply to my use case I assume.

I don’t understand what backend to provide, but I have tried all backends in VPIBackend so I’m ruling that one out.

The only one that is left is

• Buffer type in data isn’t supported.

But I’m on an nvidia jetson orin. I thought VPI_IMAGE_BUFFER_EGLIMAGE is specifically meant for the jetson with the unified memory.

What else can I check?

[AastaLLL]

Hi,

Not sure if it helps but could you try to update the NULL in the below line into nullptr?

VPIStatus status = vpiImageCreateWrapper(&image_data, NULL, 0, &vpi_image);

We can wrap the EGL image without error in a very similar way:

VPIImageData data;
VPIImage img;

memset(&data, 0, sizeof(data));
data.bufferType = VPI_IMAGE_BUFFER_EGLIMAGE;
data.buffer.egl = image;

CHECK_VPI_STATUS(vpiImageCreateWrapper(&data, nullptr, 0, &img));

Thanks.

[vladimir32]

No, changing NULL to nullptr doesn’t fix the issue.

Here’s a full minimal example of the issue. We have a Hawk camera connected to our jetson orin agx.

#include <Argus/Argus.h>
#include <Argus/Ext/SensorTimestampTsc.h>
#include <EGLStream/EGLStream.h>
#include <vpi/Image.h>

#include <algorithm>
#include <iostream>
#include <memory>
#include <vector>

int main() {
  Argus::Status status;
  Argus::UniqueObj<Argus::CameraProvider> camera_provider(
      Argus::CameraProvider::create(&status));
  auto i_camera_provider =
      Argus::interface_cast<Argus::ICameraProvider>(camera_provider);

  std::vector<Argus::CameraDevice *> camera_devices;
  i_camera_provider->getCameraDevices(&camera_devices);

  std::vector<Argus::CameraDevice *> stereo_devices;
  stereo_devices.push_back(camera_devices[0]);

  Argus::UniqueObj<Argus::CaptureSession> capture_session(
      i_camera_provider->createCaptureSession(stereo_devices));

  Argus::UniqueObj<Argus::Request> request(
      Argus::interface_cast<Argus::ICaptureSession>(capture_session)
          ->createRequest());

  auto *i_request = Argus::interface_cast<Argus::IRequest>(request);
  auto *i_capture_session =
      Argus::interface_cast<Argus::ICaptureSession>(capture_session);

  Argus::UniqueObj<Argus::OutputStreamSettings> stream_settings(
      i_capture_session->createOutputStreamSettings(Argus::STREAM_TYPE_EGL));

  auto *i_egl_stream_settings =
      Argus::interface_cast<Argus::IEGLOutputStreamSettings>(stream_settings);

  i_egl_stream_settings->setPixelFormat(Argus::PIXEL_FMT_YCbCr_420_888);
  i_egl_stream_settings->setResolution({1920, 1200});
  i_egl_stream_settings->setMetadataEnable(true);

  Argus::interface_cast<Argus::IOutputStreamSettings>(stream_settings)
      ->setCameraDevice(stereo_devices[0]);
  Argus::UniqueObj<Argus::OutputStream> stream(
      i_capture_session->createOutputStream(stream_settings.get()));

  i_request->enableOutputStream(stream.get());

  Argus::UniqueObj<EGLStream::FrameConsumer> consumer(
      EGLStream::FrameConsumer::create(stream.get()));

  Argus::interface_cast<Argus::ICaptureSession>(capture_session)
      ->capture(request.get());

  Argus::Status frame_status;
  Argus::UniqueObj<EGLStream::Frame> argus_frame(
      Argus::interface_cast<EGLStream::IFrameConsumer>(consumer)->acquireFrame(
          1'000'000'000, &frame_status));
  assert(frame_status == Argus::STATUS_OK);

  auto *i_frame = Argus::interface_cast<EGLStream::IFrame>(argus_frame);

  VPIImageData image_data;
  VPIImage vpi_image;

  memset(&image_data, 0, sizeof(image_data));
  image_data.bufferType = VPI_IMAGE_BUFFER_EGLIMAGE;
  image_data.buffer.egl = i_frame->getImage();

  VPIStatus create_wrapper_status =
      vpiImageCreateWrapper(&image_data, nullptr, 0, &vpi_image);

  if (create_wrapper_status == VPI_SUCCESS) {
    std::cout << "Success." << std::endl;
  } else {
    std::cout << "Failure: " << create_wrapper_status << std::endl;
  }
}

[AastaLLL]

Hi,

Thanks for sharing the sample.

Could you also share the Makefile or CMakefile with us to build the sample locally?
More, do you have another CSI camera on hand?
If yes, please help us to check if this issue is camera-dependent.

Thanks.

[vladimir32]

Hello,

I do not have another camera.

Here’s a cmake file:

find_package(vpi REQUIRED)
find_package(Argus REQUIRED)
add_executable(main main.cpp)
target_include_directories(
  main
  PUBLIC ${ARGUS_INCLUDE_DIRS})
target_link_libraries(
  main
  PUBLIC
         vpi
         ${ARGUS_LIBRARIES})

[AastaLLL]

Hi,

When we test the sample, the cmake command fails with the Argus cmake file.
It seems we don’t provide such a file. Do you implement it manually?

CMake Error at CMakeLists.txt:2 (find_package):
  By not providing "FindArgus.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Argus", but
  CMake did not find one.

  Could not find a package configuration file provided by "Argus" with any of
  the following names:

    ArgusConfig.cmake
    argus-config.cmake

  Add the installation prefix of "Argus" to CMAKE_PREFIX_PATH or set
  "Argus_DIR" to a directory containing one of the above files.  If "Argus"
  provides a separate development package or SDK, be sure it has been
  installed.

Thanks.

[vladimir32]

You can find it here: /usr/src/jetson_multimedia_api/argus/cmake/FindArgus.cmake

[AastaLLL]

Hi,

Thanks for your patience.

Confirmed that we can reproduce this issue internally.
We are checking this with our internal team. Will provide more info to you later.

Thanks.

[AastaLLL]

Hi,

Thanks for your patience.
The error is because VPIImage doesn’t support any planar YUV420 formats:
https://docs.nvidia.com/vpi/ImageFormat_8h.html

A workaround is to use NvBuffer (iNativeBuffer->createNvBuffer() & iNativeBuffer->copyToNvBuffer()).
Once you get the NvBufSurface, please select a supported format to wrap into VPI.

You can find below sample to get the NvBufSurface:

/usr/src/jetson_multimedia_api/samples/09_argus_camera_jpeg/

Thanks.