Getting Argus MetadataContainer without a display or event or FrameConsumer?

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Is there a way to get Argus capture metadata without a display, using the Event class, or the FrameConsumer? I have a camera class, and it works, but now I need capture metadata. Based on the samples, there seems to be three ways of doing that:

  • from a egl stream and a display (optionally for a specific frame):
UniqueObj<EGLStream::MetadataContainer> metadataContainer(
    EGLStream::MetadataContainer::create(g_display.get(), m_eglStream));
EGLStream::IArgusCaptureMetadata *iArgusCaptureMetadata =
    interface_cast<EGLStream::IArgusCaptureMetadata>(metadataContainer);

Problem there is my camera class does not use a display at the moment (will nullptr work for that parameter?).

  • from an EventProvider, EventQueue, and Event like
const CaptureMetadata* metaData = iEventCaptureComplete->getMetadata();
const ICaptureMetadata* iMetadata = interface_cast<const ICaptureMetadata>(metaData);

Problem there is I’m not using this system. I adapted my code from the debayering example

  • using a FrameConsumer
        UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());
        if (!frame)
            break;

...

        // Print out some capture metadata from the frame.
        IArgusCaptureMetadata *iArgusCaptureMetadata = interface_cast<IArgusCaptureMetadata>(frame);
        if (!iArgusCaptureMetadata)
            ORIGINATE_ERROR("Failed to get IArgusCaptureMetadata interface.");
        CaptureMetadata *metadata = iArgusCaptureMetadata->getMetadata();
        ICaptureMetadata *iMetadata = interface_cast<ICaptureMetadata>(metadata);

Problem there is the Image returned from Frame::getImage only supports mapping for CPU access and i need Gpu access.

class IImage : public Argus::Interface
{
...
    /**
     * Maps a buffer for CPU access and returns the mapped pointer.
     * How this data is laid out in memory may be described by another Frame interface,
     * or if the pixel format is UNKNOWN then it should be defined by the stream's producer.
     * @param[in] index The buffer index to map.
     * @param[out] status An optional pointer to return an error status code.
     */
    virtual const void* mapBuffer(uint32_t index, Argus::Status* status = NULL) = 0;

    /**
     * Maps the first/only buffer for CPU access and returns the mapped pointer.
     * How this data is laid out in memory may be described by another Frame interface,
     * or if the pixel format is UNKNOWN then it should be defined by the stream's producer.
     * @param[out] status An optional pointer to return an error status code.
     */
    virtual const void* mapBuffer(Argus::Status* status = NULL) = 0;

Are there any other ways to get at the CaptureMetadata for a given capture? I am using cuEGLStreamConsumerAcquireFrame to get a frame from the OutputStream. I like the FrameConsumer class. Unfortunately it doesn’t suit my purposes wihout something I can use with CUDA.

3

What do you mean without display? Didn’t connect HDMI display but Xserver aka display driver still loaded? If you should be able run the sample code without display but display driver loaded. Like run the app from the ssh.

You need export DISPLAY=:0 before run the sample code from ssh connection.

4

@ShaneCCC

So, i’m using the examples to create something new. I have a camera library that can capture a raw bayer frame to GpuMat, so mission half accomplished. Issue is that now I want metadata along with that. The problem is here:

UniqueObj<EGLStream::MetadataContainer> metadataContainer(
    EGLStream::MetadataContainer::create(g_display.get(), m_eglStream));

It seems that in order to create a metadata container, is it mandatory to have a display (g_display.get() is an EGLDisplay (a void*)), but my camera library is not designed to require one, so I could use this if nullptr was acceptable for that parameter. To me, it doesn’t seem logical to attach metadata to the stream and display instead of individual frames. The Frame created by FrameConsumer seems to address this (provides IArgusCaptureMetadata) but only provides a CPU, not GPU buffer. I’m hoping there’s another way, since I’m currently blocked without metadata.

5

I think you should use argus API to get the metadata. And for the buffer with GPU you can reference to the multimedia sample code cudaHistogram/cudaBayerDemosaic

6

Thanks. I can already get a GPU buffer. I just can’t easily get metadata at the same time. Do you have an example to do that?

7

Have a check the source of the camera_argus

8

Thanks fro the tip. I took a look at the reference app, however when I run it, the app freezes when switching color formats (to P016, which seems to be the only thing available). Likewise, raw, Bayer, capture is not available, and dumping YUV data to file is not useful to our use case.

My client’s needs are:

  • raw capture
  • mappable to cv::GpuMat with few if any copies (latency matters, since OpenCV controls camera movement)
  • metadata

If there were something like a raw + processed capture example, that would work, but I can’t find one.

In summary, I have tried:

  • cudaBayerDemosaic but the code does not produce a correct result out of the box.
  • mapping bayer data to GpuMat and demosaicing in OpenCV, This does work, but capture of metadata is not tied to a particular buffer unless I use FrameConsumer which only provides Cpu allocated buffers per the docs (pasted above).
  • argus_camera only lists two YUV formats and locks up when using one of them.