Argus: Getting timestamp for frames acquired by cuEGLStreamConsumerAcquireFrame

Hello,

I have been experimenting with the cudaHistogram and the cudaBayerDemosaic examples in the tegra_multimedia_api code on my Jetson Xavier. I’m trying to get the timestamp and get acces to the pixel data on the planes.

If I use FrameConsumer::acquireFrame, and then cast the Frame to an IFrame, I can use getTime() to get the timestamp. However, I haven’t found any functions on the IFrame or IFrame2D interfaces that allow you to access the data - all I see is a way to get the fd for a buffer.

In the CUDA examples, I can use cudaEGLStreamConsumerAcquireFrame to pull an image from the EGL stream. I can map the cudaResource to a cudaArray, so I can access the data. However, I can’t see how I can get the timestamp using this method.

Any suggestions? I’m looking for a way to get the frame timestamp from the cudaEGLStreamComsumerAcquireFrame call (ex: cudaHistogram or cudaDebayerMosaic examples), OR, a way to access the pixel data and planes using the FrameConsumer/Frame/IFrame method.

Thanks!

Hi,
It is a limitation. Please check similar topics:
https://devtalk.nvidia.com/default/topic/1055835/jetson-tx2/timestamp-from-cueglframe/post/5352952/#5352952
https://devtalk.nvidia.com/default/topic/1042097/jetson-tx2/creating-a-working-nveglstreamsrc-gt-omxh264enc-gstreamer-pipeline/post/5287243/#5287243

Have a check yuvJpeg sample code.

./samples/yuvJpeg/main.cpp:                       static_cast<unsigned long long>(iMetadata->getSensorTimestamp()),

Hi BareMetalCoder,
If you are using CudaConsumer you can get the Metadata from the EGLStream using the MetadataContainer object. Please check the following code snippet.
This sample has been implemented in the CudaBayerDemosaicConsumer::threadExecute() function.

+ UniqueObj<EGLStream::MetadataContainer> container(MetadataContainer::create(<replace EGLDisplay>, <replace EGLStream>);
+ IArgusCaptureMetadata *iArgusMeta = interface_cast<IArgusCaptureMetadata>(container.get());
+ if (!iArgusMeta)
+    ORIGINATE_ERROR("Failed to create Metadata Container");
+ CaptureMetadata *metadata = iArgusMeta->getMetadata();
+ ICaptureMetadata *iCaptureMeta = interface_cast<ICaptureMetadata>(metadata);
+ if (!iCaptureMeta)
+    ORIGINATE_ERROR("Unable to get Capture Metadata from frame");
+ uint64_t sensorTimeStamp = iCaptureMeta->getSensorTimestamp();

// On the first frame, print the information contained in the CUDA EGL frame structure.
if (frameID == 0)
{

Hi Waiss,

Thanks for the suggestion - I had already tried that, but I got an error from src/eglstream/MetadataContainerImpl.cpp:

(Argus) Error InvalidState: Frame does not have data

Unfortunately, this error is in the argus library so I can’t dig any deeper.

I am using EGL_NO_DISPLAY – could that be a problem?

Ok, found it. I didn’t know I had to enable metadata while setting up my EGLStream! EGL_NO_DISPLAY causes no problems.

IEGLOutputStreamSettings * iEGLStreamSettings = interface_cast<IEGLOutputStreamSettings>(streamSettings);
if (!iEGLStreamSettings) {
    ORIGINATE_ERROR("Failed to create OutputStreamSettings");
<b>iEGLStreamSettings->setMetadataEnable(true);</b>

Waiss, You saved me a lot of time, as I was preparing to reimplement our pipeline using the suggestions that preceded your comment. I am deeply thankful.

Hi BareMetalCoder,
Aplologies for the late response looks like you have already figured out the change.
Anyways I am glad that I could be of your help. :)

Hi guys,

I tried the MetadataContainer::create method as proposed in this thread.
It works great for one stream, but if I use two streams it always returns the metadata of one stream.
e.g. if you adapt the syncSensor to use the following code, it will return the same metadata twice

    UniqueObj<EGLStream::MetadataContainer> metadataContainerLeft(
        EGLStream::MetadataContainer::create(m_leftStream->getEGLDisplay(), m_leftStream->getEGLStream()));
    EGLStream::IArgusCaptureMetadata *iArgusCaptureMetadataLeft =
        interface_cast<EGLStream::IArgusCaptureMetadata>(metadataContainerLeft);
    if (iArgusCaptureMetadataLeft)
    {
        CaptureMetadata *metadata = iArgusCaptureMetadataLeft->getMetadata();

        ICaptureMetadata *iMetadata = interface_cast<ICaptureMetadata>(metadata);
        if (!iMetadata)
            ORIGINATE_ERROR("Failed to get ICaptureMetadata interface.");
        CONSUMER_PRINT("\tSensor Timestamp: %llu, LUX: %f\n",
                   static_cast<unsigned long long>(iMetadata->getSensorTimestamp()),
                   iMetadata->getSceneLux());
    }

    UniqueObj<EGLStream::MetadataContainer> metadataContainerRight(
        EGLStream::MetadataContainer::create(m_rightStream->getEGLDisplay(), m_rightStream->getEGLStream()));
    EGLStream::IArgusCaptureMetadata *iArgusCaptureMetadataRight =
        interface_cast<EGLStream::IArgusCaptureMetadata>(metadataContainerRight);
    if (iArgusCaptureMetadataRight)
    {
        CaptureMetadata *metadata = iArgusCaptureMetadataRight->getMetadata();
        ICaptureMetadata *iMetadata = interface_cast<ICaptureMetadata>(metadata);
        if (!iMetadata)
            ORIGINATE_ERROR("Failed to get ICaptureMetadata interface.");
        CONSUMER_PRINT("\tSensor Timestamp: %llu, LUX: %f\n",
                   static_cast<unsigned long long>(iMetadata->getSensorTimestamp()),
                   iMetadata->getSceneLux());
    }
1 Like

Hi Nico,

Please help to open a new topic if it’s still an issue. Thanks