Argus timestamps not continuous

[MarkusHess]

Hi, I use the following function for capturing images and metadata from Argus using an EGLStream:

bool ArgusFrameConsumer::readNext(Image &image, uint64_t &timestamp, uint32_t timeout_us)
{
    CUgraphicsResource cudaResource = 0;
    CUstream cudaStream = 0;

    CUresult cuResult = cuEGLStreamConsumerAcquireFrame(&m_cudaConnection, &cudaResource, &cudaStream, timeout_us);

    if (cuResult != CUDA_SUCCESS)
    {
        std::cout << "Unable to acquire an image frame from the EGLStream: "
                         << getCudaErrorString(cuResult);
        return false;
    }

    CUeglFrame cudaEGLFrame;
    cuResult = cuGraphicsResourceGetMappedEglFrame(&cudaEGLFrame, cudaResource, 0, 0);
    if (cuResult != CUDA_SUCCESS)
    {
        std::cout << "Unable to get the CUDA EGL frame: " << getCudaErrorString(cuResult);
        return false;
    }

    // process
    readCudaEglFrame(cudaEGLFrame, image);

    Argus::UniqueObj<EGLStream::MetadataContainer> container(
        EGLStream::MetadataContainer::create(EGL_NO_DISPLAY, m_outputStream->getEGLStream()));
    EGLStream::IArgusCaptureMetadata *iArgusMeta = Argus::interface_cast<EGLStream::IArgusCaptureMetadata>(container.get());
    if (!iArgusMeta)
        throw(std::runtime_error("Unable to create MetadataContainer"));
    Argus::ICaptureMetadata *iMeta = Argus::interface_cast<Argus::ICaptureMetadata>(iArgusMeta->getMetadata());

    timestamp = iMeta->getSensorTimestamp();

    cuResult = cuEGLStreamConsumerReleaseFrame(&m_cudaConnection, cudaResource, &cudaStream);
    if (cuResult != CUDA_SUCCESS)
    {
        std::cout << "Unable to release the last frame acquired from the EGLStream:" << getCudaErrorString(cuResult);
        return false;
    }

    return true;
}

The function is called continuously inside a grabbing loop and the timestamp is printed:

uint64_t prevTimestamp = 0;
while(true)
{
    Image img;
    uint64_t timestamp;
    if (consumer->readNext(img, timestamp, ARGUS_CAPTURE_TIMEOUT))
    {
        std::cout << timestamp << std::endl;
        if (timestamp <= prevTimestamp)
            std::cout << "timestamp not continuous" << std::endl;

        prevTimestamp = timestamp;
    }
}

I noticed, that the timestamp sometimes jumps backwards:

10984269744000
10984294697000
10984319658000
10984344626000
10984369594000
10984394575000
10984444500000
10984469576000
10984419532000
timestamp not continuous
10984519474000
10984594310000
10984670081000
10984694229000
10984719258000
10984744123000
10984769086000
10984794079000
10984819026000

Is that a bug or do we something wrong here?

These are the output stream settings used:

iEGLStreamSettings->setPixelFormat(Argus::PIXEL_FMT_YCbCr_420_888);
iEGLStreamSettings->setResolution(resolution);
iEGLStreamSettings->setMetadataEnable(true);

[ShaneCCC]

Could you check the MMAPI sample of yuvJpeg.

Thanks

[MarkusHess]

Hi ShaneCCC, it happens also with the yuvJpeg sample. I just removed the CONSUMER_PRINT parts and added the following instead:

bool ConsumerThread::threadExecute()
{
...
    uint64_t prevTimestamp = 0;

    while (true)
    {
...
        uint64_t timestamp = iMetadata->getSensorTimestamp();
        std::cout << timestamp << std::endl;
        if (prevTimestamp > timestamp)
            std::cout << "timestamp error" << std::endl;
        prevTimestamp = timestamp;
...

I got:

147476118993000
147476143965000
147476343713000
147476218869000
timestamp error
147476193900000
timestamp error
147476368679000
147476168932000
timestamp error
147477542204000
147477567173000
147477692032000

Note: It seems to happen on high load/memory consumption.

[ShaneCCC]

What’s the BSP version?
Could you verify by boost the system by nvpmodel.

sudo nvpmodel -m 0
sudo jetson_clocks&

[MarkusHess]

Hi ShaneCCC,

I’m on L4T 32.7.4. nvpmodel and jetson_clocks are already enabled.

[ShaneCCC]

I would suggest to drop the frame when the timestamp is not continuous as temp solution.