NV12 saving by EGLStream::IImageHeaderlessFile is different than saving from dma buffer

Hi, I’m using TX2 NX & JP 4.6.3.

I’m having an issue with video when looking at very high dynamic range area (very bright part in corner, and the rest is dark). I saw some “wavy effect” or “color banding effect”. Try to narrow down where it comes from and see that it starts happening since a Nv Buffer is created from IImageNativeBuffer interface.

EGLStream::NV::IImageNativeBuffer *iNativeBuffer =
                Argus::interface_cast<EGLStream::NV::IImageNativeBuffer>(image);
int new_fd = iNativeBuffer->createNvBuffer(size,
                                                               NvBufferColorFormat_NV12,
                                                               NvBufferLayout_Pitch);

I modified argus_camera code to save two NV12 images from single capture in two ways (using IImageHeaderlessFile::writeHeaderlessFile and using Nv Buffer.
This is the patch file: argus_camera.patch (18.6 KB)
basically, when capturing still image with NV12 type, after NV12 file is stored as original way, I created a Nv Buffer, copy image to this buffer, then save it as NV12 file (referenced from 07_video_convert example How to save YUV420 buffer with Pitch layout into YUV file?). So these two NV12 files are coming from the same capture.

case STILL_FILE_TYPE_HEADERLESS:
        {
            // Get the HEADERLESS_FILE interface.
            EGLStream::IImageHeaderlessFile *iHeaderlessFile =
                Argus::interface_cast<EGLStream::IImageHeaderlessFile>(image);
            if (!iHeaderlessFile)
                ORIGINATE_ERROR("Failed to get IImageHeaderlessFile interface.");

            EGLStream::IImage2D *i2D =
                Argus::interface_cast<EGLStream::IImage2D>(image);
            if (!i2D)
                ORIGINATE_ERROR("Failed to get IImage2D interface.");
            const Argus::Size2D<uint32_t> size = i2D->getSize();

            // build the file name
            std::ostringstream fileName;
            fileName << dispatcher.m_outputPath.get();
            if (dispatcher.m_outputPath.get() != "/dev/null")
            {
                fileName << "/image_" <<
                    size.width() << "x" << size.height() << "_" <<
                    std::setfill('0') << std::setw(4) << m_captureIndex <<
                    "." << dispatcher.m_captureYuvFormat.toString();
            }

            PROPAGATE_ERROR(validateOutputPath(fileName.str().c_str()));

            // Write a headerless, unencoded image to disk.
            if (iHeaderlessFile->writeHeaderlessFile(fileName.str().c_str()) == Argus::STATUS_OK)
            {
                PROPAGATE_ERROR(dispatcher.message("Captured a still image to '%s'\n",
                                                   fileName.str().c_str()));
            }
            else
            {
                ORIGINATE_ERROR("Failed to write headerless raw image to '%s'\n",
                                fileName.str().c_str());
            }

            EGLStream::NV::IImageNativeBuffer *iNativeBuffer =
                Argus::interface_cast<EGLStream::NV::IImageNativeBuffer>(image);
            if (!iNativeBuffer) {
                PROPAGATE_ERROR(dispatcher.message("IImageNativeBuffer not supported by Image\n"));
            }
            {
                int new_fd = iNativeBuffer->createNvBuffer(size,
                                                               NvBufferColorFormat_NV12,
                                                               NvBufferLayout_Pitch);
                fileName.str("");
                fileName.clear();
                fileName << dispatcher.m_outputPath.get();
                if (dispatcher.m_outputPath.get() != "/dev/null")
                {
                    fileName << "/image_" <<
                        size.width() << "x" << size.height() << "_" <<
                        std::setfill('0') << std::setw(4) << m_captureIndex <<
                        "_nvbuf." << dispatcher.m_captureYuvFormat.toString();
                }
                int ret = save_dmabuf_nv12(new_fd, fileName.str().c_str());
                printf("buffer fd %d created, saving into '%s' return %d\n", new_fd, fileName.str().c_str(), ret);
                NvBufferDestroy(new_fd);
            }

            dispatcher.printAllSettings(stillRequest.get(), stillStream.get());
        }
        break;

But the output I have is having color difference.
The file named image_1932x1090_0000.nv12 is saved by original way (using IImageHeaderlessFile::writeHeaderlessFile), and the another file image_1932x1090_0000_nvbuf.nv12 is saved from dma buffer as you can see in the code above.
image_1932x1090_0000.nv12 (3.0 MB)
image_1932x1090_0000_nvbuf.nv12 (3.0 MB)
Open these .nv12 files by vooya (Download vooya), I can see the file stored by IImageHeaderlessFile (the later one in these two images) is more smooth and brighter, the other image having some wavy effect on it.

int new_fd = iNativeBuffer->createNvBuffer(size,
                                                               NvBufferColorFormat_NV12,
                                                               NvBufferLayout_Pitch);

It looks like copying the image into Nv Buffer (by call above) reducing the image quality & light.

Is there any explaination why this happened? And how to avoid the quality lost when copy native buffer into NvBuffer?

Thank you so much.

Hi,
Please apply the prebuilt lib and check if the issue persists:
Jetson/L4T/r32.7.x patches - eLinux.org

[MMAPI] 07_video_convert sample result not match as expected

Hi Dane,

Thanks for the reply.
I tested the original nvbuf_util library and it doesn’t have problem with copying buffer. I did the test similarly to Copying YUV420 buffer using NvBufferTransform cause pixel value change - #7 by kazunori.kimura

However, the my issue above persists even with the patched library.

Hi,
Please try 07_video_convert and see if you can replicate the issue. If yes, please share the command so that we can check.

Hi Dane,

As I said in previous reply, 07_video_convert sample doesn’t have problem. The buffer copying is working fine.

The only problem is when this call is used to create NvBuffer from Native buffer, the NvBuffer has worse quality.

int new_fd = iNativeBuffer->createNvBuffer(size,
                                                               NvBufferColorFormat_NV12,
                                                               NvBufferLayout_Pitch);

Hi,
Please share a patch to 09_camera_jpeg_capture or 10_camera_recording so that we can replicate the issue.

Hi Dane,

This is updated 09_camera_jpeg code.
main.cpp (31.3 KB)
Compare to original app, I added one more option:

--nv12        Capture NV12 frame at specified index [Default -1]

The command I used:

camera_jpeg_capture --img-res 1932x1090 --nv12 20 --cap-time 2

The resolution is equal to my camera’s native resolution. Beside jpeg files as original app, this will save 20th buffer into two NV12 files, one saving from EGLImage using EGLStream::IImageHeaderlessFile (see CaptureConsumerThread::processEglImage() function), and another saving from NvBuffer (see CaptureConsumerThread::processV4L2Fd() function).

These are output images of that 20th frame. The jpeg file looks similar to _nvbuf.nv12 (NvBuffer) file, but the _eglimage.nv12 is brighter.


output020_1932x1090_eglimage.nv12 (3.0 MB)
output020_1932x1090_nvbuf.nv12 (3.0 MB)

Hi,

Please try this method and see if the color is expected. Please keep the default code:
m_dmabuf = iNativeBuffer->createNvBuffer(iEglOutputStream->getResolution(),
NvBufferColorFormat_YUV420,
NvBufferLayout_BlockLinear);
And call NvBufferTransform() to convert m_dmabuf to NV12 pitch linear.

Hi Dane,
Updated as your suggestion, but the result is still the same. EglImage is brighter than NvBuffer.
main.cpp (31.5 KB)

Hi @DaneLLL,
Do you have any update/suggestion on this or you are still working on it? Thank you.

Hi,
We are checking it. Will update when there is further finding.

1 Like

Hi,
Please create NvBuffer in NV12_ER:

m_dmabuf = iNativeBuffer->createNvBuffer(iEglOutputStream->getResolution(),
                                         NvBufferColorFormat_NV12_ER,
                                        NvBufferLayout_BlockLinear);

And call NvBufferTransform() to convert m_dmabuf to NV12_ER pitch linear.

1 Like

Hi Dane,

Thank you for pointing it out. This answers my problem. It’s even better if NvBufferColorFormat_NV12_709_ER is used.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.