NvVideoConverter artifacts appears

[peclatj]

Hello,

I have a problem with the NvVideoConverter on the Jetson AGX Xavier.
JetPack 4.5.1 : nvidia-l4t-jetson-multimedia-api_32.5.1-20210219084708_arm64
R32 (release), REVISION: 5.1, GCID: 27362550, BOARD: t186ref, EABI: aarch64, DATE: Wed May 19 18:16:00 UTC 2021

It seems that some pixels of my image are becoming 0 once converted to another format.

To simplify the problem to the maximum I convert the format of my image from V4L2_PIX_FMT_UYVY to V4L2_PIX_FMT_UYVY, and yet the artifacts are still appearing. As you can see on the pictures below:

Camera.bmp (4.5 MB)
VideoConverter.bmp (4.5 MB)

Here is my code without error handling.

m_inPixFmt = m_outPixFmt = V4L2_PIX_FMT_UYVY;
m_width = 1456;
m_height = 1088;

m_conv = NvVideoConverter::createVideoConverter("conv0");
m_conv->setOutputPlaneFormat(m_inPixFmt, m_width, m_height, V4L2_NV_BUFFER_LAYOUT_PITCH);
m_conv->setCapturePlaneFormat(m_outPixFmt, m_width, m_height, V4L2_NV_BUFFER_LAYOUT_PITCH);
m_conv->output_plane.setupPlane(V4L2_MEMORY_MMAP, m_outputPlaneBufferNbr, true, false);
m_conv->capture_plane.setupPlane(V4L2_MEMORY_MMAP, m_capturePlaneBufferNbr, true, false);
m_conv->output_plane.setStreamStatus(true);
m_conv->capture_plane.setStreamStatus(true);
m_conv->capture_plane.setDQThreadCallback(capturePlaneCallback);
m_conv->capture_plane.startDQThread(this);
m_conv->output_plane.setDQThreadCallback(outputPlaneCallback);
m_conv->output_plane.startDQThread(this);

// Enqueue all the empty capture plane buffers.
for (uint32_t buffer_idx = 0; buffer_idx < m_conv->capture_plane.getNumBuffers(); buffer_idx++)
{
    queueOutputBuffer(buffer_idx); // Queue capture plane buffers (output of the converter)
}

Then to write on the output plane buffer (input buffer of the converter)
I get the list of output plane buffers.

for(int bufferIndex = 0; bufferIndex < nBuffers; bufferIndex++)
{
    NvBuffer *pBuffer = inputBuffers[bufferIndex];
    VmbUchar_t *pMemory = pBuffer->planes[0].data;
    VmbInt64_t nBufferSize = pBuffer->planes[0].length;
    pBuffer->planes[0].bytesused = 3168256; // payload size of my image (width * height * 2 bytes per pixel)
}

And I write the image into my pMemory pointer.

My end goal is to convert the image into other format V4L2_PIX_FMT_YUV420M, but it has the same issue.
Also I use the same method to save the bmp file from the capture_plane.buffers.planes[0].data. So I’m pretty sure the issue is from the converter. I do it in the callback “capturePlaneCallback”.

Thank you in advance for your answers.

[peclatj]

I figured it out.

I forgot about the stride, my camera was dumping its buffer without respecting the stride imposed by the converter’s NvBuffer. And when I wrote the bitmap exporter I did not consider the stride as well.

I had to memcpy the buffer of my camera line per line to consider the padding of the stride. It is a waste of computational power however.

// Converter input buffer
NvBuffer* inputBuffer = m_pVideoConverter->getNextAvailableInputBuffer();
unsigned char* in_data = inputBuffer->planes[0].data;

// Converter input buffer format
NvBuffer::NvBufferPlaneFormat& fmt = inputBuffer->planes[0].fmt;
uint32_t stride = fmt.stride;
uint32_t bpp = fmt.bytesperpixel;
uint32_t width = fmt.width;
uint32_t height = fmt.height;
            
// Camera output buffer
VmbUchar_t* out_data;
pFrame->GetImage(out_data);
            
// Copy while respecting the stride
for(unsigned int y = 0; y < height; y++) {
    std::memcpy(in_data, out_data, width * bpp);
    in_data += stride;
    out_data += width * bpp;
}

inputBuffer->planes[0].bytesused = stride * height;

My problem is solved, hope it can help.

[kayccc]

Glad to know you resolved it, thanks for the update.