RGBA -> NV12 -> Encoder issues (Xavier-NX 35.5.0)

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
1

Hello,

I currently have an implementation with this flow:
I have a cuda RGBA frame (2020x519)

I Create a NVBufSurface

NvBufSurfaceAllocateParams base_params = {0};
base_params.params.gpuId = 0;
base_params.params.height = 519;
base_params.params.width = 2020;
base_params.params.memType = NVBUF_MEM_CUDA_UNIFIED;
base_params.params.colorFormat = NVBUF_COLOR_FORMAT_RGBA
base_params.params.layout = NVBUF_LAYOUT_PITCH;
NvBufSurfaceAllocate(&surface, 1, &base_params)

Result is height 519, pitch 8207 (expected 8080 [2020*4]) so we have some extra padding in the rows

in order to get RGBA cuda frame to the bvBufSurface we use MemCpy2d:

auto res = cudaMemcpy2D(dst_buf->GetPtr(), 8207, 
                            cuda_buffer->GetPtr(), 8080, 
                            8080, 519, cudaMemcpyHostToDevice);

Then, to prepare for the encoder we need to create a NV12 frame

NvBufSurfaceAllocateParams base_params = {0};
base_params.params.gpuId = 0;
base_params.params.height = 519;
base_params.params.width = 2020;
base_params.params.memType = NVBUF_MEM_CUDA_UNIFIED;
base_params.params.colorFormat = NVBUF_COLOR_FORMAT_NV12;
base_params.params.layout = NVBUF_MEM_SURFACE_ARRAY;
NvBufSurfaceAllocate(&surface, 1, &base_params))

Now, we need to Transform the RGBA→ NV12 to prepare for encoding… I am thinking the expectation is to do something like this:

NvCommonTransformParams transform_params;
    transform_params.src_top = 0;
    transform_params.src_left = 0;
    transform_params.src_width = 8207/h_;
    transform_params.src_height = h_;
    transform_params.dst_top = 0;
    transform_params.dst_left = 0;
    transform_params.dst_width = nv12_frame_buffer->GetSize()/519;
    transform_params.dst_height = 519;
    transform_params.flag = NVBUFSURF_TRANSFORM_FILTER;
    transform_params.flip = NvBufSurfTransform_None;
    transform_params.filter = NvBufSurfTransformInter_Default;

    const auto surf_in_dma_fd = surf_in->surfaceList[0].bufferDesc;
    const auto surf_out_dma_fd = surf_out->surfaceList[0].bufferDesc;

ret = NvBufSurf::NvTransform(&transform_params, surf_in_dma_fd, surf_out_dma_fd

Then we send the resulting NV12 frame to the NvVideoEncoder…

So as you can see we have some extra padding in the initial allocated RGBA NvBufSurface

We expect 8080 (2020*4) but end up with 8207 instead… I thought memcpy2d would resolve this as we have indicated the src pitch size and dest pitch size accordingly… But the end result after NVTransform is misaligned… Is there a step I am missing? For most resolutions I have tested this with I have not seen this misalignment … so I can only see it in this uncommon 2020x519 resolution (possibly others, but not yet found on my side).

How can I fix this alignment issue? Maybe a different intermediate conversion before encoder even might be safer?

3

Hi,
It may not work properly if either width or height is in odd value. If the fame data is in 2020x519, we would allocate NvBufSurface in 2020x520 and send it to encoder. You would need to pad the last line.

4

Thank you, that is the solution! we are able to get the encoder to produce an expected video frame, albeit at a slightly higher resolution 2020x520…

In this case after encoding we will need to crop the image and remove this padding… At the moment of setDQThreadCallback we have a NVBuffer, is there a safe way we can remove this extra padding here before we write it to any streams?

5

@DaneLLL I am guessing maybe not this is not possible?

6

Hi,
It is not possible. Encoded H264(or H265) bitstream is in 2020x520.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.