[MMAPI sample] Something wrong in initializing NvVideoConvert in video_convert_main.cpp

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

I’m developing a video streaming server on TX1,
and making a function to change the image resolution using NvVideoConverter.
Then I’m refering a MMAPI sample.

To analyze a sample code, I suspect that this code of initializing has a bug.

video_convert_main.cpp (MMAPI sample)
int main(int argc, char *argv)

// Set conv0 output plane format
    ret = ctx.conv0->setOutputPlaneFormat(ctx.in_pixfmt, ctx.in_width,
                ctx.in_height, V4L2_NV_BUFFER_LAYOUT_PITCH);
    TEST_ERROR(ret < 0, "Could not set output plane format for conv0", cleanup);

    if (ctx.in_buftype == BUF_TYPE_NVBL)
    {
    // Set conv0 capture plane format
    ret = ctx.conv0->setCapturePlaneFormat(ctx.in_pixfmt, ctx.in_width,
                ctx.in_height, V4L2_NV_BUFFER_LAYOUT_BLOCKLINEAR);
    }
    else if (ctx.out_buftype == BUF_TYPE_NVBL)
    {
    ret = ctx.conv0->setCapturePlaneFormat(ctx.out_pixfmt, ctx.out_width,
                ctx.out_height, V4L2_NV_BUFFER_LAYOUT_BLOCKLINEAR);
    }
    else
    {
    ret = ctx.conv0->setCapturePlaneFormat(ctx.out_pixfmt, ctx.out_width,
                ctx.out_height, V4L2_NV_BUFFER_LAYOUT_PITCH);
    }
    TEST_ERROR(ret < 0, "Could not set capture plane format for conv0", cleanup);

    if (ctx.in_buftype == BUF_TYPE_NVBL)
    {
    // Set conv1 output plane format
    ret =
        ctx.conv1->setOutputPlaneFormat(ctx.in_pixfmt, ctx.in_width,
                    ctx.in_height, V4L2_NV_BUFFER_LAYOUT_BLOCKLINEAR);
    }
    else if (ctx.out_buftype == BUF_TYPE_NVBL)
    {
    ret =
        ctx.conv1->setOutputPlaneFormat(ctx.out_pixfmt, ctx.out_width,
                    ctx.out_height, V4L2_NV_BUFFER_LAYOUT_BLOCKLINEAR);
    }
    TEST_ERROR(ret < 0, "Could not set output plane format for conv1", cleanup);

    if (ctx.conv1)
    {
        // Set conv1 capture plane format
        ret =
            ctx.conv1->setCapturePlaneFormat(ctx.out_pixfmt, ctx.out_width,
                    ctx.out_height, V4L2_NV_BUFFER_LAYOUT_PITCH);
        TEST_ERROR(ret < 0, "Could not set capture plane format for conv1",
                cleanup);
    }

This code:

  • When the input format is BL, capturePlane of conv0 and outputPlane of conv1 are set as BL.
  • When the output format is BL, capturePlane of conv0 and outputPlane of conv1 are set as BL.
  • Always outputPlane of conv0 and capturePlane of conv1 are set as PITCH.

I understand that the NvVideoConverter has some limitation about BLOCK_LINEAR(BL) format.
Then this sample code use 2 NvVideoConverter objects, and one of them is used for conversion between BL and pitch.

When input and/or output format is BL, 2 NvVideoConverter are connected bellow:
Input → conv0 → conv1 → Output

Question 1:

  • What is the limitation of BLOCK_LINEAR format ?

Question 2:

  • Does the sample code have a bug ?

I think

  • When the input format is BL, outputPlane of conv0 will be set as BL.
  • When the output format is BL, capturePlane of conv1 will be set as BL.
  • The other planes will be set as PITCH.
2

Hi mynaemi,
We will check 07_video_convert and get back to you.

Besides, please refer to the backend sample for an example of using the video converter.

3

Hi mynaemi,
Question 1:

  • What is the limitation of BLOCK_LINEAR format ?
    No limitation. You can do BLINEAR → BLINEAR conversion also.

Question 2:

  • Does the sample code have a bug ?
    Please refer to the design concept:
    ubuntu@tegra-ubuntu:~/tegra_multimedia_api/samples/07_video_convert$ ./video_convert ~/dump.yuv 640 480 YUV420M ~/640_480.yuv 640 480 YUV420M --input-nvbl
    conv0 out PITCH
    conv0 cap BLINEAR
    conv1 out BLINEAR
    conv1 cap PITCH

Here, actual requested conversion operation is happening with conv1, where conv1 output plane is set to blocklinear (corresponds to --input-nvbl cmdline).
Though in order to create the required blocklinear input for conv1, app uses conv0 where pitch linear buffer is provided as input & converted to blocklinear.

ubuntu@tegra-ubuntu:~/tegra_multimedia_api/samples/07_video_convert$ ./video_convert ~/dump.yuv 640 480 YUV420M ~/640_480.yuv 640 480 YUV420M --output-nvbl
conv0 out PITCH
conv0 cap BLINEAR
conv1 out BLINEAR
conv1 cap PITCH

Here, actual requested conversion operation is happening with conv0, where conv0 capture plane is set to blocklinear (corresponds to --output-nvbl cmdline).
conv0 captureplane provides requested converted blocklinear buffer though for verification purpose app uses conv1 where blocklinear buffer is again converted to pitch linear (which can be file dump)

Without ‘–input-nvbl’ and ‘–output-nvbl’, it runs
conv0 out PITCH
conv0 cap PITCH

The sample does not demonstrate BLINEAR → BLINEAR, which also works fine.

4

Hi DaneLLL,

I appreciate your support.
I’ve understood “No Limitation” and “No Bug”.

The reason that I thought “some limitations?” is:

[video_convert_main.cpp]

int
main(int argc, char *argv[])
{

    TEST_ERROR(ctx.in_buftype == BUF_TYPE_NVBL && ctx.out_buftype == BUF_TYPE_NVBL,
            "NV BL -> NV BL conversions are <b>not supported</b>", cleanup);

What does this mean ?

Before your reply,
I felt strange what is the meaning of using 2 instances of NvVideoConverter, if without any limitation.
In your answer,
With “–input-nvbl”, conv0 converts dump.yuv into BLOCK LINEAR format to create the required blocklinear input.
With “–output-nvbl”, conv1 converts the processed data into PITCH LINEAR format for verification purpose.

Unfortunately, I could not find “THE DESIGN CONCEPT”.
(1) tegra_multimedia_api/doc/html/ → NVIDIA Tegra Multimedia API Framework Documentation
(2) Tegra Linux Driver Package Developer Guide → API → MMAPI Sample Applications → Video Scaling and Color Format Conversions Sample
(3) tegra_multimedia_api/README

Would you teach me what document this concept is written in?

5

Hi mynaemi,
It should be ‘NV BL → NVBL is not supported in this sample’. We have confirmed it is supported and will improve the documentary.

6

Hi DaneLLL,

Thanks a lot!
I understood.