Jetson TX2 omxh264enc pipeline differs from Jetson TX1 in C++ code

[anviksingh]

Hi All,
I have a working C++ code for Jetson TX1 (L4T r24) which does H.264 file compression on a raw gray scale movie and stores frames in an AVI container. The intention is to compress a raw movie file with different compression ratios to control the compressed movie size.
When I use the same code on TX2 (L4T r32) the same pipeline doesn’t work. There are several issues that I see on TX2, which are listed below.

1. I made the control-rate=0 for omxh264enc for TX1 (saw few suggestions on this forum to use control-rate for h264 encoder pipelines), when I use the same setting on TX2, it makes the video too choppy. So, in the code I made a compilation flag for TX1 only to use “control-rate”, and TX2 runs with default flag or control-rate=2.

2. I made “qp-range” = “10,50:10,50:0,1” for best video quality (which gives me ~4X compression) for our purpose, the same parameter for TX2 give me ~300X compression and choppy video.

3. If I use the “h264parse” in TX2 pipeline, it freezes the entire pipeline, I can see that the around 7 frames were read from the file, but none reached to compressed file and the pipeline froze, so I made it available only for TX1.
   So, now the pipeline works for TX2, but the video quality is too bad and compression is ~300X regardless
   of different qp-range.

4. I tune qp-range for TX2 experimentally to produce same compression ratio as TX1, but, now video file is short of few frames, for example, 170 frame raw movie would only generate 140 frames compressed movie (the code in PLATFORM_TX2 compilation flag). I am sending EOS when all frames from the source movie is consumed by the pipeline.

5. I don’t have a good understanding of “qp-range”, but we already have a solution with TX1 qp-range parameters, we now want to tune TX2 to give same compression ratio.

Any help is appreciated, to understand why I see so many differences on TX2 compared to TX1, is there any fundamental change on TX2 that I am missing ?

Please find the pipeline source code function below:

GstDisplayPtr GstApp::buildH264FileDisplay(uint32_t displayId, GstApp *app)
{
    GstElement *displayBin = NULL;
    GstElement *preQueue = NULL;
    GstElement *queue = NULL;
    GstElement *converter = NULL;
    GstElement *converterFilter = NULL;
    //GstElement *videorate = NULL;
    //GstElement *videorateFilter = NULL;
    GstElement *encoder = NULL;
    GstElement *encoderParser = NULL;
    GstElement *encoderFilter = NULL;
    GstElement *parse = NULL;
    GstElement *container = NULL;
    GstElement *containerFilter = NULL;
    GstElement *sink = NULL;
    GstPad *ghostPad = NULL;
    GstCaps *converterCaps = NULL;
    GstCaps *videorateCaps = NULL;
    GstCaps *encoderCaps = NULL;
    GstCaps *containerCaps = NULL;
    GstDisplayPtr display(new struct GstDisplay);

    uint32_t videorateMax = 120;
    display->id = displayId;
    display->filterCounter = 0;
    display->type = DisplayType::H264_FILE_D;
    display->bin = NULL;
    display->preFilter = NULL;
    display->postFilter = NULL;

    std::string binName = "H264FileBin_" + std::to_string(displayId);

    displayBin = gst_bin_new (binName.c_str());
    preQueue = gst_element_factory_make ("queue", NULL);
    queue = gst_element_factory_make ("queue", NULL);
    converter = gst_element_factory_make ("videoconvert", NULL);
    converterFilter = gst_element_factory_make ("capsfilter", NULL);
    //videorate = gst_element_factory_make("videorate", NULL);
    //videorateFilter = gst_element_factory_make ("capsfilter", NULL);
    encoder = gst_element_factory_make ("omxh264enc", NULL);
    encoderFilter = gst_element_factory_make ("capsfilter", NULL);
    encoderParser = gst_element_factory_make ("h264parse", NULL);
    container = gst_element_factory_make ("avimux", NULL);
//    container = gst_element_factory_make ("qtmux", NULL);
    sink = gst_element_factory_make ("filesink", NULL);

    uint32_t width = getDisplayWidth();
    uint32_t height = getDisplayHeight();

    fill_dr_matrix(width, height);

    std::string memTypeString = "video/x-raw";
    std::string formatTypeString = "format=NV12";
    std::string widthString = "width=" + std::to_string(width);
    std::string heightString = "height=" + std::to_string(height);
    //std::string framerate = "framerate=" + std::to_string(videorateMax);

    std::string converterCapsString = memTypeString + ',' +
            formatTypeString + ',' + widthString + ',' +
            heightString;

    converterCaps = gst_caps_from_string(converterCapsString.c_str());

    //videorateCaps = gst_caps_new_simple("video/x-raw",
    //            "framerate", GST_TYPE_FRACTION, videorateMax, 1,
    //            NULL);

    std::string encoderCapsString = "video/x-h264, stream-format=(string)byte-stream";
    encoderCaps = gst_caps_from_string(encoderCapsString.c_str());

    g_object_set (G_OBJECT (converterFilter), "caps", converterCaps, NULL);

    //g_object_set (G_OBJECT (videorateFilter), "caps", videorateCaps, NULL);
    g_object_set (G_OBJECT (encoderFilter), "caps", encoderCaps, NULL);

#if defined(PLATFORM_TX1)
    g_object_set (G_OBJECT (encoder), "control-rate", 0, NULL);

    switch(app->qfactor) {
    case COMP_QUALITY_FACTOR_BEST:
        g_object_set (G_OBJECT (encoder), "qp-range", "10,50:10,50:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_MODERATE:
        g_object_set (G_OBJECT (encoder), "qp-range", "20,50:20,50:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_GOOD:
        g_object_set (G_OBJECT (encoder), "qp-range", "25,50:25,50:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_AVERAGE:
        g_object_set (G_OBJECT (encoder), "qp-range", "30,50:30,50:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_FAIR:
        g_object_set (G_OBJECT (encoder), "qp-range", "35,50:35,50:0,1", NULL);
        break;
    default:
        LOG_INFO("quality param ", app->qfactor,", fallback to moderate");
        g_object_set (G_OBJECT (encoder), "qp-range", "20,50:20,50:0,1", NULL);
        break;
    }
#elif defined(PLATFORM_TX2)
    g_object_set (G_OBJECT (encoder), "control-rate", 2, NULL);

    switch(app->qfactor) {
    case COMP_QUALITY_FACTOR_BEST:
        g_object_set (G_OBJECT (encoder), "qp-range", "5,10:5,10:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_MODERATE:
        g_object_set (G_OBJECT (encoder), "qp-range", "15,20:15,20:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_GOOD:
        g_object_set (G_OBJECT (encoder), "qp-range", "20,23:20,23:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_AVERAGE:
        g_object_set (G_OBJECT (encoder), "qp-range", "23,29:23,29:0,1", NULL);
        break;
    case COMP_QUALITY_FACTOR_FAIR:
        g_object_set (G_OBJECT (encoder), "qp-range", "25,32:25,32:0,1", NULL);
        break;
    default:
        LOG_INFO("quality param ", app->qfactor,", fallback to moderate");
        g_object_set (G_OBJECT (encoder), "qp-range", "15,20:15,20:0,1", NULL);
        break;
    }
#endif

    g_object_set (G_OBJECT (encoder), "quant-i-frames", 0, NULL);
    g_object_set (G_OBJECT (encoder), "quant-p-frames", 0, NULL);

    /* override host field, else need to add new strings in cmd args */
    IsxcStorage *isxcStoragePtr = app->getIsxcStorage();
    const char *isxcFileName = isxcStoragePtr->isxcFileName.c_str();

    if(isxcFileName) {
        LOG_WARN("isxc file is : ", isxcFileName);
        g_object_set (G_OBJECT (sink), "location", (char *)isxcFileName, NULL);
    } else {
        LOG_WARN("isxc file is : ", isxcFileName, ", fixing to final.isxc");
        g_object_set (G_OBJECT (sink), "location", "final.isxc", NULL);
    }

    g_object_set (G_OBJECT (sink), "append", true, NULL);

    gst_bin_add_many (GST_BIN (displayBin), preQueue, queue, converter,
            converterFilter, encoder, encoderFilter,
#if defined(PLATFORM_TX1)
            encoderParser,
#endif
            container, sink, NULL);

    if(gst_element_link_many(preQueue, queue, converter, converterFilter,
            encoder, encoderFilter,
#if defined(PLATFORM_TX1)
            encoderParser,
#endif
            container,
            sink, NULL) != TRUE)
    {
        LOG_ERROR("Elements for H264 file sink could not be linked");
        return display;
    }

    ghostPad = gst_element_get_static_pad (preQueue, "sink");
    gst_element_add_pad (displayBin, gst_ghost_pad_new ("sink", ghostPad));
    gst_object_unref (GST_OBJECT (ghostPad));

    if(m_bitDepth == 16)
    {
        GstPad *pad = gst_element_get_static_pad (preQueue, "src");
        gst_pad_add_probe (pad, GST_PAD_PROBE_TYPE_BUFFER,
                (GstPadProbeCallback) GstApp::interceptDataProbeGray16Yc,
                (gpointer) app, NULL);
        gst_object_unref (pad);
    }

    LOG_INFO("H264 file sink built");

    display->bin = displayBin;
    display->preFilter = queue;
    display->postFilter = converter;
    display->sink = sink;
    display->fpsThread = std::thread(GstApp::trackFps, display);
    display->enabled = true;
    //display->videorateFilter = videorateFilter;

    return display;
}

[DaneLLL]

Hi,
Please use v4l2 plugins such as nvv4l2h264enc. We are deprecating omx plugins on r32 releases. Please refer to 5.12 GStreamer Plugin gst-omx Deprecated in release notes.

[anviksingh]

Hi @DaneLLL :
Thanks a lot for the reply, I am using r32.2 on TX2, somehow missed on the release notes about the omx plugins deprecation.
Using nvv4l2h264enc works fine so far for me, still testing.
I selected the qp-range and control-rate empirically for TX1 to get expected compression ratio, same values don’t give similar compression ratio on TX2, I am still tuning it for TX2.
Is there any document that can help me find what values are good for control-rate and qp-range or how to control compression ratio for file compression and why I need different values on both platforms (TX1 vs Tx2) to get similar compression ratio ?

[DaneLLL]

Hi,
There might be deviation in default setting when implementing v4l2 plugins. Generally q value of each frame impacts image quality and compression rate. Please do comparison through JMdecoder and check if the q values are different. Here is an example of using JM decoder:

Detected images are blurred even at 25FPS

Hi, Here is an example of using JM decoder: You can download it from Karsten Suehring

And you can try constant bit rate along with vbv-size:

H264 vs H265

Hi, For comparing h264 and h265, please use same YUV source and compare PSNR. For example, you may download the source: https://media.xiph.org/video/derf/y4m/park_joy_1080p50.y4m Run $ gst-launch-1.0 filesrc location= park_joy_1080p50.y4m ! y4mdec ! nvvidconv ! nvv4l2h264enc bitrate=14000000 control-rate=1 vbv-size=450000 ! h264parse ! qtmux ! filesink location= park_joy.mov $ gst-launch-1.0 filesrc location= park_joy_1080p50.y4m ! y4mdec ! nvvidconv ! nvv4l2h265enc bitrate=14000000 control…