Encoding varying-size and -format images with the same JPEG encoder produces incorrect result in Jetpack 5.1.1

Hi, we use the same NvJPEGEncoder to encode images with different sizes and format, other format will be transformed to YUV420 on necessary, the output is incorrect sometimes.

Here is a sample code to reproduce this problem:

#include "NvJpegEncoder.h"
#include "nvbufsurface.h"
#include "nvbufsurftransform.h"
#include <assert.h>
#include <stdlib.h>
#include <memory>
#include <iostream>
#include <fstream>

NvBufSurface *allocate(int width, int height, NvBufSurfaceColorFormat format) {
    NvBufSurfaceAllocateParams create_params;
    memset(&create_params, 0, sizeof(create_params));
    create_params.params.width = width;
    create_params.params.height = height;
    create_params.params.memType = NVBUF_MEM_SURFACE_ARRAY;
    create_params.params.layout = NVBUF_LAYOUT_PITCH;
    create_params.params.colorFormat = format;
    create_params.memtag = NvBufSurfaceTag_VIDEO_CONVERT;
    NvBufSurface *surf = nullptr;
    int ret = NvBufSurfaceAllocate(&surf, 1, &create_params);
    assert(ret == 0);
    surf->numFilled = 1;
    return surf;
}

void destroy(NvBufSurface *surf) {
    int ret = NvBufSurfaceDestroy(surf);
    assert(ret == 0);
}

void transform(NvBufSurface *src, NvBufSurface *dst) {
    NvBufSurfTransformParams transform_params;
    memset(&transform_params, 0, sizeof(transform_params));
    transform_params.transform_flag = NVBUFSURF_TRANSFORM_FILTER;
    transform_params.transform_filter = NvBufSurfTransformInter_Nearest;
    int ret = NvBufSurfTransform(src, dst, &transform_params);
    assert(ret == 0);
}

void encode(std::shared_ptr<NvJPEGEncoder> enc, NvBufSurface *surf, std::string name) {
    int width = surf->surfaceList[0].width;
    int height = surf->surfaceList[0].height;
    unsigned long buf_size = width * height * 3 / 2;
    unsigned char *buf = new unsigned char[buf_size];
    NvBufSurface *tmp;
    if (surf->surfaceList[0].colorFormat == NVBUF_COLOR_FORMAT_YUV420) {
        tmp = surf;
    } else {
        tmp = allocate(width, height, NVBUF_COLOR_FORMAT_YUV420);
        transform(surf, tmp);
    }
    int fd = tmp->surfaceList[0].bufferDesc;
    int ret = enc->encodeFromFd(fd, JCS_YCbCr, &buf, buf_size, 90);
    assert(ret == 0);
    if (tmp != surf) {
        destroy(tmp);
    }
    std::ofstream out("output_ " + name + ".jpg");
    out.write((char *) buf, buf_size);
    delete[] buf;
}

int main(int argc, char *argv[]) {
    auto enc = std::shared_ptr<NvJPEGEncoder>(NvJPEGEncoder::createJPEGEncoder("enc"));
    NvBufSurface *yuv420 = allocate(1920, 1080, NVBUF_COLOR_FORMAT_YUV420);
    int ret;
    ret = NvBufSurfaceMemSet(yuv420, 0, 0, 128);
    assert(ret == 0);
    ret = NvBufSurfaceMemSet(yuv420, 0, 1, 64);
    assert(ret == 0);
    ret = NvBufSurfaceMemSet(yuv420, 0, 2, 192);
    assert(ret == 0);
    encode(enc, yuv420, "yuv420");

    NvBufSurface *nv12s = allocate(1920 / 2, 1080 / 2, NVBUF_COLOR_FORMAT_NV12);
    transform(yuv420, nv12s);
    encode(enc, nv12s, "nv12s");
    destroy(nv12s);

    NvBufSurface *nv12 = allocate(1920, 1080, NVBUF_COLOR_FORMAT_NV12);
    transform(yuv420, nv12);
    encode(enc, nv12, "nv12");
    destroy(nv12);

    return 0;
}

To compile this sample:

g++ -std=c++11 -I/usr/src/jetson_multimedia_api/include \
    -I/usr/src/jetson_multimedia_api/include/libjpeg-8b \
    main.cc \
    /usr/src/jetson_multimedia_api/samples/common/classes/NvBuffer.cpp \
    /usr/src/jetson_multimedia_api/samples/common/classes/NvElement.cpp \
    /usr/src/jetson_multimedia_api/samples/common/classes/NvElementProfiler.cpp \
    /usr/src/jetson_multimedia_api/samples/common/classes/NvLogging.cpp \
    /usr/src/jetson_multimedia_api/samples/common/classes/NvJpegEncoder.cpp \
    -L/usr/lib/aarch64-linux-gnu/tegra \
    -lnvbufsurface -lnvbufsurftransform -lnvjpeg -o main

The incorrect image is output_nv12.jpg:

Which is expected to be pure orange.

FYI: if we create a new encoder for each image, everything is just fine.

Hi,
For information, do you have input frames with identical resolutions(identical width and height)? If width and height may vary, please re-create NvJpegEncoder when resolution changes.

Yes, the input resolution varies, but re-creating NvJpegEncoder is not required in Jetpack 4.x.

Hi,
Pease check the suggestion in
NvJPEGEncoder cannot be reused for varying size images in JetPack 5.0.2 - #9 by DaneLLL

That topic is about decoder (sorry about the typo in title), and it has been fixed in 5.1.1. Yet this on is about encoder in 5.1.1.

Hi,
The titile is updated:
NvJPEGDecoder cannot be reused for varying size images in JetPack 5.0.2

For this we would suggest re-create NvJpegEncoder once the resolution changes. Or keep multiple NvJpegEncoder classes for possible resolutions in the use-case.

Thanks for the suggestion.

We’ve indeed considered this approach, unfortunately it’s not applicable for our business, our images are cropped targets of which sizes are completely different, even LRU won’t contribute to the performance.

Would you kindly investigate what cause the different behavior between 4.x & 5.x?

Hi,
For Jetpack 4, it destroys and re-create the class when resolution changes in prebuilt software stack. Since we can implement the same in application level, would suggest add the handling on Jetpack 5 release.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.