NvJPEGDecoder cannot be reused for varying size images in JetPack 5.0.2

Hi, we are migrating the usage of MMAPI to JetPack 5.x from 4.x, and we use /usr/src/jetson_multimedia_api/include/NvJpegDecoder.h for JPEG decoding purpose.

We found that if we use the same decoder to decode multiple JPEG images with different sizes, the later ones will fail to decode.

Here attached the demo code:

#include <fstream>
#include <memory>
#include <cassert>
#include "NvJpegDecoder.h"
#include "NvJpegEncoder.h"
#include "nvbufsurface.h"
#include "nvbufsurftransform.h"

void test_decode(std::shared_ptr<NvJPEGDecoder> dec, const std::string &path, int index) {
    // read
    std::ifstream in(path);
    in.seekg(0, std::ios::end);
    const std::streamsize size = in.tellg();
    std::string data;
    data.resize(size);
    in.seekg(0, std::ios::beg);
    in.read(const_cast<char *>(data.data()), size);
    in.close();

    // decode
    int decoded_fd = -1;
    uint32_t pixfmt = 0;
    uint32_t width = 0;
    uint32_t height = 0;
    int ret = dec->decodeToFd(decoded_fd, (unsigned char *) data.data(), data.size(), pixfmt, width, height);
    assert(ret == 0);

    NvBufSurface *decoded_surf = nullptr;
    ret = NvBufSurfaceFromFd(decoded_fd, (void **) &decoded_surf);
    assert(ret == 0 && decoded_surf != nullptr);

    // allocate
    NvBufSurfaceAllocateParams create_params;
    memset(&create_params, 0, sizeof(create_params));
    create_params.params.width = width;
    create_params.params.height = height;
    create_params.params.memType = NVBUF_MEM_SURFACE_ARRAY;
    create_params.params.layout = NVBUF_LAYOUT_PITCH;
    create_params.params.colorFormat = NVBUF_COLOR_FORMAT_YUV420;
    create_params.memtag = NvBufSurfaceTag_VIDEO_CONVERT;
    
    NvBufSurface *cropped_surf;
    ret = NvBufSurfaceAllocate(&cropped_surf, 1, &create_params);
    assert(ret == 0);

    // crop
    NvBufSurfTransformParams transform_params;
    memset(&transform_params, 0, sizeof(transform_params));
    NvBufSurfTransformRect src_rect;
    src_rect.left = 0;
    src_rect.top = 0;
    src_rect.width = width;
    src_rect.height = height;
    transform_params.transform_flag = NVBUFSURF_TRANSFORM_CROP_SRC;
    transform_params.src_rect = &src_rect;
    ret = NvBufSurfTransform(decoded_surf, cropped_surf, &transform_params);
    assert(ret == 0);

    // encode
    int cropped_fd = cropped_surf->surfaceList[0].bufferDesc;
    auto enc = std::shared_ptr<NvJPEGEncoder>(NvJPEGEncoder::createJPEGEncoder("enc"));
    unsigned long buf_size = width * height * 3 / 2;
    unsigned char *buf = new unsigned char[buf_size];
    ret = enc->encodeFromFd(cropped_fd, JCS_YCbCr, &buf, buf_size, 90);
    assert(ret == 0);
    std::ofstream out("out_" + std::to_string(index) + ".jpg");
    out.write((char *) buf, buf_size);
    delete[] buf;

    ret = NvBufSurfaceDestroy(cropped_surf);
    assert(ret == 0);
}

int main(int argc, char *argv[]) {
    auto dec = std::shared_ptr<NvJPEGDecoder>(NvJPEGDecoder::createJPEGDecoder("dec"));
    for (int i = 1; i < argc; ++i) {
        test_decode(dec, argv[i], i);
    }
}

Usage:

./jpeg_decode xxx.jpg yyy.jpg

The output would be like:
image

If we create a new decoder every time, it just works fine:

int main(int argc, char *argv[]) {
    for (int i = 1; i < argc; ++i) {
        auto dec = std::shared_ptr<NvJPEGDecoder>(NvJPEGDecoder::createJPEGDecoder("dec"));
        test_decode(dec, argv[i], i);
    }
}

However, this is not required in JetPack 4.4.

Since it’s not appropriate to create a new decoder per image, anything we can do to fix this problem?

Hi,
We will check if we can support the case in later release. For information, do you use latest Jetpack 5.1.1 and observe the phenomenon?

Not yet, we don’t have a JP5.1.1 env for the moment. We’re curious why JP4.4 could support this (with green extra pixels).

In addition, some of our business envs are using JP5.0.2 and cannot be fully upgraded for some reasons (yet replacing several so files might be acceptable), hence we have to support JP5.0.2 in any case.

Hi,
We run the commands on Jetpack 5.1.1 and the decoded YUVs are correct:

$ gst-launch-1.0 videotestsrc num-buffers=1 ! jpegenc ! filesink location=test1.jpg
$ gst-launch-1.0 videotestsrc num-buffers=1 ! video/x-raw,width=640,height=480 ! jpegenc ! filesink location=test2.jpg
$ gst-launch-1.0 videotestsrc num-buffers=1 ! video/x-raw,width=1280,height=720 ! jpegenc ! filesink location=test3.jpg
06_jpeg_decode$ ./jpeg_decode num_files 3 test1.jpg test1.yuv test2.jpg test2.yuv test3.jpg test3.yuv

Please give it a try.

The decoded YUVs in test2.yuv and test3.yuv are incorrect:

$ hexdump test1.yuv | head -10
0000000 ebeb ebeb ebeb ebeb ebeb ebeb ebeb ebeb
*
0000020 ebeb ebeb ebeb ebeb ebeb eaea d2ec d2d1
0000030 d2d2 d2d2 d2d2 d2d2 d2d2 d2d2 d2d2 d2d2
*
0000050 d2d2 d2d2 d2d2 d2d2 d2d2 a8d3 a7ab a9ac
0000060 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
*
0000080 aaaa aaaa aaaa aaaa 8eab 9092 8f92 9192
0000090 9191 9191 9191 9191 9191 9191 9191 9191

$ hexdump test2.yuv | head -10
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
004b000 8080 8080 8080 8080 8080 8080 8080 8080
*
004b020 8080 8080 8080 8080 847f 817b 4881 1010
004b030 1010 1010 1010 1010 1010 1010 1010 1010
*
004b050 1010 1010 1010 1010 0e0f a710 a3a7 a6ac
004b060 a6a6 a6a6 a6a6 a6a6 a6a6 a6a6 a6a6 a6a6
*

$ hexdump test3.yuv | head -10
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
00e1000 8080 8080 8080 8080 8080 8080 8080 8080
*
00e1050 8080 8080 8080 8080 7f82 2b82 150d 0f0f
00e1060 1010 1010 1010 1010 1010 1010 1010 1010
*
00e10b0 100d 0d13 1010 9835 9a9a 9a9a 9a9a 9a9a
00e10c0 9a9a 9a9a 9a9a 9a9a 9a9a 9a9a 9a9a 9a9a
*

Repeated eb bytes are expected, not 00.

Hi,
Please try Jetpack 5.1.1. We can see decoded test2.yuv like:

~$ hexdump test2.yuv | head -10
0000000 ebeb ebeb ebeb ebeb ebeb ebeb ebeb ebeb
*
0000050 ebeb ebeb ebeb ebeb eceb d0eb d3d2 d2d2
0000060 d2d2 d2d2 d2d2 d2d2 d2d2 d2d2 d2d2 d2d2
*
00000b0 d2d2 d2d2 d1d2 aaaa aaaa aaaa aaaa aaaa
00000c0 aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa
*
0000110 a9aa 9290 9390 9190 9191 9191 9191 9191
0000120 9191 9191 9191 9191 9191 9191 9191 9191

And can see correct result in 7yuv viewer.

Verified, seems it has been fixed in JP5.1.1, the related so file is libnvjpeg.so, would it possible be back-ported to 5.0.2?

Hi,
We would suggest upgrade to latest Jetpack 5.1.1. For using 5.0.2, if the resolution of JPEG files varies, please create/destroy NvJPEGDecoder in decoding each file.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.