NvJPEGEncoder brightness problem

[fire9953]

https://forums.developer.nvidia.com/t/nvjpegencoder-color-error/249073

DaneLLL
I met this problem too, and i have tryed –encode-fd ，the color still looks Covered with a layer of gray。

Is this a bug? Please give me some help, thanks.

        NvBufSurf::NvCommonAllocateParams params;
        params.memType = NVBUF_MEM_SURFACE_ARRAY;
        params.width = 1920;
        params.height = 1080;
        params.layout = NVBUF_LAYOUT_PITCH;
        params.colorFormat = NVBUF_COLOR_FORMAT_YUV420;
        params.memtag = NvBufSurfaceTag_VIDEO_CONVERT;

        int ret = NvBufSurf::NvAllocate(&params, 1, &src_dma_fd);

        ......... //  read_dmabuf

        NvJPEGEncoder *jpegenc = NvJPEGEncoder::createJPEGEncoder("jpenenc");
        jpegenc->enableProfiling();
        jpegenc->encodeFromFd(src_dma_fd, JCS_YCbCr, &out_buf, out_buf_size, 95);

My L4t info:
R35 (release), REVISION: 4.1, GCID: 33958178, BOARD: t186ref, EABI: aarch64, DATE: Tue Aug 1 19:57:35 UTC 2023

yuv file (yuv420，1920 * 1080)
i420.yuv.zip (1.5 MB)

This is encode with opencv:

This is encode with NvJPEGEncoder:

[DaneLLL]

H,
Please send frame data in NVBUF_COLOR_FORMAT_YUV420_ER(or NVBUF_COLOR_FORMAT_NV12_ER) to NvJPEGEncoder and check again.

[fire9953]

DaneLLL I have tryed this, But still have this problem.

params.colorFormat = NVBUF_COLOR_FORMAT_YUV420_ER;

I guess maybe NvJPEGEncoder did something extra, eg. gamma correction ？

[DaneLLL]

Hi,
NVBUF_COLOR_FORMAT_YUV420_ER is with data range [0, 255]. Please ensure the frame data is in the range instead of [16, 235]

[fire9953]

I have checked my YUV frame data , the data range is [0, 255], it looks no difference whether i choose NVBUF_COLOR_FORMAT_YUV420 or NVBUF_COLOR_FORMAT_YUV420_ER. The NV encoded jpeg still looks different with the OpenCV encoded jpeg. And i don’t know why and how to fix this issue;

[DaneLLL]

Hi,
Please share a patch to the sample:

/usr/src/jetson_multimedia_api/samples/05_jpeg_encode/

So that we can replicate the issue and check.

And latest Jetpack release is 6.0GA(r36.3). Would be great if you can upgrade to latest version and try.

[fire9953]

It can be replicate like this,
1，build /usr/src/jetson_multimedia_api/samples/05_jpeg_encode/
2，Run this command:( i420.yuv is in the attachment i420.yuv.zip)

./jpeg_encode  ./i420.yuv 1920 1080 ./420-nv.jpeg --encode-fd -quality 95 -f 1

3，Write a simple opencv encode jpeg code, and encode the i420.yuv to jpeg

#include <fstream>
#include <iostream>
#include <opencv2/opencv.hpp>
#include <vector>
#include <iomanip>
#include <chrono>

using namespace std;

void writeToFile(const std::string &filePath, const std::vector<uint8_t> &data) {
    std::ofstream file(filePath, std::ios::binary);
    if (!file) {
        throw std::runtime_error("Unable to open file for writing: " + filePath);
    }

    file.write(reinterpret_cast<const char *>(data.data()), data.size());

    if (!file) {
        throw std::runtime_error("Error occurred when writing to file: " + filePath);
    }
}

cv::Mat readYUV420ToBGR(const std::string &filename, int width, int height) {
    std::ifstream file(filename, std::ios::binary);
    if (!file) {
        throw std::runtime_error("Unable to open file: " + filename);
    }

    std::vector<uint8_t> buffer(width * height * 3 / 2);
    file.read(reinterpret_cast<char *>(buffer.data()), buffer.size());

    cv::Mat yuv420(height * 3 / 2, width, CV_8UC1, buffer.data());
    cv::Mat bgr;
    cv::cvtColor(yuv420, bgr, cv::COLOR_YUV2BGR_I420);
    return bgr;
}

int main(int argc, char* argv[]) {
    if (argc < 2) {
        std::cerr << "Usage: " << argv[0] << " <option>\n";
        std::cout << "./cvtest -YUV420tojpg 1080filePath" << std::endl;
        return 1;
    }

    if(std::string(argv[1]) == "-YUV420tojpg") {
        std::string inputFilePath = argv[2];
        std::string outputFilePath = inputFilePath.substr(0, inputFilePath.find_last_of(".")) + "_cv.jpg";
        std::cout << "YUV420tojpg output:" << outputFilePath << std::endl;
        cv::Mat image = readYUV420ToBGR(inputFilePath, 1920, 1080);
        std::vector<uchar> buffer;
        cv::imencode(".jpg", image, buffer);
        writeToFile(outputFilePath, buffer);
    }

    return 0;
}

4，Compare the two jpeg images, and you will see the image color difference.

[DaneLLL]

Hi,
Do you adopt our suggestion to change format to YUV420_ER in 05_jpeg_encode sample? And please not to do format conversion in OpenCV. In OpenCV, the conversion seems to be non-linear and this may trigger deviation.

Please compare encoding YUV420 in both cases.

[fire9953]

yes, i have tested both YUV420_ER and YUV420 in 05_jpeg_encode sample, but the two encoded jpeg looks no difference.

[DaneLLL]

Hi,
The attached i420.yuv is in NVBUF_COLOR_FORMAT_YUV420_ER(full range [0,255]). So you need to allocate both src_dma_fd and dst_dma_fd in NVBUF_COLOR_FORMAT_YUV420_ER/NVBUF_LAYOUT_PITCH. And the encoded JPEG will be in full range. You can then decode the JPEG and check the value pixel by pixel.

[fire9953]

@DaneLLL

So you need to allocate both src_dma_fd and dst_dma_fd in NVBUF_COLOR_FORMAT_YUV420_ER/NVBUF_LAYOUT_PITCH

Yes，i have tested this，the nv-encode jpeg still looks difference with opencv-encode jpeg, and the opencv-encode jpeg looks little better.

[DaneLLL]

Hi,
We again compare the original YUV and decode YUV. And the brightness is identical. It seems to be an issue in the OpenCV function that it applies non-linear conversion to frame data in bt601 YUV420 [0,255]. And converted BGR gets brighter. The brightness is changed while comparing to the original YUV.

[fire9953]

DaneLLL
If it is issue in the OpenCV function , i think this problem can decrease priority.
But I still have some doubts, in this link:NvJpegEncoder color error
the ffmpeg-encoded jpeg looks same, much closer to the opencv-encoded jpeg?

[DaneLLL]

Hi,
In the topic, the color range [0,255] or [16,235] may not be correctly set. It shall be correct to allocate NVBUF_COLOR_FORMAT_YUV420_ER for [0,255] data and NVBUF_COLOR_FORMAT_YUV420 for [16,235] data.