Hello,
i have a program that encodes a video input to jpg and mp4 files, i’m using NvJpegEncoder and NvVideoEncoder for encoding. The video input is converted with NvVideoConverter to a V4L2_PIX_FMT_YUV420M NvBuffer, this is then feed to NvVideoEncoder and NvJpegEncoder(encodeFromFd). In the image appended you can see the color shift, left is the video and right the image from the NvJpegEncoder. Looks like fog in the image, see also the histogram attached.
Hi,
PLease set q value to 100 or 99 in NvJpegEncoder and check again. It is 75 by default.
Hi DaneLL,
the quality parameter would only cause steps in the histogram, what i see is a compressed histogram.
I have found a way to correct the values. The input YUV value range [16, 235] vs [0, 255] (see Numerical approximations YUV - Wikipedia) of the NvJpegEncoder and the NvVideoEncoder are different.
NvBufferColorFormat_YUV420_ER is needed by NvJpegEncoder to get the colors right, maybe someone from NVIDIA @dusty_nv can confirm or help with a better solution than the following.
Unpleasant solution(because of an additional NvBufferTransform):
NvBufferCreateParams jpg_in_format = {0};
jpg_in_format.payloadType = NvBufferPayload_SurfArray;
jpg_in_format.width = 1920;
jpg_in_format.height = 1080;
jpg_in_format.layout = NvBufferLayout_BlockLinear;
jpg_in_format.colorFormat = NvBufferColorFormat_YUV420_ER;
jpg_in_format.nvbuf_tag = NvBufferTag_VIDEO_DEC;
NvBufferRect rect;
rect.top = 0;
rect.left = 0;
rect.width = 1920;
rect.height = 1080;
NvBufferTransformParams transform_params = {0};
transform_params.transform_flag = NVBUFFER_TRANSFORM_FILTER;
transform_params.transform_flip = NvBufferTransform_None;
transform_params.transform_filter = NvBufferTransform_Filter_Nearest;
transform_params.src_rect = rect;
transform_params.dst_rect = rect;
int jpg_in_fd;
NvBufferCreateEx (&jpg_in_fd, &jpg_in_format);
NvBufferTransform(buffer->planes[0].fd, jpg_in_fd, &transform_params);
Hi,
Is the source format in NvBufferColorFormat_YUV420[16,235] or NvBufferColorFormat_YUV420_ER[0,255]?
Form the description, the issue looks like if we feed [0,255] to NvJpegEncoder, it is automatically converted to[16,235].
Hello DaneLL,
yes the source format is NvBufferColorFormat_YUV420. Would be interesting what the NvJpegEncoder source Format is, NvBufferColorFormat_YUV420 or NvBufferColorFormat_YUV420_ER?
Hi,
Could you share how to do histogram comparison? So that we can try to generate JPEGs through 05_jpeg_encode and compare the before/after histograms.
Looks like [0, 255] is converted to [16, 235], and [16,235] is converted to more limited range. We will need to investigate this further. Please share us the tool and steps.
Hello DaneLL,
the histogram was made with Preview (macOS).
I also think there is something wrong with the ranges. I have reproduced the error in my application with the sample applications. The test file is a any 1920x1080 jpg file input.jpg. First i decode the jpg with 06_jpeg_decode to yuv(V4L2_PIX_FMT_YUV420M) and then encode it to jpg with 05_jpeg_encode, then i copy multiple test.yuv files to video.yuv to get a video, after this step i encode the video.yuv with 01_video_encode to a h264 video, ffmpeg is used to convert the h264 to a mp4 file. Then you have to open the file test.jpg and video.mp4 with a web browser in 2 taps and switch between the taps, then you can see the difference in the shadows and highlights.
I have also checked the input formatfor 05_jpeg_encode and 01_video_encode, which are both V4L2_PIX_FMT_YUV420M.
./jpeg_decode num_files 1 input.jpg test.yuv
../05_jpeg_encode/jpeg_encode test.yuv 1920 1080 test.jpg -f 1 -crop 0 0 1920 1080
cat tes.yuv >> video.yuv (multiple(30-40) times to get a video, my video.yuv is 306MB big)
../01_video_encode/video_encode video.yuv 1920 1080 H264 video.h264
ffmpeg -i video.h264 -vcodec copy -an video.mp4
I hope you can reproduce my problem with this steps!
Hi,
Would be great if there is tool on Windows. If you know any, please share the information.
Hi,
We don’t observe the issue in comparing the result of software encoder and hardware encoder on r32.3.1/TX2.
// generate YUV data
$ gst-launch-1.0 videotestsrc num-buffers=1 ! video/x-raw,width=1920,height=1080 ! filesink location=I420.yuv
// encode with software encoder
$ gst-launch-1.0 filesrc location= I420.yuv ! videoparse format=2 width=1920,height=1080 ! jpegenc ! filesink location=a.jpg
// encode with hardware encoder
$ ./jpeg_encode /home/nvidia/I420.yuv 1920 1080 /home/nvidia/b.jpg -crop 0 0 1920 1080
The histogram is identical. Do you use r32.3.1?
Hi DaneLLL,
I’m on L4T 32.2.1/TX2. You could use gimp (2.5. Histogram dialog) to see the histogram.
I also see no difference between the jpg images. Could you try to encode a video with 01_video_encode and compare it to the jpg image:
// generate YUV data
$ gst-launch-1.0 videotestsrc num-buffers=60 ! video/x-raw,width=1920,height=1080 ! filesink location=I420.yuv
// encode with hardeware video encoder
$ ./video_encode /home/nvidia/I420.yuv 1920 1080 H264 /home/nvidia/video.h264
// pack into mp4
$ ffmpeg -i /home/nvidia/video.h264 -vcodec copy -an /home/nvidia/video.mp4
Maybe you can also share the jpg and the mp4 with the upload function in this forum?
Hi,
Please share a JPEG or YUV420 file so that we can see difference in using software JPEG encoder and hardware JPEG encoder. The test pattern generated through videotestsrc may not be able to show the deviation. We have tried hardware video encoding and don’t observe difference.
Hello DaneLLL,
You can find following files in this zip file:
https://www.dropbox.com/s/j6rmmpsg2dtg4hv/color_mismatch.zip?dl=1
original.jpg my original image
result.yuv original.jpg → 06_jpeg_decode → result.yuv
result.jpg result.jpg → 05_jpeg_encode → result.jpg
result.yuv result.yuv(multiple times) → 01_video_encode → result.mp4
Left is the result.jpg and right the result.mp4, you can see the differences in dark wood on the left side.
The steps are the same like mentioned in the previous post. Please also share your jpg and mp4 result from my jpg file.
Hi,
Attach the JPEG I get on r32.3.1/TX2
hw_jpegenc.zip (1.4 MB)
I don’t see significant differencein comparing to original.jpg. Please help take a look. Probably some deviation should be noticed but I miss it.
Hello DaneLLL,
thanks, beside the missing crop, the colors are identical. As mentioned above i only see the difference between the jpg and the mp4. Can you encode a mp4 and share it?
Hi,
Attache the h264 stream. And re-encode into JPG via software decoder and encoder.
01_video_encode$ ./video_encode ~/1920_1080_10.yuv 1920 1080 H264 01_venc.h264
$ gst-launch-1.0 filesrc location= 01_venc.h264 ! h264parse ! avdec_h264 ! jpegenc ! multifilesink location=01_venc_%02d.jpg
The histogram looks identical. Please help check it.
Not sure but maybe the issue is in packing to mp4 through ffmpeg.
01_venc.zip (446.9 KB)