Unintended contrast spread when using Jetson, GStreamer, and H265 to encode videos

Hi everyone,

While encoding grayscale images into videos using GStreamer on Jetson Xavier AGX, I experienced a shift in brightness values. After some investigation, I found out that a spreading of the contrast occurs: dark pixels became darker, light pixels became lighter.

Does anyone of you also experience this issue? Is there a way, e.g. a setting, to prevent this?

About my investigation:

I created multiple sets with test images having one dark object moving on a light background. For each set, the brightness of the background is different. The brightness of the object remains the same. A single image looks like this:

I encoded the images using a lossless H.265 encoding (see NVIDIA developer forum):

gst-launch-1.0 multifilesrc location=ramp228/image_ramp_%d.png start-index=1 stop-index=1000 caps='image/png, framerate=25/1, format=GRAY8' ! pngdec ! queue ! videoconvert ! nvvidconv ! nvv4l2h265enc control-rate=0 qp-range='0,1:0,1:0,1' quant-i-frames=0 quant-p-frames=0 ! h265parse ! qtmux ! filesink location=ramp228.mp4 -e

A contrast spreading can be seen in all resulting videos. The brightness of the dark object has changed regardless of the background by 15 units. The brightness of the backgrounds also changed, with brighter backgrounds showing a stronger change. The following table contains the original brightness values and the values after encoding:

Original Reconstructed Difference
25 10 - 15
128 130 + 2
178 188 + 10
228 246 + 18
248 255 + 7

The following figure describes the contrast spread using a histogram for the image shown above before and after encoding. It is noticeable that the two maxima (dark object and light background) shift outward.

ramp228

I hope you can help me with this issue and look forward to your responses.

Best Regards!

1 Like

Hi,
Please share how to generate image_ramp_%d.png. Can we generate it by running command like:

videotestrc ! video/x-raw,format=RGB ! pngenc ! multifilesink

I generated the images using Halcon (see https://www.mvtec.com/de/produkte/halcon).

Similar images can be generated, running:

gst-launch-1.0 videotestsrc pattern=ball background-color=-1776412 foreground-color=-15132391 num-buffers=1000 ! video/x-raw, width=512, height=512, format=RGB ! pngenc ! multifilesink location=testing2/image_ramp_%d.png

However, when encoding these images, I experience different behavior. The brightness values still shift, but all values decrease.

testing2

Hi,
Please check if the issue is present with software encoder x265enc. Run the command like:

gst-launch-1.0 multifilesrc location=ramp228/image_ramp_%d.png start-index=1 stop-index=1000 caps='image/png, framerate=25/1, format=GRAY8' ! pngdec ! queue ! videoconvert ! x265enc ! h265parse ! qtmux ! filesink location=ramp228.mp4 -e

And please share how to check brightness value. Would like to know which tool is used to generate the chart.

Yes, x265enc results in the same contrast spread. I had to add video/x-raw, format=I420 between videoconvert and x265enc to make the pipeline work.

Differences are already visually noticeable when images and videos are placed next to each other. To determine the exact brightness values, I use Python and the libaries cv2 to load images and videos (cv2 relies on a FFmpeg software decoder), numpy to analyse the image arrays and matplotlib to plot the results.

Hi,

Do you mean the brightness remain the same with x265enc?

I mean, I still notice a shift in brightness values with x265enc. So I think it is not the codec itself but some kind of conversion within the pipeline that causes this issue.

Hi everyone, despite the new realization that other encoders also results in contrast spreading, I have not made any progress regarding the issue. The contrast spread remains. Do you have any further advice on what could be a reason for this and how I can fix it?

1 Like

Hi,
Since the issue is also present in using software encoder, it seems not an issue in encoder. Please dump YUV data and check if the color is correct through YUV viewer:

gst-launch-1.0 multifilesrc location=ramp228/image_ramp_%d.png start-index=1 stop-index=1000 caps='image/png, framerate=25/1, format=GRAY8' ! pngdec ! queue ! videoconvert ! video/x-raw,format=I420 ! multifilesink location=dump_%05d.YUV

Hi, I dumped the YUV data and checked it using an online yuv viewer (see RAW Pixels). In addition, I had a look at the resulting bitstream. It is still correct.

More information on the YUV data: It consists of 512 x 512 bytes representing the Y values and two times 128 x 512 bytes representing the U and V values respectively. The Y values are equal to the gray values/intensity of the original image and the U and V values are all 128.