Lossless Hardware H.264 Encoding


I’m trying to get lossless hardware H.264 encoding working on a Xavier flashed from JetPack 4.1. I downloaded the Tegra Multimedia API and compiled sample “01_video_encode” and ran some raw YUV data through it with the --elossless option. However, when I run a binary comparison, the outputs are not the same. I tried looking at several frames in the encoded YUV output and they all had very noticeable compression artifacts.

The following are the GStreamer pipelines I used to run my tests:

# Get some frames from the camera and save them in raw YUV format
gst-launch-1.0 -e tcambin ! 'video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)30/1' \
    ! videoconvert ! "video/x-raw,format=I420" ! filesink location=~/Desktop/original.yuv

# Run the raw YUV from the camera through the hardware encoder, with lossless enabled
./video_encode ~/Desktop/original.yuv 1440 1080 H264 ~/Desktop/encoded.h264 --elossless

# Decode the encoded H.264 and dump it back to the disk
gst-launch-1.0 -e filesrc location=~/Desktop/encoded.h264 ! "video/x-h264, stream-format=(string)byte-stream" \
    ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=I420" ! filesink location=~/Desktop/encoded.yuv

Then, for each YUV frame from the original and encoded file, I loaded their luma components and performed a binary comparison between them with OpenCV. The compression artifacts are pretty obvious, both in the binary difference and just by examining the luma of the encoded image.

As a last ditch effort, I also tried running a command I’ve seen in a few threads of other people trying the same thing (see here: https://devtalk.nvidia.com/default/topic/1026493/jetson-tx1/lossless-h-264-encoding/) that takes the raw YUV as an input, encodes it losslessly, immediately decodes it, and outputs it to a file:

gst-launch-1.0 -e filesrc location=~/Desktop/original.yuv blocksize=2332800 ! \
    "video/x-raw,width=(int)1440,height=(int)1080,framerate=(fraction)30/1,format=I420" ! \
    omxh264enc control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0 ! \
    h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=I420" ! \
    filesink location=~/Desktop/encoded.yuv

This produces similar compression artifacts.

Am I missing something?


Hi mattbow,
‘–elossless’ in tegra multimedia api and ‘control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0’ in gstreamer generates h264 stream with all frames in qp value=0. Even though in this setting, there is still minor compression loss.

so you are looking for a solution origin.yuv and encoded.yuv are byte-to-byte identical?

Yes, I’m looking for something that allows me to encode origin.yuv into an H.264 stream and then decode that stream back into a file (encoded.yuv) that is bit-identical to origin.yuv. FFmpeg is capable of doing this, but of course it’s not hardware accelerated on the Xavier.

Hi mattbow,
We run below steps and get identical encode.yuv and original.yuv

~/tegra_multimedia_api/samples/01_video_encode$ gst-launch-1.0 videotestsrc num-buffers=60 ! filesink location=~/origin.yuv
~/tegra_multimedia_api/samples/01_video_encode$ ./video_encode ~/origin.yuv 320 240 H264 encode.h264 --elossless
~/tegra_multimedia_api/samples/01_video_encode$ cd ../00_video_decode/
~/tegra_multimedia_api/samples/00_video_decode$ ./video_decode H264 -o ~/encode.yuv --disable-rendering ../01_video_encode/encode.h264

Can you share us steps to reproduce the issue? Maybe share your original.yuv

So, I ran those exact same steps on my Xavier and origin.yuv and encode.yuv had different SHA-256 hashes. To make sure nothing weird was going on, I downloaded and extracted a fresh copy of the Tegra Multimedia API:

~$ sha256sum Tegra_Multimedia_API_R31.0.2_aarch64.tbz2 
d95ba40a929b70785a6c0448430d138f05635a1c78743f9426752ede4f400831  Tegra_Multimedia_API_R31.0.2_aarch64.tbz2
~$ tar -xf Tegra_Multimedia_API_R31.0.2_aarch64.tbz2
~$ cd tegra_multimedia_api/samples/01_video_encode/
~/tegra_multimedia_api/samples/01_video_encode$ make
Compiling: video_encode_csvparser.cpp
Compiling: video_encode_main.cpp
make[1]: Entering directory '/home/nvidia/tegra_multimedia_api/samples/common/classes'
Compiling: NvElementProfiler.cpp
Compiling: NvElement.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvBuffer.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvLogging.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvV4l2Element.cpp
make[1]: Leaving directory '/home/nvidia/tegra_multimedia_api/samples/common/classes'
Linking: video_encode
~/tegra_multimedia_api/samples/01_video_encode$ cd ../00_video_decode/
~/tegra_multimedia_api/samples/00_video_decode$ make
Compiling: video_decode_csvparser.cpp
Compiling: video_decode_main.cpp
Linking: video_decode
~/tegra_multimedia_api/samples/00_video_decode$ cd ../01_video_encode/
~/tegra_multimedia_api/samples/01_video_encode$ gst-launch-1.0 videotestsrc num-buffers=60 ! filesink location=~/origin.yuv
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.030539464
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
~/tegra_multimedia_api/samples/01_video_encode$ ./video_encode ~/origin.yuv 320 240 H264 encode.h264 --elossless
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
H264: Profile = 66, Level = 51 
Could not read complete frame from input file
File read complete.
App run was successful
~/tegra_multimedia_api/samples/01_video_encode$ cd ../00_video_decode/
~/tegra_multimedia_api/samples/00_video_decode$ ./video_decode H264 -o ~/encode.yuv --disable-rendering ../01_video_encode/encode.h264
Set governor to performance before enabling profiler
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Input file read complete
Video Resolution: 320x240
Query and set capture successful
Exiting decoder capture loop thread
App run was successful
~/tegra_multimedia_api/samples/00_video_decode$ cd
~$ ls -la *.yuv
-rw-rw-r-- 1 nvidia nvidia 6912000 Nov  1 14:12 encode.yuv
-rw-rw-r-- 1 nvidia nvidia 6912000 Nov  1 14:11 origin.yuv
~$ sha256sum origin.yuv 
52a548f4e4a62b0ba737738c739bc3a1b44fb49e3ad5ff888ae104f54ad6a94f  origin.yuv
~$ sha256sum encode.yuv 
6d133086eb720b109c7922a755708762f3c15be2b5bce09277f34c58375a3b1c  encode.yuv

I also tried looking at the first and second frames in the files to see if maybe something at the start or end of the file messed with the SHA-256 hash, but no dice:

~$ head -c 115200 origin.yuv | sha256sum
238d779048ea46f3ebd9d804704d10267be75a47641af03dcb215d347cb88a62  -
~$ head -c 115200 encode.yuv | sha256sum
10f6cf91724c03a8a742c75a0979ba2d671d186eea7221bcd5d35b4f88484271  -
~$ head -c 230400 origin.yuv | tail -c 115200 | sha256sum
2a98bb97368a27fe007b7d32390474980e8388c3a6c1122f4d12808903451c9b  -
~$ head -c 230400 encode.yuv | tail -c 115200 | sha256sum
454c4a27aaa48a5ac00d12d2e6d72907d8598f005893deee0a1524a5aa84e8ea  -

My Xavier is flashed with JetPack 4.1:

~$ head -n 1 /etc/nv_tegra_release
# R31 (release), REVISION: 0.2, GCID: 12860113, BOARD: t186ref, EABI: aarch64, DATE: Sat Sep 29 05:14:38 UTC 2018

I tried running the same steps on another Xavier I have with me, flashed with the same BSP version, and I got the exact same results, same SHA-256 hashes as well. I can share origin.yuv and encode.yuv if you’d like.

Hi mattbow,
I have confirmed your observation. With your steps, I can see there is difference between origin.yuv and encode.yuv.

This is a feature request and we have teams to check and evaluate it. If you need it to be prioritized, please contact NVIDIA salesperson for further cooperation. Thanks.

We don’t have plan to support this on AGX Xavier. Please kindly be informed.