Lossless Hardware H.264 Encoding

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hi!

I’m trying to get lossless hardware H.264 encoding working on a Xavier flashed from JetPack 4.1. I downloaded the Tegra Multimedia API and compiled sample “01_video_encode” and ran some raw YUV data through it with the --elossless option. However, when I run a binary comparison, the outputs are not the same. I tried looking at several frames in the encoded YUV output and they all had very noticeable compression artifacts.

The following are the GStreamer pipelines I used to run my tests:

# Get some frames from the camera and save them in raw YUV format
gst-launch-1.0 -e tcambin ! 'video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)30/1' \
    ! videoconvert ! "video/x-raw,format=I420" ! filesink location=~/Desktop/original.yuv

# Run the raw YUV from the camera through the hardware encoder, with lossless enabled
./video_encode ~/Desktop/original.yuv 1440 1080 H264 ~/Desktop/encoded.h264 --elossless

# Decode the encoded H.264 and dump it back to the disk
gst-launch-1.0 -e filesrc location=~/Desktop/encoded.h264 ! "video/x-h264, stream-format=(string)byte-stream" \
    ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=I420" ! filesink location=~/Desktop/encoded.yuv

Then, for each YUV frame from the original and encoded file, I loaded their luma components and performed a binary comparison between them with OpenCV. The compression artifacts are pretty obvious, both in the binary difference and just by examining the luma of the encoded image.

As a last ditch effort, I also tried running a command I’ve seen in a few threads of other people trying the same thing (see here: https://devtalk.nvidia.com/default/topic/1026493/jetson-tx1/lossless-h-264-encoding/) that takes the raw YUV as an input, encodes it losslessly, immediately decodes it, and outputs it to a file:

gst-launch-1.0 -e filesrc location=~/Desktop/original.yuv blocksize=2332800 ! \
    "video/x-raw,width=(int)1440,height=(int)1080,framerate=(fraction)30/1,format=I420" ! \
    omxh264enc control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0 ! \
    h264parse ! omxh264dec ! nvvidconv ! "video/x-raw,format=I420" ! \
    filesink location=~/Desktop/encoded.yuv

This produces similar compression artifacts.

Am I missing something?

Thanks!

2

Hi mattbow,
‘–elossless’ in tegra multimedia api and ‘control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0’ in gstreamer generates h264 stream with all frames in qp value=0. Even though in this setting, there is still minor compression loss.

so you are looking for a solution origin.yuv and encoded.yuv are byte-to-byte identical?

3

Yes, I’m looking for something that allows me to encode origin.yuv into an H.264 stream and then decode that stream back into a file (encoded.yuv) that is bit-identical to origin.yuv. FFmpeg is capable of doing this, but of course it’s not hardware accelerated on the Xavier.

4

Hi mattbow,
We run below steps and get identical encode.yuv and original.yuv

~/tegra_multimedia_api/samples/01_video_encode$ gst-launch-1.0 videotestsrc num-buffers=60 ! filesink location=~/origin.yuv
~/tegra_multimedia_api/samples/01_video_encode$ ./video_encode ~/origin.yuv 320 240 H264 encode.h264 --elossless
~/tegra_multimedia_api/samples/01_video_encode$ cd ../00_video_decode/
~/tegra_multimedia_api/samples/00_video_decode$ ./video_decode H264 -o ~/encode.yuv --disable-rendering ../01_video_encode/encode.h264

Can you share us steps to reproduce the issue? Maybe share your original.yuv

5

So, I ran those exact same steps on my Xavier and origin.yuv and encode.yuv had different SHA-256 hashes. To make sure nothing weird was going on, I downloaded and extracted a fresh copy of the Tegra Multimedia API:

~$ sha256sum Tegra_Multimedia_API_R31.0.2_aarch64.tbz2 
d95ba40a929b70785a6c0448430d138f05635a1c78743f9426752ede4f400831  Tegra_Multimedia_API_R31.0.2_aarch64.tbz2
~$ tar -xf Tegra_Multimedia_API_R31.0.2_aarch64.tbz2
~$ cd tegra_multimedia_api/samples/01_video_encode/
~/tegra_multimedia_api/samples/01_video_encode$ make
Compiling: video_encode_csvparser.cpp
Compiling: video_encode_main.cpp
make[1]: Entering directory '/home/nvidia/tegra_multimedia_api/samples/common/classes'
Compiling: NvElementProfiler.cpp
Compiling: NvElement.cpp
Compiling: NvApplicationProfiler.cpp
Compiling: NvVideoDecoder.cpp
Compiling: NvBuffer.cpp
Compiling: NvJpegEncoder.cpp
Compiling: NvVideoConverter.cpp
Compiling: NvDrmRenderer.cpp
Compiling: NvLogging.cpp
Compiling: NvEglRenderer.cpp
Compiling: NvUtils.cpp
Compiling: NvJpegDecoder.cpp
Compiling: NvVideoEncoder.cpp
Compiling: NvV4l2ElementPlane.cpp
Compiling: NvV4l2Element.cpp
make[1]: Leaving directory '/home/nvidia/tegra_multimedia_api/samples/common/classes'
Linking: video_encode
~/tegra_multimedia_api/samples/01_video_encode$ cd ../00_video_decode/
~/tegra_multimedia_api/samples/00_video_decode$ make
Compiling: video_decode_csvparser.cpp
Compiling: video_decode_main.cpp
Linking: video_decode
~/tegra_multimedia_api/samples/00_video_decode$ cd ../01_video_encode/
~/tegra_multimedia_api/samples/01_video_encode$ gst-launch-1.0 videotestsrc num-buffers=60 ! filesink location=~/origin.yuv
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.030539464
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
~/tegra_multimedia_api/samples/01_video_encode$ ./video_encode ~/origin.yuv 320 240 H264 encode.h264 --elossless
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
875967048
842091865
H264: Profile = 66, Level = 51 
Could not read complete frame from input file
File read complete.
App run was successful
~/tegra_multimedia_api/samples/01_video_encode$ cd ../00_video_decode/
~/tegra_multimedia_api/samples/00_video_decode$ ./video_decode H264 -o ~/encode.yuv --disable-rendering ../01_video_encode/encode.h264
Set governor to performance before enabling profiler
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Input file read complete
Video Resolution: 320x240
Query and set capture successful
Exiting decoder capture loop thread
App run was successful
~/tegra_multimedia_api/samples/00_video_decode$ cd
~$ ls -la *.yuv
-rw-rw-r-- 1 nvidia nvidia 6912000 Nov  1 14:12 encode.yuv
-rw-rw-r-- 1 nvidia nvidia 6912000 Nov  1 14:11 origin.yuv
~$ sha256sum origin.yuv 
52a548f4e4a62b0ba737738c739bc3a1b44fb49e3ad5ff888ae104f54ad6a94f  origin.yuv
~$ sha256sum encode.yuv 
6d133086eb720b109c7922a755708762f3c15be2b5bce09277f34c58375a3b1c  encode.yuv

I also tried looking at the first and second frames in the files to see if maybe something at the start or end of the file messed with the SHA-256 hash, but no dice:

~$ head -c 115200 origin.yuv | sha256sum
238d779048ea46f3ebd9d804704d10267be75a47641af03dcb215d347cb88a62  -
~$ head -c 115200 encode.yuv | sha256sum
10f6cf91724c03a8a742c75a0979ba2d671d186eea7221bcd5d35b4f88484271  -
~$ head -c 230400 origin.yuv | tail -c 115200 | sha256sum
2a98bb97368a27fe007b7d32390474980e8388c3a6c1122f4d12808903451c9b  -
~$ head -c 230400 encode.yuv | tail -c 115200 | sha256sum
454c4a27aaa48a5ac00d12d2e6d72907d8598f005893deee0a1524a5aa84e8ea  -

My Xavier is flashed with JetPack 4.1:

~$ head -n 1 /etc/nv_tegra_release
# R31 (release), REVISION: 0.2, GCID: 12860113, BOARD: t186ref, EABI: aarch64, DATE: Sat Sep 29 05:14:38 UTC 2018

I tried running the same steps on another Xavier I have with me, flashed with the same BSP version, and I got the exact same results, same SHA-256 hashes as well. I can share origin.yuv and encode.yuv if you’d like.

6

Hi mattbow,
I have confirmed your observation. With your steps, I can see there is difference between origin.yuv and encode.yuv.

This is a feature request and we have teams to check and evaluate it. If you need it to be prioritized, please contact NVIDIA salesperson for further cooperation. Thanks.

7

Hi,
We don’t have plan to support this on AGX Xavier. Please kindly be informed.