Questions about NvVideoEncoder in Tegra Multimedia SDK and VPx codecs.

Hi!

I had tried to encode video with NvVideoEncoder on Jetson TX2 and then decode it on host computer with libavcodec. Basically it works with H.265 and H.264, but I have couple of questions:

  1. What is the output format of VPx codecs (VP8 and VP9)? libavcodec does not recognize them as valid VP8/VP9 bitstreams, although mplayer plays it correclty.
  2. Is it possible to encode H.265, VP8 or VP9 with YUV 4:4:4 subsampling? It seems that encoder ignores input pixel format for H.265 and still uses YUV 4:2:0.
  3. Is it possible to encode lossless 10-bit YUV 4:4:4 (no matter what codec)? Examining source code of Tegra Multimedia API suggests that it isn’t, but maybe I’m wrong.

Hi,

In VP9, it has with IVF header
https://wiki.multimedia.cx/index.php/IVF
In VP8, it simply is raw stream
You may check the function in 00_video_decode:

static int
read_vpx_decoder_input_chunk(context_t *ctx, NvBuffer * buffer)

These are not supported. We only support H264.

You can verify with following steps:
1 Generate bitstream

$ gst-launch-1.0 videotestsrc num-buffers=60 !  'video/x-raw, width=(int)1280, height=(int)720, format=(string)Y444'  ! filesink location= /home/nvidia/origin.yuv
$ ./video_encode /home/nvidia/origin.yuv 1280 720 H264 /home/nvidia/loless.h264 --elossless

2 Decode and inspect

$ ffmpeg -i loless.h264 -c:v rawvideo -pix_fmt yuv444p out.yuv
$ head -c 230400 origin.yuv | tail -c 115200 | sha256sum
$ head -c 230400 out.yuv | tail -c 115200 | sha256sum

Thanks for your answer, VPx output really looks like provided format description, I will try to parse it (btw not only VP9, but VP8 too).

As for the lossless, I tried to run H.264 encoder on completely random input (just to test it lossless-ness in the worst case), and I encountered following error:

Opening in O_NONBLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
H264: Profile = 244, Level = 0 
tvmrVideoEncoderBitsAvailable_MSENC: ucode ERROR = 96 
NvVideoEncTransferOutputBufferToBlock: DoWork failed line# 650 
NvVideoEnc: NvVideoEncTransferOutputBufferToBlock TransferBufferToBlock failed Line=661

My code uses the following codec settings:

m_encoder->setCapturePlaneFormat(V4L2_PIX_FMT_H264, 1024, 1024, 64 * 1024 * 1024);
m_encoder->setOutputPlaneFormat(V4L2_PIX_FMT_YUV444M, m_width, m_height);
m_encoder->setProfile(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_PREDICTIVE);
m_encoder->setBitrate(80 * 1024 * 1024);
m_encoder->setMaxPerfMode(1);
m_encoder->setHWPresetType(V4L2_ENC_HW_PRESET_ULTRAFAST);
m_encoder->setConstantQp(0);
m_encoder->output_plane.setupPlane(V4L2_MEMORY_MMAP, 20, true, false);
m_encoder->capture_plane.setupPlane(V4L2_MEMORY_MMAP, 20, true, false);

Don’t mind the insane bitrates and buffer sizes, it fails even without them. They are set just to ensure that codec has sufficient space to write output into. All setup functions are checked so they don’t return -1 or anything below zero, but in this snippet these checks are omitted for simplicity.

This code runs fine for “normal” video streams, and it runs fine for random stream without HIGH_444_PREDICTIVE profile. It even runs fine for random streams with values from range 0…79 instead of 0…255. But it fails on pseudorandom data, even on low resolutions like 256x256. I’m wondering what does this error mean, and the fact itself that codec can fail in arbitrary cases isn’t good, since after that encoding stops completely without any errors returned from userland functions. It just stucks in dequeuing output plane buffer, getting EAGAIN and EAGAIN and EAGA… in endless loop.

Hi,
Please share your L4T release version. Also are you able to try

$ gst-launch-1.0 videotestsrc num-buffers=60 !  'video/x-raw, width=(int)1280, height=(int)720, format=(string)Y444'  ! filesink location= /home/nvidia/origin.yuv
$ ./video_encode /home/nvidia/origin.yuv 1280 720 H264 /home/nvidia/loless.h264 --elossless

We can run it successfully on r32.1.

My Jetson was flashed from latest JetPack (4.2.1 IIRC), has kernel 4.9.140-tegra.

As for your pipeline, codec is really lossless and SHA256 of both files matches. However, I’m interesting why codec fails on completely random input and what does these errors mean. The easiest way to reproduce them is to use /dev/urandom as source (like a “worst case picture for lossless codec”):

$ cat /dev/urandom | head -c 165888000 > origin.yuv
$ /usr/src/tegra_multimedia_api/samples/01_video_encode/video_encode origin.yuv 1280 720 H264 loless.h264 --elossless
Creating Encoder in blocking mode 
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
875967048
875711833
H264: Profile = 244, Level = 51 
tvmrVideoEncoderBitsAvailable_MSENC: ucode ERROR = 96 
VENC: NvMMLiteVideoEncDoWork: 4231: BlockSide error 0x4
NvVideoEnc: BlockError 
NvVideoEncTransferOutputBufferToBlock: DoWork failed line# 650 
NvVideoEnc: NvVideoEncTransferOutputBufferToBlock TransferBufferToBlock failed Line=661
[ERROR] (NvV4l2ElementPlane.cpp:256) <enc0> Output Plane:Error while Qing buffer: Invalid argument
Error while queueing buffer at output plane
^C

Hi,
The command does not generate YUVs we will see in real life. Please try with
https://media.xiph.org/video/derf/y4m/

For non-YUV444 data, you can convert it through:

nvidia@nvidia-desktop:~$ gst-launch-1.0 filesrc location= pedestrian_area_1080p25.y4m ! y4mdec ! videoconvert ! video/x-raw,format=Y444 ! filesink location= origin.yuv