Problem about Accelerated Decode with ffmpeg

3263503134 · November 9, 2022, 2:34am

您好！
我们公司现在在使用NVIDIA Jetson Xavier NX，
现在我们想在NVIDIA Jetson Xavier NX 上利用ffmpeg4.2支持NVIDIA硬件加速来实现编解码功能, 以便减少视频播放延时大的问题。

我们查了NVIDIA的官网，根据官网介绍(Accelerated Decode with ffmpeg — Jetson Linux<br/>Developer Guide 34.1 documentation):

Accelerated Decode with ffmpeg

Decode Functional Flow
This section describes the function flow of the ffmpeg decoding process:

Call nvv4l2dec_init_decoder() to create a new V4L2 Video Decoder object on the device node /dev/nvhost-nvdec.

Call subscribe_event() to subscribe to resolution change events.

Call set_output_plane_format() to set the format on the output plane.

Call capture_thread() to start a capture thread.

Call set_capture_plane_format() to set the format on the capture plane.

Call nvv4l2dec_decode() to read buffers from ffmpeg and start the decode process.

Call nvv4l2dec_decoder_get_frame() to get the hardware-accelerated decoded data and pass it to ffmpeg for dumping.

Call nvv4l2dec_decoder_close() to destroy the buffers and close the device.

我们从NVIDIA官网上下载了Video_Codec_SDK_10.0.26.zip源代码，但是该源代码中没有发现有上面介绍的这些接口函数。
ffmpeg4.2.8源代码中也没有发现有上面介绍的这些接口函数。

请问这些函数接口在哪个库中包含的？
可否给我们提供上面的这些接口函数的使用例子代码？

我们的项目很紧急，请尽快给我们答复。

谢谢！

DaneLLL · November 9, 2022, 4:59am

Hi,
Video Codec SDK is not supported on Jetson platforms. We have a package to enable hardware decoding with jetson_multimedia_api. Please check
Jetson TX2 and FFmpeg - Can't initialize nvrm channel - #5 by DaneLLL
Hardware accelerated video playback with L4T ffmpeg - #7 by DaneLLL

3263503134 · November 10, 2022, 6:51am

您好！
我按照您提供的Jetson TX2 and FFmpeg - Can’t initialize nvrm channel - #5 by DaneLLL

已经成功的编译好ffmpeg-4.2.7，make && make install
并且可以正常运行ffmpeg命令。

我运行 ./ffmpeg -decoders命令时，打印信息如下：
root@nvidia-desktop:/home/nvidia/working-fu/ffmpeg-4.2.7-nvidia# ./ffmpeg -decoders | grep nvv4l2
ffmpeg version 4.2.7 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
configuration: --enable-nvv4l2dec --enable-libv4l2 --extra-libs=‘-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils’ --extra-cflags=‘-I /usr/src/jetson_multimedia_api/include/’
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
V… h264_nvv4l2dec h264 (nvv4l2dec) (codec h264)
V… hevc_nvv4l2dec hevc (nvv4l2dec) (codec hevc)
V… mpeg2_nvv4l2dec mpeg2 (nvv4l2dec) (codec mpeg2video)
V… mpeg4_nvv4l2dec mpeg4 (nvv4l2dec) (codec mpeg4)
root@nvidia-desktop:/home/nvidia/working-fu/ffmpeg-4.2.7-nvidia#

可以看到ffmpeg-4.2.7的解码器有：
V… h264_nvv4l2dec h264 (nvv4l2dec) (codec h264)
V… hevc_nvv4l2dec hevc (nvv4l2dec) (codec hevc)

但是我运行 ./ffmpeg -hwaccels命令时，打印信息如下：
root@nvidia-desktop:/home/nvidia/working-fu/ffmpeg-4.2.7-nvidia# ./ffmpeg -hwaccels
ffmpeg version 4.2.7 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
configuration: --enable-nvv4l2dec --enable-libv4l2 --extra-libs=‘-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils’ --extra-cflags=‘-I /usr/src/jetson_multimedia_api/include/’
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
Hardware acceleration methods:

root@nvidia-desktop:/home/nvidia/working-fu/ffmpeg-4.2.7-nvidia#
Hardware acceleration methods: 显示是空的，也就是说没有硬件加速的？？？
这样还可以实现ffmpeg硬件加速编解码视频图像吗？

DaneLLL · November 10, 2022, 9:35am

Hi,
Hardware decoding should be enabled. You may run this command:

ffmpeg-4.2.2$ ./ffmpeg -c:v h264_nvv4l2dec -i /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4 a.yuv

And execute sudo tegrastats to check if NVDEC is up and running.

3263503134 · November 10, 2022, 11:43am

您好！
我成功编译ffmpeg-4.2.7后，是可以正常运行fffmpeg命令来实现解码的。
fffmpeg命令实现解码运行后的打印信息如下：
ffmpeg-4.2.7-nvidia# ./ffmpeg -c:v h264_nvv4l2dec -i /home/working-video/video-decenc-test/test_cut.h264 -r 30 -y /home/working-video/video-decenc-test/video-dec%03d.jpg
ffmpeg version 4.2.7 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
configuration: --enable-nvv4l2dec --enable-libv4l2 --extra-libs=‘-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils’ --extra-cflags=‘-I /usr/src/jetson_multimedia_api/include/’
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
…
Input #0, h264, from ‘/home/working-video/video-decenc-test/test_cut.h264’:
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 50 tbc
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Stream mapping:
Stream #0:0 → #0:0 (h264 (h264_nvv4l2dec) → mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x557f4fbc60] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to ‘/home/working-video/video-decenc-test/video-dec%03d.jpg’:
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: mjpeg, yuvj420p(pc), 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:
encoder : Lavc58.54.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame= 75 fps= 15 q=24.8 Lsize=N/A time=00:00:03.00 bitrate=N/A dup=11 drop=10 speed=0.61x

但是我修改./configure, 在原来基础上增加了–enable-nvdec --enable-nvenc --enable-cuda --enable-cuvid，结果./configure 报错误。

接着我继续修改./configure, 在原来基础上只增加了–enable-nvdec --enable-nvenc，结果./configure 还是报错误。

难道NVIDIA Jetson Xavier NX不支持enable-cuda 和 enable-cuvid？？？
也不支持enable-nvdec 和enable-nvenc？？？

我们的项目很紧急，请尽快给我们答复。
谢谢！

3263503134 · November 10, 2022, 11:52am

您好！
我编译好ffmpeg-4.2.7后，是可以正常运行./ffmpeg -c:v h264_nvv4l2dec -i /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4 a.yuv命令来解码的。
但是我修改./configure，在原来基础./configure --enable-nvv4l2dec --enable-libv4l2 --extra-libs=“-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils” --extra-cflags=“-I /usr/src/jetson_multimedia_api/include/”
上增加了 --enable-nvdec --enable-nvenc --enable-cuda --enable-cuvid，./configure报错误，
后来我修改./configure，在原来基础上只增加–enable-nvdec --enable-nvenc，./configure还是报错误
请问：NVIDIA Jetson Xavier NX不支持enable-cuda 和 enable-cuvid？？？
也不支持enable-nvdec 和enable-nvenc ？？？
我们的项目很紧急，请尽快给我们答复。
谢谢！

3263503134 · November 15, 2022, 6:42am

您好！

我下载、编译、安装好了NVIDIA的ffmpeg，然后运行基于 NVDEC的硬件加速解码的ffmpeg命令，输出信息如下：
ffmpeg-4.2.7-nvidia/bin# ./ffmpeg -c:v h264_nvv4l2dec -i /home/working-video/700818_023723.h264 -r 30 -y /home/working-video/video-harddec-test/video-dec%03d.jpg
ffmpeg version 4.2.7 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
configuration: --enable-nvv4l2dec --enable-libv4l2 --enable-shared --optflags=-O3 --extra-libs=‘-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils’ --extra-cflags=‘-I /usr/src/jetson_multimedia_api/include/’ --enable-rpath --prefix=/opt/nvidia/ffmpeg-4.2.7-nvidia

Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 50 tbc

Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Stream mapping:
Stream #0:0 → #0:0 (h264 (h264_nvv4l2dec) → mjpeg (native))
Output #0, image2, to ‘/home/working-video/working-video-dec/video-harddec-test/video-dec%03d.jpg’:
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: mjpeg, yuvj420p(pc), 1920x1080, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc
Metadata:
encoder : Lavc58.54.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame= 3879 fps= 18 q=24.8 Lsize=N/A time=00:02:09.30 bitrate=N/A dup=658 drop=11 speed=0.609x

硬件解码速度：speed=0.609x

我运行sudo tegrastats，显示信息中出现了 NVDEC 665 NVDEC1 665 。说明NVDEC已经在运行了。
该命令显示信息如下：
nvidia-desktop:/home/nvidia# sudo tegrastats
RAM 11128/15822MB (lfb 800x4MB) SWAP 0/7911MB (cached 0MB) CPU [55%:1420,0%:1420,45%:1420,0%:1420,5%:1420,0%:1420] EMC_FREQ 2%:1866 GR3D_FREQ 0%:114 NVDEC 665 NVDEC1 665 VIC_FREQ 0%:115 APE 150 MTS fg 0% bg 1% AO:43.5C GPU:43C PMIC:50C AUX:43C CPU:44.5C

但是我接着运行ffmpeg的软解码命令，命令如下：
nvidia-desktop:/usr/local/bin# ffmpeg -i /home/working-video/700818_023723.h264 -r 30 -y /home/working-video/video-softdec-test/video-dec%03d.jpg
ffmpeg version 4.2.7 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
configuration: --enable-nvv4l2dec --enable-libv4l2 --extra-libs=‘-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils’ --extra-cflags=‘-I /usr/src/jetson_multimedia_api/include/’

Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 1200k tbn, 50 tbc

Stream mapping:
Stream #0:0 → #0:0 (h264 (native) → mjpeg (native))
Output #0, image2, to ‘/home/working-video/working-video-dec/video-softdec-test/video-dec%03d.jpg’:
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: mjpeg, yuvj420p(pc), 1920x1080, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc
Metadata:
encoder : Lavc58.54.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame= 3879 fps= 18 q=24.8 Lsize=N/A time=00:02:09.30 bitrate=N/A dup=646 drop=0 speed=0.615x

软件解码速度：speed=0.615x 大于硬件解码速度：speed=0.609x
请问这个问题是怎么解析？？？

DaneLLL · November 15, 2022, 7:19am

Hi,
Please refer to explanation in
Question in NVDEC Accelerated Decode with ffmpeg - #3 by DaneLLL

system · December 7, 2022, 4:31am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.