Hardware decoding in mpv player

I was able to solve this. Before explaining details, I will briefly describe the solution and how to install mpv, vlc and ffmpeg with nvmpi and nvv4l2dec support. I have shared precompiled debs so it would be easy for anyone.

By the way, totem practically does not work on Jetson Nano, for me it was much slower than mpv with software decoding, and it does not come even close to mpv in therms of features anyway; gst-play-1.0 technically works but it does not support landscape mode on display with native portrait orientation which made it unusable for me (since it rotates all videos 90 degrees and I found no way to force it to landscape orientation) and lacks even more features than totem. This is exactly why I could not give up on this: mpv has a lot of features I actually need. VLC has a lot of features too, but I prefer mpv. But I got VLC working with hardware video acceleration too and made precompiled debs in case somebody prefers VLC.

==== mpv and ffmpeg with nvmpi and nvv4l2dec ====

To install, run the following in the directory where you saved mpv-nvmpi-nvv4l2dec.zip (copy and paste all three lines in the terminal):

wget http://Dragon.Studio/2021/02/mpv-nvmpi-nvv4l2dec.zip &&
unzip -n mpv-nvmpi-nvv4l2dec.zip &&
sudo apt -y --allow-change-held-packages --allow-downgrades install ./mpv-nvmpi-nvv4l2dec/*.deb &&
sudo apt-mark hold $(find mpv-nvmpi-nvv4l2dec | grep deb | xargs -n1 basename | cut -d_ -f1)

Last line is important otherwise Ubuntu may offer to “upgrade” ffmpeg or mpv with versions without nvmpi and nvv4l2dec.

Then in ~/.config/mpv/mpv.conf add the following line (create the file if it does not exist):

hwdec=nvmpi-copy

You can also try nvmpi, nvv4l2dec and nvv4l2dec-copy as hwdec but with nvmpi I got the best performance. VLC will use NVMPI by default, no need to configure anything.

You can also use ffmpeg with hardware acceleration for both decoding and encoding, some examples:

ffmpeg -y -c:v h264_nvmpi -i input.mp4 -c:v hevc_nvmpi output.mp4 # Decode input.mp4 with H264 and encode it HEVC
ffmpeg -y -c:v hevc_nvmpi -i input.mp4 -c:v h264_nvmpi output.mp4 # Decode input.mp4 with HEVC and encode it to H264

For decoding, you can also use hevc_nvv4l2dec, h264_nvv4l2dec. Besides h264 or hevc, you can also use vp9, vp8, mpeg2 or mpeg4 for decoding, just change prefix accordingly, for example: vp9_nvmpi.

==== vlc 4.0.0-dev with nvmpi and qt 5.15 + ffmpeg with nvmpi and nvv4l2dec ====

This is how to install vlc with ffmpeg, compiled with nvmpi support (ffmpeg also includes nvv4l2dec support). This allows to use hardware accelerated video encoding and decoding in Jetson Nano. Tested with Ubuntu 18.04 and JetPack 4.5.

First, uninstall vlc if you have it installed (usually by running sudo apt remove vlc).

To install, run the following in the directory where you saved mpv-nvmpi-nvv4l2dec.zip (copy and paste all three lines in the terminal):

wget http://Dragon.Studio/2021/02/vlc-nvmpi-qt515.zip &&
unzip -n vlc-nvmpi-qt515.zip &&
sudo apt -y --allow-change-held-packages --allow-downgrades install ./vlc-nvmpi-qt515/*.deb &&
sudo apt-mark hold $(find vlc-nvmpi-qt515 | grep deb | xargs -n1 basename | cut -d_ -f1)

Last line is important otherwise Ubuntu may offer to “upgrade” vlc or ffmpeg with versions without nvmpi. I had to use vlc version newer than included in Ubuntu 18.04 due to issues I encountered (I was unable to get it working with nvmpi).

Qt 5.15 is necessary for vlc 4.0.0-dev. I compiled all included Qt debs myself from sources provided by this repository Qt 5.15.2 for /opt Bionic : Stephan Binner since id did not include binaries for arm64 architecture, it took a lot of time and effort.

==== ffmpeg with nvmpi and nvv4l2dec ====

It is already included in mpv and vlc packages above, so if you installed debs from one of them, you already have it. But in case somebody needs it without mpv or vlc, I made a separate zip file. To install, run:

wget http://Dragon.Studio/2021/02/ffmpeg-3.4.8-nvmpi-nvv4l2dec.zip &&
unzip -n ffmpeg-3.4.8-nvmpi-nvv4l2dec.zip &&
sudo apt -y --allow-change-held-packages --allow-downgrades install ./ffmpeg-nvmpi-nvv4l2dec/*.deb &&
sudo apt-mark hold $(find ffmpeg-nvmpi-nvv4l2dec | grep deb | xargs -n1 basename | cut -d_ -f1)

This will install ffmpeg-3.4.8 with nvmpi and nvv4l2 support and will hold installed packages to prevent Ubuntu 18.04 from overwriting them with versions without nvmpi and nvv4l2 support during system update. In the zip file, ffmpeg is of the same version as ffmpeg included in Ubuntu 18.04 at the time of writing. Using this version provided the best compatibility. I had issues when trying to use newer versions of ffmpeg in Ubuntu 18.04, this is why I ended up backporting nvmpi and nvv4l2dec patches to it.

==== Performance ====

FullHD and 2.5K at 60FPS plays smoothly. Both mpv and VLC have performance good enough to watch 4K@30FPS at stock NVDEC frequency. But 4K@60FPS in mpv is jerky und unwatchable and in VLC does not play at all because of too many dropped frames. Even with overclocked NVDEC I managed at most 4K@50FPS in mpv. This is because media players have some overhead. But for anything below and including 4K@50FPS hardware accelerated video decoding worked great for me.

gst-play-1.0 can play 4K@60FPS. But gst-play-1.0 is useless in my case since it does not support landscape mode on monitors with native portrait orientation so I have to use gst-launch-1.0 with custom pipeline:

gst-launch-1.0 filesrc location=video.mp4 ! decodebin ! nvvidconv flip-method=1 ! nvoverlaysink`

You need to use flip-method=0 if using display with native landscape orientation (or just use gst-play-1.0). This command achieves the best video decoding performance, but obviously lacks mpv and VLC features, so I only resort to it if I encounter 4K@60FPS video. 4K@50FPS and below plays smoothly for me. Without overclocking, I guess up to 4K@40-45FPS should play smoothly in mpv, which is useful for watching 4K@30FPS at 1.5x speed (at which FPS effectively becomes 45). VLC performance is somewhat worse than mpv: VLC cannot play 4K@30FPS smoothly at 1.5x speed even with overclocking.

==== Low-latency playback in mpv and gstreamer ====

If lowest latency is needed (for example when watching real-time video from IP camera), since mpv version in Ubuntu 18.04 does not support low-latency profile, I had to use the following options to achieve the best best latency (within 0.1-0.2s range depending on IP camera and stream resolution; replace IP address with actual address of your IP camera):

mpv --no-cache --audio-buffer=0 --vd-lavc-threads=1 --cache-pause=no --no-audio --demuxer-lavf-probe-info=no --demuxer-lavf-analyzeduration=0.0 --video-sync=audio --interpolation=no --keep-open-pause=no --untimed --rtsp-transport=tcp rtsp://192.168.1.10/1

Another example of low-latency playback, this time with with gst-launch-1.0 to play video in landscape on mode on monitor with native portrait orientation (remove flip-method=1 or set it to 0 if your monitor has native landscape orientation; also adjust overlay width and height as needed, or remove all overlay-* parameters to use default values for fullscreen playback):

gst-launch-1.0 rtspsrc location=rtsp://169.254.160.104:554/av0_0 latency=0 drop-on-latency=true max-size-buffers=0 ! decodebin ! nvvidconv flip-method=1 ! nvoverlaysink overlay-x=0 overlay-y=0 overlay-w=1440 overlay-h=1920 sync=false -e

VLC is not useful for low latency playback as far as I can tell.

==== How I got nvmpi and nvv4l2dec working with mpv and vlc ====

The rest of this post is about is how I got ffmpeg, mpv and VLC to use video hardware acceleration. If you are not a programmer or not interested to know the details, feel free to skip the rest of this post.

I rebased NVidia ffmpeg patch to git version of ffmpeg, and also added on top the patch from jocover. As a result, I have ffmpeg with both nvv4l2dec (decoding only) and nvmpi (both encoding and decoding).

Unfortunately, at first, when trying to play videos with mpv, I get the following error (at the time I was working on this in January, the patch https://github.com/jocover/jetson-ffmpeg/pull/70 did not exist yet so I had no clue where to start):

mpv --vo=null --hwdec=nvmpi-copy /home/lissanro/Videos/jellyfish-10-mbps-hd-hevc.mkv
...
[ffmpeg/video] hevc_nvmpi: video_get_buffer: image parameters invalid
[ffmpeg/video] hevc_nvmpi: get_buffer() failed
Error while decoding frame (hardware decoding)!
...

This error is not specific to mpv. Somebody here Trying to add support for NVMPI ffmpeg decoders - The VideoLAN Forums attempted to add nvmpi surrport in VLC and they got the same issue. It is not nvmpi specific either, using nvv4l2dec produces almost the same error:

[ffmpeg/video] hevc_nvv4l2dec: video_get_buffer: image parameters invalid
[ffmpeg/video] hevc_nvv4l2dec: get_buffer() failed
Error while decoding frame (hardware decoding)!

Good news, ffplay does play video with hardware acceleration with both nvmpi and nvv4l2dec, and some software which depends on ffmpeg and its video acceleration also works flawlessly (for example, moonlight-qt).

But why not mpv and VLC? First, I decided to add some more debug information. The error happens in ffmpeg itself, this is why it is not specific to any particular player. This is the code where error happens (libavcodec/decode.c):

if (avctx->codec_type == AVMEDIA_TYPE_VIDEO) {
    if ((unsigned)avctx->width > INT_MAX - STRIDE_ALIGN ||
        (ret = av_image_check_size2(FFALIGN(avctx->width, STRIDE_ALIGN), avctx->height, avctx->max_pixels, AV_PIX_FMT_NONE, 0, avctx)) < 0 || avctx->pix_fmt<0) {
        av_log(avctx, AV_LOG_ERROR, "video_get_buffer: image parameters invalid\n");
        ret = AVERROR(EINVAL);
        goto fail;
    }

avctx->width = 1920, so it cannot be bigger than INT_MAX (=2147483647) - STRIDE_ALIGN (=16), and av_image_check_size2() returns 0 (which is not < 0 so it is good return value). All these avctx values are good… except one - avctx->pix_fmt turned out to be -1 (AV_PIX_FMT_NONE) which is not correct, and this is why this error is produced. It is -1 only few first frames but mpv decides not to use hardware decoding after the first error, then it becomes 0.

I tried to a hack to force avctx->pix_fmt = 0 right before the check. This worked well with --vo=null - jtop shows that hardware video decoding being used my mpv. So hardware acceleration can function in mpv. Unfortunately, this hack breaks with --vo=gpu (at the time when I was debugging this, I did not care about deb packaging and I have used newer mpv version compiled from git; older mpv for Ubuntu 18.04 which comes in precompiled deb files does not support --vo=gpu):

mpv --vo=gpu --hwdec=nvmpi-copy /home/lissanro/Videos/jellyfish-10-mbps-hd-hevc.mkv   
(+) Video --vid=1 (*) (hevc 1920x1080 29.970fps)
[ffmpeg] swscaler: No accelerated colorspace conversion found from yuv420p to rgba.
[ffmpeg] swscaler: No accelerated colorspace conversion found from yuv420p to rgba.
double free or corruption (!prev)
zsh: abort (core dumped)  mpv --vo=gpu --hwdec=nvmpi-copy

This is the point where I felt like I am on right track and closer to the solution. I remembered that on Raspberry Pi 3 B+ mpv did not work with hardware acceleration out of the box either but after building ffmpeg and mpv it was possible to make it work. It used MMAL decoder, so what it does differently with avctx->pix_fmt? In ffmmal_init_decoder() it has these two lines of code in libavcodec/mmaldec.c:

int ret = 0;
...
avctx->pix_fmt = ret;

But libavcodec/nvmpi_dec.c and libavcodec/nvv4l2_dec.c lack pix_fmt initialization. As soon as I added it, they both started to work in mpv and vlc too!

So it was never an issue with mpv. It was a bug in both ffmpeg decoders, this is why many other programs were affected including vlc.

When I found time to work on backporting to ffmpeg-3.4.8 and making deb packages, I updated to the latest patch from jetson-ffmpeg project, and I noticed some issues which did not exist before in nvmpi decoder. It turns out that later in January somebody independently tried to develop a similar fix for nvmpi decoder but it breaks decoding of some video streams. In pull requests there is a patch (which is not yet accepted) which supposed to make things better but it still does not make it as reliable as just initializing avctx->pix_fmt to 0 - some videos which could decode before with nvmpi still fail to decode. This is why in the ffmpeg package provided by me, avctx->pix_fmt is always initialized to 0 in NVMPI decoder so it does not have these issues (after all, MMAL decoder for Raspberry Pi does similar thing, and it is known to work well with mpv). Perhaps I create a pull request or report an issue to jetson-ffmpeg project later when I have time.

All zip files with debs I have provided also include patches I have used in case somebody wants to compile themselves.

2 Likes

Hi Lissanro

Fantastic news! mpv is one of the main apps I have been wanting to run properly (hw accelerated) on my Nano since I got it but there’s no way I would’ve been able to pull off what you’ve supposedly achieved here. There is no real competition for mpv when it comes to open source video players.

I have tried to install your packages using the suggested command under a fresh install of Jetpack 4.5.1 and they all installed successfully except libavcodec-extra57_3.4.8-0ubuntu0.2_arm64.deb which gives the error:

dependency problems prevent configuration of libavcodec-extra57:arm64:
 libavcodec-extra57:arm64 depends on libnvmpi; however:
  Package libnvmpi is not installed.

So I tried apt-file find libnvmpi but got no results. Do these packages need rebuilding for 4.5.1? I have tried mpv but h264 video playback isn’t hw accelerated so I presume that libavcodec package I can’t install is not optional.

Do you plan to try to upstream your patches to ffmpeg and mpv? If that’s not an option I’d like to see packages for an accelerated mpv added to the nvidia jetpack repo.

Do you think it would be possible to use nvmpi and this version of mpv and ffmpeg under a 3rd party Linux distro for the nano such as Armbian or does it rely on closed NV libs?

Thanks!

1 Like

Sorry for the delay, I was away for a few days. Looks like I forgot to include libnvmpi in mpv archive (it is possible to compile it yourself, since the patches I used are included, but it may be difficult if you do not have experience building packages). I added libnvmpi to the mpv-nvmpi-nvv4l2dec.zip so please download it and try again (I added it to other zip files as well). For me it works well with JetPack 4.5.1. Please report back if it works for you (do not forget to edit mpv.conf to use hardware acceleration in mpv).

There is an open issue about this Submitting FFMPEG Patch · Issue #53 · jocover/jetson-ffmpeg · GitHub but NVMPI author did not reply to it. I do not know if the original author wants help with upstreaming (perhaps there are some reasons why upstreaming is not an option). This is not the only issue though, I ported the patches to current (git) ffmpeg version and experienced strange crashes when trying to reencode some files. So besides the improvements I did to the code, additional fixes are needed to get it working properly with recent ffmpeg version. This does not happen with older versions of ffmpeg, so the version I provided here should work correctly.

Anyway, even if the patches would get upstream, in the best case scenario updated ffmpeg would be included in Ubuntu 22.04 or 24.04 LTS, and it would take at least 1-2 years before Nvidia will support it after release. That means most Jetson Nano users would get updated ffmpeg by year 2024 or by 2026 (or never, since by then it may not be feasible to support Jetson Nano any longer). So I suppose this is why it is simpler to support 3rd party repository or provide zip files with debs than trying to upstream the patches, this way if somebody finds the packages they can get mpv and ffmpeg with hardware acceleration right now instead of waiting few years.

My guess Nvidia does not consider this a priority so they do not even include their own version of ffmpeg with nvv4l2_dec by default, and their deb package has some issues in Ubuntu 18.04 because they used much newer version than included in Ubuntu and they did not package it according with Ubuntu standard (perhaps this is why they do not include it by default).

Only open source was used, and NVMPI project is under permissive MIT license, so it should be possible. If maintainers of mpv/ffmpeg packages from these distros are interested, they are welcome to use included patches in zip files, or contact me if they need help to get them working with newer versions of mpv/ffmpeg. The same is true if somebody wants to create and support ppa repository, they are welcome to use my debs or patches.

1 Like

Fantastic!

After installing libnvmpi and libavcodec-extra57_3.4.8-0ubuntu0.2_arm64.deb I now have hardware accelerated mpv and it seems to work wonderfully with both 4K h264 and h265 vids. Veeeeery nice! Congrats in pulling off this long awaited feat. Great to hear it is only using open source code too!

I’m not aware of any other ARM based platforms that have working hw accel decoding of both h265 and h265 under mpv, are you? The RPi 4 can only handle h265 unfortunately (if you can get it to work, which I have not with mpv) and h264 is still more common.

Thanks a lot for this work. How may I undo the changes if needed, say for mpv or vlc ?
Maybe Docker containers should be better in this case ?

These are deb packages, so you can just uninstall or unhold them as usually (or install Ubuntu deb packages on top of them, then they will be uninstalled automatically). For example, you can unhold all packages by running sudo apt-mark unhold $(apt-mark showhold). In case of mpv you need to remove hwdec=nvmpi-copy from the config file because Ubuntu version of mpv does not support it.

I tried to make the packages as identical to Ubuntu packages as possible, but with added nvmpi and nvv4l2_dec support, so there should be no need to downgrade/remove them. If you encounter any issues which force you to remove them, please let me know.

I’ve been looking into open source hw accelerated video decoding under ARM Linux and the only supposedly working (I’ve not tested it) alternative I can find to what Nvidia offer is the Cedrus project for the Allwinner SOCs such as the H6 (eg the Orange Pi 3) and the H3 series.

https://linux-sunxi.org/Cedrus

The Orange Pi 3 only has 2 GB RAM but still has 4 USB3 ports and the spec sheet says it should be able to decode 4K@60fs h265 so it could be a little better than the Nano for video playback?

EDIT

I’ve just spoken to one of the cedrus devs. He says it can handle 60 fps @ 1080p but not at 4K yet.

Curiosity has got the better of me and I’ve ordered a Sunvell T95 MAX Android TV box. It cost me £36 plus £4 postage. It uses the Allwinner H6 chipset and you can install Armbian Linux on it so I’m going to see if it makes for a better (and cheaper) mpv box than a Jetson Nano but I won’t receive mine for a month or so yet.

Unlike the Nano, it comes complete with a case and remote control but it only has 100Mb NIC, 1x USB 3 and 1x USB 2 so the Nano is superior connectivity-wise in those respects. I believe Armbian for the H6 SoCs comes with mpv by default so it may be easier for a newb to get going than setting up a Nano.

I found that nvmpi , nvv4l2dec will use CPU decoding rather than NVDEC. The ones with -copy use the NVDEC

With CPU decoding the video was not smooth.

Apologies for hijacking this thread but I feel the need to correct myself re the state of hw accelerated video decoding under ARM Linux after having spoken to one of the cedrus devs on IRC about this.

It would seem nvmpi/nvv4l2dec is likely the best solution currently available for HW accelerated decoding under ‘desktop’ ARM Linux. I am currently aware of three alternatives - cedrus for Allwinner SoCs, rkvdec for Rockchip and Meson for Amlogic.

Armbian isn’t geared towards multimedia so I discovered that in order for me to use HW video decoding on my T95 Max / Allwinner H6 based TV box I need to rebuild the kernel, ffmpeg and mpv after adding some out-of-tree patches and even then it won’t be usable under X. I haven’t got round to this yet so I cannot confirm it actually works.

Using a decodebin will choose the best decoder, usually a nvv4l2*. That’s what i’ve found is the easiest. uridecodebin is even better since it configures a source and demuxer for you. For files you can just use a file:///path/to/the/video.mp4 as the uri parameter and it just works. uridecodebin uri=... ! autovideosink and you’re done. Or just gst-launch-1.0 /path/to/video.mp4 which will do just about that.

Hello Lissanro,

I got referenced to your post as I’m trying to get ffmpeg encoding in hardware.

I donwloaded the zip file for ffmpeg alone, but got the next error:

sudo apt -y --allow-change-held-packages --allow-downgrades install ./ffmpeg-nvmpi-nvv4l2dec/*.deb
Leyendo lista de paquetes... Hecho
E: Se ha suministrado el fichero no admitido ./ffmpeg-nvmpi-nvv4l2dec/*.deb en la línea de órdenes

Sorry is in Spanish. The error says it has been provided a non admited file in the command line.

Listing the files in the directory:

ls -l ffmpeg-3.4.8-nvmpi-nvv4l2dec
total 16848
-rw-r--r-- 1 jnebrera jnebrera 1570716 feb 18 10:03 ffmpeg_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera 2018468 feb 18 10:03 ffmpeg-doc_3.4.8-0ubuntu0.2_all.deb
-rw-r--r-- 1 jnebrera jnebrera 4581600 feb 18 10:04 libavcodec-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   25296 feb 18 10:04 libavcodec-extra_3.4.8-0ubuntu0.2_all.deb
-rw-r--r-- 1 jnebrera jnebrera 4096052 feb 18 10:04 libavcodec-extra57_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   78444 feb 18 10:03 libavdevice57_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   94484 feb 18 10:03 libavdevice-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera  894636 feb 18 10:03 libavfilter-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   25284 feb 18 10:03 libavfilter-extra_3.4.8-0ubuntu0.2_all.deb
-rw-r--r-- 1 jnebrera jnebrera  745928 feb 18 10:03 libavfilter-extra6_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera  852180 feb 18 10:03 libavformat57_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera 1066020 feb 18 10:03 libavformat-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   50304 feb 18 10:03 libavresample3_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   59584 feb 18 10:03 libavresample-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera  184152 feb 18 10:03 libavutil55_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera  293336 feb 18 10:03 libavutil-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   63960 feb 12 11:34 libnvmpi_2021-02-12-1_all.deb
-rw-r--r-- 1 jnebrera jnebrera   44608 feb 18 10:03 libpostproc54_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   45276 feb 18 10:03 libpostproc-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   56480 feb 18 10:03 libswresample2_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera   67144 feb 18 10:03 libswresample-dev_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera  127364 feb 18 10:03 libswscale4_3.4.8-0ubuntu0.2_arm64.deb
-rw-r--r-- 1 jnebrera jnebrera  145284 feb 18 10:03 libswscale-dev_3.4.8-0ubuntu0.2_arm64.deb
drwxrwxr-x 2 jnebrera jnebrera    4096 feb 21 09:53 patches
-rw-rw-r-- 1 jnebrera jnebrera    1194 feb 21 10:31 README.txt

It is clear the .deb files are there, but don’t know why, it complaints the *.deb format as unsupported.

I’m using a Nvidia Jetpack 4.5.1 just installed, with all packages updated and cmake upgraded to latest from source.

I need to confirm my Jetson Nano 2GB board will be able to cope with encoding at realtime in my python project. In software alone is unable to do so (like 7FPS).

I discovered my alternative Vizi AI board will be unable too, as the Atom with HD 500 + Myriad X is unable to reach that performance (mainly because I just discovered the Myriad X Video Processing Unit is not usable to encode video which is a very fun discovery considering the naming)

Thank you for any help. Kind regards

There is a typo, a missing *. Sorry about that. The command should have been:

sudo apt -y --allow-change-held-packages --allow-downgrades install ./ffmpeg-*nvmpi-nvv4l2dec/*.deb

Since I cannot edit the original post, I reuploaded the zip file so it would unpack into ffmpeg-nvmpi-nvv4l2dec directory (instead of ffmpeg-3.4.8-nvmpi-nvv4l2dec). This way provided command will work as is.

The * made the trick and packages installed.

But in next step:

sudo apt-mark hold $(find ffmpeg-nvmpi-nvv4l2dec | grep deb | xargs -n1 basename | cut -d_ -f1)
find: ‘ffmpeg-nvmpi-nvv4l2dec’: No such file or directory
basename: missing operand
Try 'basename --help' for more information.
E: No se encontró ningún paquete

In Spanish it says No package was found

The same issue, a missing *. It should have been:

sudo apt-mark hold $(find ffmpeg-*nvmpi-nvv4l2dec | grep deb | xargs -n1 basename | cut -d_ -f1)

That made the trick. Now I only have to test if it did work :D

Ok, I run:

ffmpeg -y -vsync 0 -hwaccel vaapi -hwaccel_output_format vaapi -i sample.mp4 -c:a copy -c:v h264_nvmpi -b:v 5M output.mp4

An it run !!! Around 70fps with a 1920x1080 file in my Jetson Nano 2GB

The command I used is inspired by the one found in the NVIDIA documentation on how to compile ffmpeg with cuda support, just replacing cuda with vaapi

It is true that while running, I see this message as output:

[AVHWDeviceContext @ 0x55c3e76ec0] Failed to initialise VAAPI connection: -1 (unknown libva error).
Device creation failed: -5.
[h264 @ 0x55c3e7f670] No device available for decoder (device type vaapi for codec h264).
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvmpi))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[h264 @ 0x55c3fc5ff0] A hardware device or frames context is required for VAAPI decoding.

That seems related to the decoding part that worries me less as the python library I’m using doesn’t enable me to pass parameters to ffmpeg, but it does for encoding !!!

1 Like

Assuming your sample.mp4 uses h264 format, in your command you are missing -c:v h264_nvmpi before -i (position is important, -c:v before -i tells ffmpeg how to decode the stream, and after -i it says how to encode it). Also, options -hwaccel vaapi -hwaccel_output_format vaapi are not needed and just cause errors as far as I can tell. However, the errors seem to be harmless and do not affect performance. But providing just a single -c:v option instead of both does affect the performance.

I suggest to use the following command instead:

ffmpeg -y -vsync 0 -c:v h264_nvmpi -i sample.mp4 -c:a copy -c:v h264_nvmpi -b:v 5M output.mp4

You may need to use hevc_nvmpi instead of h264_nvmpi if sample.mp4 contains HEVC stream.

1 Like

Just an update. I could make docker containers out of the packages and it runs well. Since I am running Ubuntu 20.04 on nano I could not install as it were.

https://forums.developer.nvidia.com/t/docker-container-runtime-issue/173436/8