Cuda support for ffmpeg encoding with GTX1080

I’m having trouble getting transcoding supported on a debian 11 based VM (running Jellyfin via docker) with a GTX 1080 GPU. Keep getting an error suggesting CUDA not supported/ available.

I’ve dropped logs and details below. Let me know if anyone has been able to overcome this.

CPU: 8700k
GPU: Nvidia 1080
OS: TrueNAS SCALE-22.02.3
VM: Debian 11.4 x86_64 (installed without a desktop environment)
Docker: Official Jellyfin docker image

how I installed nvidia drivers and the container toolkit on the VM:

sudo add-apt-repository contrib
sudo add-apt-repository non-free

sudo apt-get update && sudo apt-get upgrade

# if you have a CPU with 64bit op-mode per 'lscpu | grep CPU':
sudo apt-get install linux-headers-amd64 -y

sudo apt-get install firmware-misc-nonfree -y

sudo apt-get install nvidia-driver -y

sudo reboot

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
    && curl -s -L | sudo apt-key add - \
    && curl -s -L$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update && sudo apt-get upgrade

sudo apt-get install nvidia-container-toolkit -y

sudo systemctl restart docker

docker-compose for jellyfin:

version: '3.8'
    image: jellyfin/jellyfin:latest
    container_name: jellyfin-nvidia
      - PUID=1000
      - PGID=1000
      - TZ=US/Central
      - JELLYFIN_PublishedServerUrl=${PUBLISHED_SERVER}
    user: 1000:1000
      - ${DATA_PATH}/jellyfin:/config
      - 8096:8096
      - 8920:8920 #optional
      - 7359:7359/udp #optional
      - 1900:1900/udp #optional
    restart: unless-stopped
            - capabilities: [gpu][/CODE]

Jellyfin transcoding settings:

per my GPU: Nvidia NVDEC - Wikipedia

Enable hardware decoding for:

Enable enhanced NVDEC decoder (checked)

Enable hardware encoding (checked) ← I can avoid the ffmpeg error noted (no cuda support) below if I uncheck this

Allow encoding in HEVC format (checked)

nvidia-smi on the VM, which also matches the nvidia-smi output on the container:

| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|   0  NVIDIA GeForce ...  On   | 00000000:00:08.0 Off |                  N/A |
| 32%   30C    P8    14W / 180W |      1MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |

| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|  No running processes found                                                 |

transcode error log:

ffmpeg version 5.1-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[AVHWDeviceContext @ 0x555d5df0be00] cu->cuInit(0) failed -> CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
Device creation failed: -542398533.
Failed to set value 'cuda=cu:0' for option 'init_hw_device': Generic error in an external library
Error parsing global options: Generic error in an external library


Hi there @buswedg!

Looking at the logs it seems all the setup and driver installation is correct.

What I suspect is some permission issue when running your ffmpeg command. I am not a Linux expert, but it might be connected to restrictive systemd behavior.

In any case, the driver category here might not give you the perfect exposure for your question. I will not move the topic, but I suggest you have a look at CUDA Setup and Installation - NVIDIA Developer Forums since there also docker issues are discussed.

I am sorry you are running into trouble and I hope you will be able to resolve it!