I’m using FFmpeg to apply a GLSL .frag shader to a video. I do it with this command
docker run --rm \
--gpus all \
--device /dev/dri \
-v $(pwd):/config \
lscr.io/linuxserver/ffmpeg \
-init_hw_device vulkan=vk:0 -v verbose \
-i /config/input.mp4 \
-vf "libplacebo=custom_shader_path=/config/shader.frag" \
-c:v h264_nvenc \
/config/output.mp4 \
2>&1 | less -F
but the extremely low speed made me suspicious
frame=16 fps=0.3 q=45.0 size=0KiB time=00:00:00.43 bitrate=0.9kbits/s speed=0.00767x elapsed=0:00:56.52
The CPU activity was at 99.3% and the GPU at 0%. So I searched through the verbose output and found this:
[Vulkan @ 0x63691fd82b40] Using device: llvmpipe (LLVM 18.1.3, 256 bits)
For context:
I’m using an EC2 instance (g6f.xlarge) with ubuntu 24.04.
I’ve installed the NVIDIA GRID drivers following the official AWS guide, and the NVIDIA Container Toolkit following this other guide.
Vulkan can see the GPU outside of the container
ubuntu@ip-172-31-41-83:~/liquid-glass$ vulkaninfo | grep -A2 "deviceName"
'DISPLAY' environment variable not set... skipping surface info
deviceName = NVIDIA L4-3Q
pipelineCacheUUID = 178e3b81-98ac-43d3-f544-6258d2c33ef5
More info that might be useful
ubuntu@ip-172-31-41-83:~$ docker run --rm --gpus all --device /dev/dri nvidia/vulkan:1.3-470 vulkaninfo --summary
Unable to find image 'nvidia/vulkan:1.3-470' locally
1.3-470: Pulling from nvidia/vulkan
ea362f368469: Pull complete
33ad2f6e0261: Pull complete
91ce2d96e937: Pull complete
9b499af792c6: Pull complete
c04906891ebc: Pull complete
ebf96a6776b2: Pull complete
dfd3c8943767: Pull complete
49eaa5311171: Pull complete
Digest: sha256:de1866cacf9cf0edad969a07504b0aaf24da57fc069b166a913b33188fe60231
Status: Downloaded newer image for nvidia/vulkan:1.3-470
Cannot create Vulkan instance.
This problem is often caused by a faulty installation of the Vulkan driver or attempting to use a GPU that does not support Vulkan.
ERROR at /vulkan-sdk/1.3.204.1/source/Vulkan-Tools/vulkaninfo/vulkaninfo.h:649:vkCreateInstance failed with ERROR_INCOMPATIBLE_DRIVER
ubuntu@ip-172-31-41-83:~$ docker info | grep -i nvidia
Runtimes: nvidia runc io.containerd.runc.v2
ubuntu@ip-172-31-41-83:~$ ls -la /usr/share/vulkan/icd.d/
total 36
drwxr-xr-x 2 root root 4096 Sep 17 18:13 .
drwxr-xr-x 6 root root 4096 Sep 17 18:13 ..
-rw-r--r-- 1 root root 167 Aug 18 08:26 gfxstream_vk_icd.x86_64.json
-rw-r--r-- 1 root root 169 Aug 18 08:26 intel_hasvk_icd.x86_64.json
-rw-r--r-- 1 root root 163 Aug 18 08:26 intel_icd.x86_64.json
-rw-r--r-- 1 root root 161 Aug 18 08:26 lvp_icd.x86_64.json
-rw-r--r-- 1 root root 165 Aug 18 08:26 nouveau_icd.x86_64.json
-rw-r--r-- 1 root root 164 Aug 18 08:26 radeon_icd.x86_64.json
-rw-r--r-- 1 root root 164 Aug 18 08:26 virtio_icd.x86_64.json
ubuntu@ip-172-31-41-83:~$ ls -la /dev/dri/
total 0
drwxr-xr-x 3 root root 120 Sep 17 18:38 .
drwxr-xr-x 17 root root 3520 Sep 17 19:19 ..
drwxr-xr-x 2 root root 100 Sep 17 18:38 by-path
crw-rw---- 1 root video 226, 0 Sep 17 18:38 card0
crw-rw---- 1 root video 226, 1 Sep 17 18:38 card1
crw-rw---- 1 root render 226, 128 Sep 17 18:38 renderD128
ubuntu@ip-172-31-41-83:~$ dpkg -l | grep nvidia
ii libnvidia-container-tools 1.17.8-1 amd64 NVIDIA container runtime library (command-line tools)
ii libnvidia-container1:amd64 1.17.8-1 amd64 NVIDIA container runtime library
ii nvidia-container-toolkit 1.17.8-1 amd64 NVIDIA Container toolkit
ii nvidia-container-toolkit-base 1.17.8-1 amd64 NVIDIA Container Toolkit Base
ubuntu@ip-172-31-41-83:~/liquid-glass$ sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
Thu Sep 18 10:41:16 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.172.08 Driver Version: 570.172.08 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
What I tried
- I tried locating the
nvidia_icd.jsonfile and passing it manually in two different ways
docker run --rm \
--gpus all \
--device /dev/dri \
-v $(pwd):/config \
-v /etc/vulkan/icd.d:/etc/vulkan/icd.d \
-v /usr/share/vulkan/icd.d:/usr/share/vulkan/icd.d \
lscr.io/linuxserver/ffmpeg \
-init_hw_device vulkan=vk:0 -v verbose \
-i /config/input.mp4 \
-vf "libplacebo=custom_shader_path=/config/shader.frag" \
-c:v h264_nvenc \
/config/output.mp4 \
2>&1 | less -F
docker run --rm \
--gpus all \
--device /dev/dri \
-v $(pwd):/config \
-v /etc/vulkan/icd.d:/etc/vulkan/icd.d \
-e VULKAN_ICD_FILENAMES=/etc/vulkan/icd.d/nvidia_icd.json \
-e NVIDIA_VISIBLE_DEVICES=all \
-e NVIDIA_DRIVER_CAPABILITIES=all \
lscr.io/linuxserver/ffmpeg \
-init_hw_device vulkan=vk:0 -v verbose \
-i /config/input.mp4 \
-vf "libplacebo=custom_shader_path=/config/shader.frag" \
-c:v h264_nvenc \
/config/output.mp4 \
2>&1 | less -F
- I tried installing other packages that ended up breaking the NVIDIA driver (then I wiped everything out and started again from scratch with new working drivers)
sudo apt install nvidia-driver-570 nvidia-utils-570
ubuntu@ip-172-31-41-83:\~$ nvidia-smi
NVIDIA-SMI couldn't find libnvidia-ml.so library in your system. Please make sure that the NVIDIA Display Driver is properly installed and present in your system. Please also try adding directory that contains libnvidia-ml.so to your system PATH.
- I tried setting
vk:1instead ofvk:0
[Vulkan @ 0x5febdd1e7b40\] Supported layers:
[Vulkan @ 0x5febdd1e7b40\] GPU listing:
[Vulkan @ 0x5febdd1e7b40\] 0: llvmpipe (LLVM 18.1.3, 256 bits) (software) [Vulkan @ 0x5febdd1e7b40\] Unable to find device with index 1!
-
Tried both adding
--volume /dev/dri:/dev/driand putting it instead of--device /dev/dri. Also removing--device /dev/dri. -
Adding
--runtime=nvidia -
Giving full permissions
sudo chmod 777 /dev/dri/*