nvEncOpenEncodeSessionEx fails with NV_ENC_ERR_INVALID_DEVICE in Docker

Our application is running on both Windows and Linux. On Windows as a regular application (where it still works), and on Linux through Docker (where it’s now broken).

This was working fine for many years. Our docker container used the Nvidia base image nvidia/cudagl:11.1-base-ubuntu20.04 and everything just worked.

Recent changes to our build requirements for other, unrelated products required us to upgrade the docker OS to Ubuntu 22.04.

Seeing as the cudagl images are no longer maintained, it looked like we could just use a basic Ubuntu image as the base of our container, and let the Nvidia Container Toolkit manage everything for us.

So that’s what I tried: FROM ubuntu:22.04

Our application can see the host GPU just fine, and has access to CUDA.

OpenGL rendering works just fine.
Video decoding with nvDec also works just fine.

However, calling nvEncOpenEncodeSessionEx fails. For reasons that I don’t understand.

I tried to use nvidia/cuda:12.0.0-base-ubuntu22.04 as the base for our docker container, but that made no difference.

So I’m a little lost as to what I’m doing wrong.

Should nvEnc just work out of the box like OpenGL and nvDec? Or should I be doing something extra in my dockerfile (or how we run the container) to ensure that it works as before?

And if it should just work, how can I figure out what the problem is? The device shouldn’t be invalid. It’s the same device that worked before and continues to work for rendering and video decoding. (Nothing else has changed in that system for at least two years.)

(I should also point out that everything still works fine outside of Docker.)

Cheers,
James

Just to clarify, we’re using an OpenGL device for video encoding on Linux, not a CUDA one:

NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS sessionParam = {};

sessionParam.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER;
sessionParam.deviceType = NV_ENC_DEVICE_TYPE_OPENGL;
sessionParam.apiVersion = NVENCAPI_VERSION;

ret = api.nvEncOpenEncodeSessionEx(&sessionParam, &encoder);

So the solution was to use the Nvidia CUDA image as a base:

FROM nvidia/cuda:12.0.0-base-ubuntu22.04

and manually install some Open GL libraries in the docker file:

apt -y install libglvnd-dev libgl1-mesa-glx libegl1-mesa

Which, I guess, does the same as the old Nvidia CUDAGL image we were using previously.

But I still don’t understand why this is necessary. (The Nvidia documentation really doesn’t explain anything about what is actually needed here.) Our NvDec-based video decoder uses CUDA (and that was working fine), and our renderer uses OpenGL (which was also working fine).

This smells like a bug somewhere to me.