NV_ENC_ERR_UNSUPPORTED_DEVICE in a TCC device when one WDDM has "hardware scheduling" activated

In a system with 3 RTX 4000 GPU’s, under Windows, we configure 2 of them in TCC and the other one in WDDM.

The WDDM GPU is required since it has an screen plugged, and the user interacts with the system, using it.

We use nvenc API to encode FullHD videos in all the 3 GPU’s.

If we enable “hardware scheduling” in the WDDM GPU, CUDA things run faster, but nvenc returns NV_ENC_ERR_UNSUPPORTED_DEVICE when trying to use any of the TCC GPU’s.

Is this a known bug? Do we have to do any change in the code?

Actually, we had to modify the nvenc encoding examples in order to support TCC GPU’s, because they are not WDDM nor Linux, but something in between. It works as long as we don’t activate “hardware scheduling”.

Is anywhere a code example to use nvenc with TCC GPU’s under Windows?

System configuration:
OS: Windows 10 Enterprise (21H1)
NVIDIA driver: 462.59 (WHQL)

CPU: AMD EPYC 7401P 24-Core Processor (configured as a single NUMA node of 24 cores)
RAM: 64GB 2666MHz
GPU’s: 3x Quadro RTX 4000

We are looking into this issue and tracked internally as 200736151.

we had to modify the nvenc encoding examples in order to support TCC GPU’s

Can you share these changes?


Hi Mandar,

The changes where done in order to have nvenc working on TCC gpu’s, and it works with a single WDDM GPU and two TCC GPU’s. It stops working when the WDDM GPU has “hardware scheduling” on.

The nvenc code we have is pretty old, is based on an example from Video Codek SDK 9.1, and we are using CUDA 10.1 and VC SDK 9.1 API.

The changes made affect the code inside “#if defined(_WIN32)”

Those macros assume two things, that we are on Windows (ok yes), but it is also assuming that we are using WDDM. And in this case it’s not always true. When the GPU is in TCC mode, some Windows API calls will not work, like the CreateEvent call, from “synchapi.h”

So we basically added code to detect wether the GPU being used is in WDDM or TCC, and then we execute or not some of this parts accordingly.

Other calls that will not work when the GPU is in TCC are: WaitForSingleObject, stOutputBfr.bWaitOnEvent = true; (has to be false), nvEncRegisterAsyncEvent and nvEncUnregisterAsyncEvent.