Multiple encoder sometime fails on NvEncRegisterResource

We have a use case where a single process can encode a single stream for multiple rates. The input to NVenc is CUDA surfaces so we go through the NvEncRegisterResource facility for ingestion. This generally works very well but we sometimes get NV_ENC_ERR_RESOURCE_REGISTER_FAILED as a result.

This problem has been around for a while and still occurs with the latest 381.22 driver. As a workaround we simply ignore the error and retry registering, which work most of the time, but there are cases where the retry won’t work for too many iterations and we have to bail out by dropping the frame. Once in this state, it can take as much as 10 frames to recuperate. Obviously this isn’t desirable as it is noticeable on playback.

We’re running CentOS 7.3 with Kernel 3.10.0-514.21.1.el7.x86_64, our hardware varies but the behavior is consistent with these cards:

  • Quadro M6000
  • TITAN X (Pascal)
  • TITAN Xp

Because Linux is bullshit, Nvidia codec SDK is only Windows 10.