NvEncLockBitstream sometimes returns NV_ENC_ERR_INVALID_PARAM

I have a bug in my integration of NVENC where locking the bitstream for the first frame will sometimes return with NV_ENC_ERR_INVALID_PARAM. Most of the time the code works fine.

If I let this error occur a couple times, it can break the system; making any CUDA call block indefinitely. This can only be solved by rebooting the machine.

The problem happens using driver 378.13 and Video SDK 7.1.9 on CentOS 7.3 hosts, one with a Quadro M6000, the other with a Titan X.

The error is not very verbose, I suppose the bitstream buffer is the cause somehow, but I’ve checked the handles and they’re all valid. Maybe a bug in the driver itself?

Inspecting the process with cuda-gdb, after the error occurs, there’s one thread stuck here:

Thread 2 (Thread (LWP 16551)):
#0  in cuVDPAUCtxCreate () from /usr/lib/libcuda.so.1
#1  in cudbgApiDetach () from /usr/lib/libcuda.so.1
#2  in ?? () from /usr/lib/libcuda.so.1
#3  in cuArrayDestroy () from /usr/lib/libcuda.so.1
#4  in cuvidCreateVideoParser () from /usr/lib/libnvcuvid.so.1
#5  in cuvidCtxLockCreate () from /usr/lib/libnvcuvid.so.1
#6  in cuvidCreateVideoParser () from /usr/lib/libnvcuvid.so.1
#7  in cuvidCreateVideoSource () from /usr/lib/libnvcuvid.so.1
#8  in ?? () from /usr/lib/libnvidia-encode.so.1
#9  in ?? () from /usr/lib/libnvidia-encode.so.1
#10 in ?? () from /usr/lib/libnvidia-encode.so.1
#11 in NvEncodeAPIGetMaxSupportedVersion () from /usr/lib/libnvidia-encode.so.1
#12 in nvenc_ctx_clear (ctx=0x7ffff24813b0) at nvenc.c:184

This is most likely due to a known bug, which has been fixed in our R381 branch driver and upcoming (yet unreleased) drivers.

The fix is not present in 378.xx drivers.

Can you please check if the problem reproduces with driver here: http://www.nvidia.com/Download/driverResults.aspx/117002/en-us

Okay we’ve reverted to driver 375 and can’t reproduce the issue. We’ll test the beta driver shortly and report back.

Confirmed, the beta driver 381.09 doesn’t have the issue, and is even more stable than 375.39

Hi Abhijit, can you say whether driver versions prior to 378.xx (352.101 in particular) are affected by this bug? I don’t have admin access on the machine I’m working with, so I’m unable to try upgrading to the new drivers. Is there any alternative solution besides upgrading the driver?

Details: Tesla K20X / GK110, running under Linux with the 352.101 driver. I have two versions of my code, one which converts to YUV on the CPU and uses NVENC input buffers, and one which converts to YUV in CUDA and uses registered and mapped CUDA resources as input buffers. The two versions are otherwise identical, and all code relating to the output bitstreams is the same. However, the version using CUDA interop fails at the first NvEncLockBitstream with a NV_ENC_INVALID_PARAM error.

Hi @Nleaf, 352.xx drivers do not have the bugfix. Unfortunately, there is no alternative.

Is this fix present in new release 375.66?

Hi Abhijit,

I am using Tesla V100 on x64 Linux Ubuntu 16.04. Installed driver 384.9. When I try to run NvEncoder sample application, I am getting below error.

…/common/src/NvHWEncoder.cpp line 1068: lock bitstream function failed

nvdia-smi output is below:

Thu Dec 7 10:35:56 2017
| NVIDIA-SMI 384.90 Driver Version: 384.90 |
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| 0 Tesla V100-SXM2… Off | 00000000:00:1E.0 Off | 0 |
| N/A 37C P0 44W / 300W | 512MiB / 16152MiB | 0% Default |

| Processes: GPU Memory |
| GPU PID Type Process name Usage |
| 0 1662 C ./NvEncoder 501MiB |

Can you please help?

Best Regards,

Hi Abhijit,

To add to above, I’m getting same issue using FFmpeg also. Below is ffmpeg cli:

ffmpeg -i big_buck_bunny_1080p_h264.mov -c:v h264_nvenc bbb.mp4

Error is below. Process hangs until killed.

[h264_nvenc @ 0x16321c0] Failed locking bitstream buffer: invalid param (8)

Can you please help?

Best Regards,