seg fault with ffmpeg when using -rc-lookahead > 0

When transcoding content using ffmpeg libraries if rc-lookahead is configured with any value above zero ffmpeg immediatly seg faults with the following stack trace. I’m using latest ffmpeg build.

Here the settings used in the command line

-b:v 5.5M -c:v:0 h264_nvenc -rc-lookahead:v 30 -preset llhq -cbr 1 -coder cabac -profile:v high -level:v 4.1 -gpu 0 -refs:v 4 -g:v 116 -keyint_min 116 -strict_gop 1 -b_adapt 0 -bf 2 -no-scenecut 1 -forced-idr 1 -zerolatency 0 -aud 1 -rc cbr_hq

Thread 1 “ffmpeg” received signal SIGSEGV, Segmentation fault.
0x00007ffff60d6c47 in __memmove_sse2_unaligned_erms () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007ffff60d6c47 in __memmove_sse2_unaligned_erms () from /usr/lib/libc.so.6
#1 0x00007fffd0aae0b7 in ?? () from /usr/lib/libnvidia-encode.so.1
#2 0x00007fffd0aae347 in ?? () from /usr/lib/libnvidia-encode.so.1
#3 0x00007fffd0aaa7eb in ?? () from /usr/lib/libnvidia-encode.so.1
#4 0x00007fffd0aaa89e in ?? () from /usr/lib/libnvidia-encode.so.1
#5 0x00007fffd0aaad9b in ?? () from /usr/lib/libnvidia-encode.so.1
#6 0x00007fffd0aa9659 in ?? () from /usr/lib/libnvidia-encode.so.1
#7 0x00007fffd0abb482 in ?? () from /usr/lib/libnvidia-encode.so.1
#8 0x00007ffff6a780cf in ff_nvenc_send_frame (avctx=0x555555655180, frame=0x555556b6e440) at libavcodec/nvenc.c:2104
#9 0x00007ffff67761f6 in avcodec_send_frame (avctx=0x555555655180, frame=0x555556b6e440) at libavcodec/encode.c:403
#10 0x0000555555579594 in do_video_out (of=0x5555555fb340, ost=0x555555681d40, next_picture=0x555556b6e440, sync_ipts=)
at fftools/ffmpeg.c:1286
#11 0x000055555557cad7 in reap_filters (flush=0) at fftools/ffmpeg.c:1503
#12 0x0000555555581441 in transcode_step () at fftools/ffmpeg.c:4645
#13 transcode () at fftools/ffmpeg.c:4689
#14 0x00005555555607ee in main (argc=302, argv=0x7fffffffd0d8) at fftools/ffmpeg.c:4891

Version used
nvidia-smi
Thu Jan 17 10:18:20 2019
±----------------------------------------------------------------------------+
| NVIDIA-SMI 410.57 Driver Version: 410.57 |
|-------------------------------±---------------------±---------------------+

Hi, could you provide the following information:

  1. SDK version
  2. Driver version
  3. GPU being used

Thanks,
Ryan Park

SDK = 10.0.130
Driver = 410.57
GPU - I’ve tested on Tesla P100, GTX 1060/1080 and Quadro P5000. All with same result.

As this is the first result on google when I search for - rc lookahead nvidia not working - I’d like to say that I’m running into the same issues, attempting to use ffmpeg with -rc-lookahead of any value other than 0 leads to an immediate segfault. Without running a debug build of ffmpeg it only shows that an error has occurred, but when I run with debug I get a stack track that looks exactly like Gabe’s – Except the nvenc.c and encode.c lines are a little off as I’m using a different commit of ffmpeg.

  1. SDK - 10.0.130
  2. Driver - Anything from 410.57 to current. I haven’t dived deep enough to see exactly which revision it broke on.
  3. GTX 1080

I’ve also noticed some other forum posts (Mainly on geforce.com) about various programs like OBS/youcam not working recently, with roughly the same driver revisions. They’re probably using rc_lookahead as well.

I found that by decoding the content using the GPU as well, rc-lookahead works. Even more interesting is that if I start a second decoder without the workaround - it does not seg fault.

That’s interesting, after some testing it seems that if the frames are kept on the GPU it works, but when ffmpeg has to copy from GPU —> RAM —> GPU it fails. So running

-hwaccel cuvid -c:v h264_cuvid
-hwaccel nvdec
both work

-hwaccel dxva2
-hwaccel cuvid -c:v h264_cuvid -vf hwupload,tonemap_opencl=t=bt709:tonemap=hable:format=nv12,hwdownload,format=nv12
Will not work, as both commands require copying the frames from vram to ram to vram.

@gabe1938182 and @collisionc, which version of FFmpeg was used in the earlier comments? Can you provide a link to a sample input sequence with which this bug is reproducible?

Have you solved this problem?

A bug report has been filed internally for this and an engineer has been assigned to look into it, but i can’t say when the fix will be available.

It looks like this problem appears only for streams with closed captions. With ffmpeg built at or after commit dc2d56a66b8ad8b, -a53cc 0 can be appended to the ffmpeg command line to bypass the problematic code path, until this bug is fixed.

This makes sense - although we need the closed captioning so hopefully this gets fixed quickly.

Hello, any updates on this? Having same issue. Playing live without -rc_lookahead results in video stutters.