HEVC NVENC with uhq tuning profile in ffmpeg segfaults at End of File

Gaming and Visualization Technologies Video Codec, PyNv & OFA
, , ,
1

I am attempting to use ffmpeg to transcode some Blu-Ray rips. When I use the hevc_nvenc codec with the ‘-tune uhq’ flag, ffmpeg will segfault at the very end of the transcoding session, dropping the last 10-15 seconds of video and leaving a malformed but playable mkv file.

Example:

ffmpeg -i 'Lava (2015).mkv' -c:v hevc_nvenc -tune uhq test.mkv

This does not occur with all input files, but will always occur given the same input file, and occurs consistently and reproducibly with most of the h264 Blu-Ray rips I have tested.

This occurs with the latest git build of ffmpeg on Arch Linux with a GTX 1660Ti Mobile and the latest Nvidia 580.95.05 Drivers

I initially reported this to the ffmpeg project, but based on the gdb/valgrind logs they determined that the crash was occuring in an nvidia library and that there was likely nothing they could do about it..

nvidia-bug-report.log.gz (729.0 KB)

2

Updating - I have been able to replicate this issue on a different laptop with an Nvidia RTX 3060, running Windows 11 and the Windows version of ffmpeg.

3

Same goes for av1_nvenc :(

In this case it happends on all sources compared to what OP is mentioning, atleast haven’t found anyone working

4

Thanks for reporting the issue.

>This does not occur with all input files, but will always occur given the same input file, and occurs consistently and reproducibly

Can you share the content that reproduces this issue to help us debug quickly?

5

I am working on this, but it may take some time. I have plenty of examples, but I am struggling to find one that is both small enough to upload and not copyright-encumbered. So far I have not been able to trigger the issue with any publicly available content.

6

Alright, I found one. This Blu-Ray rip of the public domain film Detour on Archive.org triggers the bug on both my Linux and Windows systems:

https://archive.org/download/detour.-1945.1080p.-blu-ray.-remux.-avc.-lpcm.-1.0-fgt/Detour.1945.1080p.BluRay.REMUX.AVC.LPCM.1.0-FGT.mkv

I ran ffmpeg on this as follows:

ffmpeg -i “./Detour.1945.1080p.BluRay.REMUX.AVC.LPCM.1.0-FGT.mkv” -c:v hevc_nvenc -tune uhq -c:a copy -c:s copy ./test.mkv

7

I’ve also run into this problem. I’m on fedora 43 workstation, using a NVIDIA GeForce RTX 5070 with drivers 580.95.05. To confirm the OP, I’ve only been able to reproduce this segfaults encoding from h264 using the -tune:v uhq, it does not happen using -tune:v hq.

I’ve not been able to find a wiki that explains the parameters that uhq uses over hq, does one exist ? I’d be willing to run some tests to help pin point what could be causing this.

This post Improving Video Quality with the NVIDIA Video Codec SDK 12.2 for HEVC | NVIDIA Technical Blog gives a little detail regarding UHQ tuning info but its vague without specifics.

8

Any update on this serious bug. I am running latest ffmpeg 8 591.74 on win 11 and I was beating my head on this. If I put the uhq flag on the EOF the sync of muxing would just not complete and crash. Source is multiple h264 and VC1 blue ray (both HD and UHD) so this is general issues. It also doesnt like PGS titles either but it crashed either way.

For now I moved to hq tune, and other optimizations, here you go on video flags, in this config I can rip a blu-ray h264 HD around 300fps which is great, uhq was around 140 until it crashed at EOF.

— VIDEO (Blackwell AV1 Optimized) —

"-c:v", "av1_nvenc",
"-preset", "p7",
"-tune", "hq",                      # Stay on HQ to avoid segfault
"-cq", "32",
"-pix_fmt", "p010le",
"-rc-lookahead", "32",
"-spatial-aq", "1",
"-temporal-aq", "1",
"-aq-strength", "8",
"-multipass", "fullres",
"-b_ref_mode", "each",              # High-efficiency B-frame referencing
"-split_encode_mode", "forced",     # Multi-NVENC utilization
"-b:v", "0",

# --- AUDIO (Opus 5.1) ---
"-map", "0:v:0", 
"-map", "0:a:0", 
"-af", "aformat=channel_layouts=5.1",
"-c:a", "libopus", "-b:a", "256k", "-vbr", "on", "-mapping_family", "1",

# --- SUBTITLES (PGS Testing) ---
"-map", "0:s?",                     # Map all subtitle streams if they exist
"-c:s", "copy",                     # Try copy first; change to 'pgs' if re-encoding

# --- STABILITY & FLUSH ---
"-max_muxing_queue_size", "8192",
"-max_interleave_delta", "0",

Card: 5070ti Zotac.

9

Is there any intent to patch this issue? I’m having the same issue on two different cards in both Linux and Windows.

Cards: 3090 FE & 4060

10

Hi everyone, I reproduced locally and root-caused to the driver’s temporal-filter lookahead path (which -tune uhq silently enables via -tf_level 4). The crash triggers only when the input carries SEI metadata that FFmpeg forwards to NVENC (closed captions, timecode, user-data-unregistered, HDR, etc.). I filed a bug internally. As soon as I have any update, I will update this thread.

Temporary workarounds while we wait for a driver fix — pick either:

  1. Add -extra_sei 0 to your FFmpeg command:

    ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mkv -c:v hevc_nvenc -tune uhq -extra_sei 0 output.mkv

    This disables FFmpeg’s user-SEI forwarding to NVENC.

  2. Decode on the GPU via h264_cuvid / hevc_cuvid — CUVID deoders do not propagate input SEI side data to FFmpeg frames, so nothing reaches NVENC to trigger the bug:

    ffmpeg -c:v h264_cuvid -i input.mkv -c:v hevc_nvenc -tune uhq output.mkv

Side effect of both: any A/53 closed captions, timecode, or user-data SEI from the source won’t be carried into the encoded output. If you don’t need those, either workaround is clean.

av1_nvenc -tune uhq is affected the same way, and the same workarounds apply.

11

Update: This issue appears to be fixed in driver 610.43.02.

I re-tested with the Detour clip on an RTX 5090 running driver 610.43.02 (CUDA 13.3) and ffmpeg 8.0 (git HEAD). All three previously-crashing scenarios now complete without a segfault:

Command Result
hevc_nvenc -tune uhq ✅ Completed
hevc_nvenc -tune hq -tf_level 4 -bf 4 ✅ Completed
av1_nvenc -tune uhq ✅ Completed

This covers the HEVC + UHQ case from the OP, the explicit temporal-filter path (-tf_level 4), and the AV1 variant reported by @raymond35.