HEVC NVENC with uhq tuning profile in ffmpeg segfaults at End of File

Gaming and Visualization Technologies Video Processing & Optical Flow
, , ,
1

I am attempting to use ffmpeg to transcode some Blu-Ray rips. When I use the hevc_nvenc codec with the ‘-tune uhq’ flag, ffmpeg will segfault at the very end of the transcoding session, dropping the last 10-15 seconds of video and leaving a malformed but playable mkv file.

Example:

ffmpeg -i 'Lava (2015).mkv' -c:v hevc_nvenc -tune uhq test.mkv

This does not occur with all input files, but will always occur given the same input file, and occurs consistently and reproducibly with most of the h264 Blu-Ray rips I have tested.

This occurs with the latest git build of ffmpeg on Arch Linux with a GTX 1660Ti Mobile and the latest Nvidia 580.95.05 Drivers

I initially reported this to the ffmpeg project, but based on the gdb/valgrind logs they determined that the crash was occuring in an nvidia library and that there was likely nothing they could do about it..

nvidia-bug-report.log.gz (729.0 KB)

2

Updating - I have been able to replicate this issue on a different laptop with an Nvidia RTX 3060, running Windows 11 and the Windows version of ffmpeg.

3

Same goes for av1_nvenc :(

In this case it happends on all sources compared to what OP is mentioning, atleast haven’t found anyone working

4

Thanks for reporting the issue.

>This does not occur with all input files, but will always occur given the same input file, and occurs consistently and reproducibly

Can you share the content that reproduces this issue to help us debug quickly?

5

I am working on this, but it may take some time. I have plenty of examples, but I am struggling to find one that is both small enough to upload and not copyright-encumbered. So far I have not been able to trigger the issue with any publicly available content.

6

Alright, I found one. This Blu-Ray rip of the public domain film Detour on Archive.org triggers the bug on both my Linux and Windows systems:

https://archive.org/download/detour.-1945.1080p.-blu-ray.-remux.-avc.-lpcm.-1.0-fgt/Detour.1945.1080p.BluRay.REMUX.AVC.LPCM.1.0-FGT.mkv

I ran ffmpeg on this as follows:

ffmpeg -i “./Detour.1945.1080p.BluRay.REMUX.AVC.LPCM.1.0-FGT.mkv” -c:v hevc_nvenc -tune uhq -c:a copy -c:s copy ./test.mkv

7

I’ve also run into this problem. I’m on fedora 43 workstation, using a NVIDIA GeForce RTX 5070 with drivers 580.95.05. To confirm the OP, I’ve only been able to reproduce this segfaults encoding from h264 using the -tune:v uhq, it does not happen using -tune:v hq.

I’ve not been able to find a wiki that explains the parameters that uhq uses over hq, does one exist ? I’d be willing to run some tests to help pin point what could be causing this.

This post Improving Video Quality with the NVIDIA Video Codec SDK 12.2 for HEVC | NVIDIA Technical Blog gives a little detail regarding UHQ tuning info but its vague without specifics.

8

Any update on this serious bug. I am running latest ffmpeg 8 591.74 on win 11 and I was beating my head on this. If I put the uhq flag on the EOF the sync of muxing would just not complete and crash. Source is multiple h264 and VC1 blue ray (both HD and UHD) so this is general issues. It also doesnt like PGS titles either but it crashed either way.

For now I moved to hq tune, and other optimizations, here you go on video flags, in this config I can rip a blu-ray h264 HD around 300fps which is great, uhq was around 140 until it crashed at EOF.

— VIDEO (Blackwell AV1 Optimized) —

"-c:v", "av1_nvenc",
"-preset", "p7",
"-tune", "hq",                      # Stay on HQ to avoid segfault
"-cq", "32",
"-pix_fmt", "p010le",
"-rc-lookahead", "32",
"-spatial-aq", "1",
"-temporal-aq", "1",
"-aq-strength", "8",
"-multipass", "fullres",
"-b_ref_mode", "each",              # High-efficiency B-frame referencing
"-split_encode_mode", "forced",     # Multi-NVENC utilization
"-b:v", "0",

# --- AUDIO (Opus 5.1) ---
"-map", "0:v:0", 
"-map", "0:a:0", 
"-af", "aformat=channel_layouts=5.1",
"-c:a", "libopus", "-b:a", "256k", "-vbr", "on", "-mapping_family", "1",

# --- SUBTITLES (PGS Testing) ---
"-map", "0:s?",                     # Map all subtitle streams if they exist
"-c:s", "copy",                     # Try copy first; change to 'pgs' if re-encoding

# --- STABILITY & FLUSH ---
"-max_muxing_queue_size", "8192",
"-max_interleave_delta", "0",

Card: 5070ti Zotac.