Video decoding:

tamirg · May 28, 2019, 2:56pm

Hi,
I’m having issues decoding a video with “-c:v h264_cuvid”, while “-hwaccel cuda” succeeds.

-c:v h264_cuvid
-------------------
command line:

ffmpeg.exe -loglevel verbose -c:v h264_cuvid -i "video.mkv" -f null -

output:

ffmpeg version 4.0.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64
  configuration: --enable-asm --enable-yasm --enable-nvdec --disable-doc --enable-shared --disable-static --disable-bzlib --disable-libopenjpeg --disable-iconv --disable-zlib --disable-dxva2 --disable-d3d11va --enable-decoder=tscc --toolchain=msvc --arch=x86_64 --extra-cflags=-MD --extra-ldflags='/NODEFAULTLIB:libcmt' --enable-cuda --enable-cuvid --enable-nvenc --enable-libopenh264 --extra-cflags=-I/d/Sources/ExternalLibs/CudaToolkit_10_0/include --extra-ldflags='/LIBPATH:D:/Sources/ExternalLibs/gperftools-master/x64/Release-Patch' --extra-ldflags=libtcmalloc_minimal.lib --extra-ldflags='/INCLUDE:__tcmalloc' --prefix=/d/Sources/5.4.1_1/ExternalLibs/ffmpeg_4_0_BC
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
[h264 @ 0000020BA397EDC0] Ignoring NAL type 0 in extradata
    Last message repeated 1 times
[h264 @ 0000020BA397EDC0] Ignoring NAL type 6 in extradata
[h264 @ 0000020BA397EDC0] Ignoring NAL type 0 in extradata
    Last message repeated 1 times
[h264 @ 0000020BA397EDC0] Ignoring NAL type 6 in extradata
[h264 @ 0000020BA397EDC0] sps_id 32 out of range
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (1677732013 > 79990).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 79999
[h264 @ 0000020BA397EDC0] SEI type 5 size 1416 truncated at 159
[h264 @ 0000020BA397EDC0] Reinit context to 1280x720, pix_fmt: yuv420p
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9562).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9571
[h264 @ 0000020BA397EDC0] SEI type 5 size 352 truncated at 159
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9474).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9483
[h264 @ 0000020BA397EDC0] SEI type 5 size 352 truncated at 159
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9467).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9476
[h264 @ 0000020BA397EDC0] SEI type 5 size 352 truncated at 159
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9429).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9438
[h264 @ 0000020BA397EDC0] SEI type 5 size 352 truncated at 159
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9572).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9581
[h264 @ 0000020BA397EDC0] SEI type 5 size 352 truncated at 159
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9347).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9356
[h264 @ 0000020BA397EDC0] SEI type 5 size 352 truncated at 159
[h264 @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9555).
[h264 @ 0000020BA397EDC0] missing picture in access unit with size 9564
Input #0, matroska,webm, from '\bcfs\Rndnew\Support\Tickets763# C118 Departures Front Curb2019-03-08T173118.414Z.mkv':
  Metadata:
    encoder         : JEBML v2.2
    creation_time   : 2050-09-03T14:26:01.397664Z
    VX_INITIAL_TIMESTAMP: 1552066278414
    VX_INITIAL_TIMESTAMP-eng: 1552066278414
    ExporterVersion : 2.0
    ExporterVersion-eng: 2.0
  Duration: 00:15:00.56, start: 0.000000, bitrate: 2866 kb/s
    Stream #0:0(eng): Subtitle: ass (forced)
    Metadata:
      title           : timestamp
    Stream #0:1(eng): Video: h264 (High), 1 reference frame, yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn, 60 tbc (forced)
    Metadata:
      title           : video track #2
[AVHWDeviceContext @ 0000020BA394E040] No context has been set. Creating a private context, degraded performance
[h264_cuvid @ 0000020BA397F240] CUVID capabilities for h264_cuvid:
[h264_cuvid @ 0000020BA397F240] 8 bit: supported: 1, min_width: 48, max_width: 4096, min_height: 16, max_height: 4096
[h264_cuvid @ 0000020BA397F240] 10 bit: supported: 0, min_width: 0, max_width: 0, min_height: 0, max_height: 0
[h264_cuvid @ 0000020BA397F240] 12 bit: supported: 0, min_width: 0, max_width: 0, min_height: 0, max_height: 0
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (h264_cuvid) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[NULL @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9546).
[NULL @ 0000020BA397EDC0] missing picture in access unit with size 9555
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[NULL @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9381).
[NULL @ 0000020BA397EDC0] missing picture in access unit with size 9390
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[NULL @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9327).
[NULL @ 0000020BA397EDC0] missing picture in access unit with size 9336
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[NULL @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9233).
[NULL @ 0000020BA397EDC0] missing picture in access unit with size 9242
[h264_cuvid @ 0000020BA397F240] av_bsf_receive_packet failed
Error while decoding stream #0:1: Invalid argument
[NULL @ 0000020BA397EDC0] Invalid NAL unit size (86775045 > 9294).
[NULL @ 0000020BA397EDC0] missing picture in access unit with size 9303

That continues until the video ends without decoding any frames

-hwaccel cuda
-----------------
command line:

ffmpeg.exe -loglevel verbose -hwaccel cuda -i "video.mkv" -f null -

output:

ffmpeg version 4.0.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64
  configuration: --enable-asm --enable-yasm --enable-nvdec --disable-doc --enable-shared --disable-static --disable-bzlib --disable-libopenjpeg --disable-iconv --disable-zlib --disable-dxva2 --disable-d3d11va --enable-decoder=tscc --toolchain=msvc --arch=x86_64 --extra-cflags=-MD --extra-ldflags='/NODEFAULTLIB:libcmt' --enable-cuda --enable-cuvid --enable-nvenc --enable-libopenh264 --extra-cflags=-I/d/Sources/ExternalLibs/CudaToolkit_10_0/include --extra-ldflags='/LIBPATH:D:/Sources/ExternalLibs/gperftools-master/x64/Release-Patch' --extra-ldflags=libtcmalloc_minimal.lib --extra-ldflags='/INCLUDE:__tcmalloc' --prefix=/d/Sources/5.4.1_1/ExternalLibs/ffmpeg_4_0_BC
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  3.100
  libavfilter     7. 16.100 /  7. 16.100
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
[h264 @ 00000222C87BEDC0] Ignoring NAL type 0 in extradata
    Last message repeated 1 times
[h264 @ 00000222C87BEDC0] Ignoring NAL type 6 in extradata
[h264 @ 00000222C87BEDC0] Ignoring NAL type 0 in extradata
    Last message repeated 1 times
[h264 @ 00000222C87BEDC0] Ignoring NAL type 6 in extradata
[h264 @ 00000222C87BEDC0] sps_id 32 out of range
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (1677732013 > 79990).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 79999
[h264 @ 00000222C87BEDC0] SEI type 5 size 1416 truncated at 159
[h264 @ 00000222C87BEDC0] Reinit context to 1280x720, pix_fmt: yuv420p
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9562).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9571
[h264 @ 00000222C87BEDC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9474).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9483
[h264 @ 00000222C87BEDC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9467).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9476
[h264 @ 00000222C87BEDC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9429).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9438
[h264 @ 00000222C87BEDC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9572).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9581
[h264 @ 00000222C87BEDC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9347).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9356
[h264 @ 00000222C87BEDC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9555).
[h264 @ 00000222C87BEDC0] missing picture in access unit with size 9564
Input #0, matroska,webm, from '\bcfs\Rndnew\Support\Tickets763# C118 Departures Front Curb2019-03-08T173118.414Z.mkv':
  Metadata:
    encoder         : JEBML v2.2
    creation_time   : 2050-09-03T14:26:01.397664Z
    VX_INITIAL_TIMESTAMP: 1552066278414
    VX_INITIAL_TIMESTAMP-eng: 1552066278414
    ExporterVersion : 2.0
    ExporterVersion-eng: 2.0
  Duration: 00:15:00.56, start: 0.000000, bitrate: 2866 kb/s
    Stream #0:0(eng): Subtitle: ass (forced)
    Metadata:
      title           : timestamp
    Stream #0:1(eng): Video: h264 (High), 1 reference frame, yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn, 60 tbc (forced)
    Metadata:
      title           : video track #2
[AVHWDeviceContext @ 00000222C878E040] No context has been set. Creating a private context, degraded performance
[h264 @ 00000222C87C08C0] Ignoring NAL type 0 in extradata
    Last message repeated 1 times
[h264 @ 00000222C87C08C0] Ignoring NAL type 6 in extradata
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[h264 @ 00000222C87C08C0] SEI type 5 size 1416 truncated at 159
[h264 @ 00000222C87C08C0] NVDEC capabilities:
[h264 @ 00000222C87C08C0] format supported: yes, max_mb_count: 65536
[h264 @ 00000222C87C08C0] min_width: 48, max_width: 4096
[h264 @ 00000222C87C08C0] min_height: 16, max_height: 4096
[h264 @ 00000222C87C08C0] Reinit context to 1280x720, pix_fmt: cuda
[h264 @ 00000222C87C0D40] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87C11C0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87C1640] SEI type 5 size 352 truncated at 159
[h264 @ 00000222C87C1AC0] SEI type 5 size 352 truncated at 159
[h264 @ 00000222CA3E4040] SEI type 5 size 352 truncated at 159
[h264 @ 00000222CA3E44C0] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9546).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9555
[h264 @ 00000222CA3E4940] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9381).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9390
[h264 @ 00000222CA3E4DC0] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9327).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9336
[h264 @ 00000222CA3E5240] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9233).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9242
[h264 @ 00000222CA3E56C0] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9294).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9303
[h264 @ 00000222CA3E5B40] SEI type 5 size 352 truncated at 159
[h264 @ 00000222CA3E5FC0] SEI type 5 size 352 truncated at 159
[graph 0 input from stream 0:1 @ 00000222C878AA80] w:1280 h:720 pixfmt:nv12 tb:1/1000 fr:30/1 sar:1/1 sws_param:flags=2
Output #0, null, to 'pipe:':
  Metadata:
    ExporterVersion : 2.0
    ExporterVersion-eng: 2.0
    VX_INITIAL_TIMESTAMP: 1552066278414
    VX_INITIAL_TIMESTAMP-eng: 1552066278414
    encoder         : Lavf58.12.100
    Stream #0:0(eng): Video: wrapped_avframe, 1 reference frame, nv12(left), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (forced)
    Metadata:
      title           : video track #2
      encoder         : Lavc58.18.100 wrapped_avframe
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9182).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9191
[h264 @ 00000222C87C08C0] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 6223).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 6232
[h264 @ 00000222C87C0D40] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9151).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9160
[h264 @ 00000222C87C11C0] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9264).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9273
[h264 @ 00000222C87C1640] SEI type 5 size 352 truncated at 159
[NULL @ 00000222C87BEDC0] Invalid NAL unit size (86775045 > 9092).
[NULL @ 00000222C87BEDC0] missing picture in access unit with size 9101
[h264 @ 00000222C87C1AC0] SEI type 5 size 352 truncated at 159

That also continues until the video ends but it does manage to decode thousands of frames

Is there a way to solve this issue, or to know in advance that h264_cuvid wouldn’t work?

I won’t be possible to upload the video to the forum, since it belongs to our customer. I may be able to send it in a direct message to NVidia personal.

langdalepl · May 29, 2019, 4:51pm

You need to pass both arguments to actually do hardware accelerated decoding. Your second command line is actually just doing normal software decoding.

ffmpeg -hwaccel cuda -c:v h264_cuvid -i video.mkv

You should also consider using the nvdec hwaccel as this gives you acceleration while still using the ffmpeg parsers, which are more capable than the cuvid provided parsers.

ffmpeg -hwaccel nvdec -i video.mkv

And if you are doing a full hardware transcode with nvenc, you need

ffmpeg -hwaccel nvdec -hwaccel_output_format cuda -i video.mkv c:v h264_nvenc <etc>

tamirg · May 30, 2019, 11:49am

@langdalepl, Actually my second command line does perform hardware decoding.
I think you meant that “-hwaccel” is missing for hardware accelerated transcoding, which is not the problem I am trying to address here.

mandar_godse · June 7, 2019, 7:43am

Hi Tamir,
Thanks for providing the clip. Both “-c:v h264_cuvid” and “-hwaccel cuda” use parser code from FFmpeg.
But, they use different paths within FFmpeg. Path used by ‘-hwaccel cuda’ seems more
robust than the one used by ‘-c:v h264_cuvid’. Hence it succeeds to extract an
elementary bitstream and decode successfully, whereas later command-line fails.
You need to file a bug in FFmpeg to get this fixed for failing case.

langdalepl · July 1, 2019, 3:43pm

Actually not true. The cuvid suffixed decoders use the nvidia parser, which has known limitations vs the ffmpeg parser. That’s why the h264_cuvid example fails and the other does not. Use nvdec if you want accelerated decoding with the ffmpeg parser. There is no bug to report to ffmpeg here.