Cannot decode some H264 data on GPU (libavcodec / ffmpeg)

Hi. I am using GPU to decode raw H264 data. My implementation is based on FFmpeg/decode_video.c at master · FFmpeg/FFmpeg · GitHub . I found that codec h264_cuvid failed to decode on some data, while CPU implementation h264 / libx264 worked normally. It didn’t always fail through, maybe approx 50% of input data. I tried to use both libavcodec and ffmpeg application on two different servers and GPUs, and they seemed to perform similarly, failed to decode the same data, worked normally for the other. I also ran a GPU H265 Decoder, and it can work well all day.
So I think that maybe GPU H264 Codec is bugged, or there is some kind of H264 data format that GPU implementation cannot decode?
My environment setup:

  1. Server 1
  • GPU: RTX 3090
  • Driver: 470.161.03
  • Cuda 11.4
  • OS: Ubuntu 20.04
  • FFMPEG/libavcodec: 4.2.7 (installed using apt)
  1. Server 2
  • GPU: Tesla T4
  • Driver: 510.47.03
  • Cuda: 11.3
  • OS: Ubuntu 20.04
  • FFMPEG/libavcodec: 4.4.2 (installed from source, following Official Guide and Pytorch Guide

FFMPEG command:
CPU (worked):

  • ffmpeg -vcodec h264 -i 1000_raw_data.h264 output.mp4
  • ffplay -vcodec h264 1000_raw_data.h264

GPU (failed):

  • ffmpeg -vcodec h264_cuvid -i 1000_raw_data.h264 output.mp4
  • ffplay -vcodec h264_cuvid 1000_raw_data.h264

I will provide my sample data here One Drive
Any help appreciated.

Hi,
The FFMPEG H264 GPU decoding is implemented based on the NVIDIA Video SDK.
The SDK pass the input data through it own h264 parser to identify the h264 nal units and other h264 parameters. I tried to use the input you provided to the h264 decode sample of the SDK and got a lot of “CONCEALED ERRORS” the resulted video was not “green” as when using " * ffplay -vcodec h264_cuvid 1000_raw_data.h264" - I did get a few normal frames every few seconds but the video as a whole was not playble (lots of artifacts in the video)

When using the CPU decoding ( * ffplay -vcodec h264 1000_raw_data.h264") I noticed video is OK but there are a lot of “Invalid NAL Units” Errors - it is possible that the h264 Parser that is used by the NVIDIA SDK is not handling the Invalid NAL unit properly and try to decode them which lead to a lot of errors.

You should contact the NVIDIA SDK development team and send them the input file so they can fix thier parser.

As for the FFPLAY displaying green screen - I Know previous version of the SDK (i’m currently using version 11.1.5) caused green screen in certain scenarios but in the version i’m using it was fixed - maybe FFMPEG 4.4.2 is using the old version of the SDK - have you tried using FFMPEG version 5 ?

1 Like

Thank you very much for spending time to check my issue. I will try to contact the dev team. Can I use this form https://developer.nvidia.com/contact to contact them?. They redirect technical issues to the forum, but it seems like the developers are not very active here.

I never had reason to contact the developers so I can’t help you there.

I hope you get a response

קבל ‏Outlook עבור Android‏