NVDec did not output the correct sequence of bitstream parsing and decoding

Gaming and Visualization Technologies Video Processing & Optical Flow
1

a-3.ts (3.9 MB)
The 147th frame (starting as 0) of the H264 bitstream in the file a-3.ts cannot be output correctly.

This frame PTS of 440287 and no B frame. It is a code stream output by an unknown encoder. My customer told me that using soft decoding can record all images, but using NVDec cannot record all images.

The Computer Info：
System
OsName: Microsoft Windows 11
GPU
NVIDIA RTX A4000
Diver: 31.0.15.2667
Date: 2022/11/2
DirectX : 12 (FL 12.1)
CPU
Intel(R) Core™ i7-7820X CPU @ 3.60GHz

I use cuvidParseVideoData and cuvidDecodePicture to decode video sequences. Print frame information when decoding output:

int HandlePictureDisplay(CUVIDPARSERDISPINFO *pDispInfo)
{
	CUVIDPROCPARAMS videoProcessingParameters = {};
	videoProcessingParameters.progressive_frame = pDispInfo->progressive_frame;
	videoProcessingParameters.second_field = pDispInfo->repeat_first_field + 1;
	videoProcessingParameters.top_field_first = pDispInfo->top_field_first;
	videoProcessingParameters.unpaired_field = pDispInfo->repeat_first_field < 0;
	videoProcessingParameters.output_stream = m_cuvidStream;

	CUdeviceptr dpSrcFrame = 0;
	unsigned int nSrcPitch = 0;
	CUDAAPI_CALL_CHECK(cuCtxPushCurrent(m_cuContext));
	CUDAAPI_CALL_CHECK(cuvidMapVideoFrame(m_hDecoder, pDispInfo->picture_index, &dpSrcFrame, &nSrcPitch, &videoProcessingParameters));

	CUVIDGETDECODESTATUS DecodeStatus;
	memset(&DecodeStatus, 0, sizeof(DecodeStatus));
	CUDAAPI_CALL_CHECK(cuvidGetDecodeStatus(m_hDecoder, pDispInfo->picture_index, &DecodeStatus));

	printf("pts=%lld picture_index=%d status=%d \r\n", pDispInfo->timestamp, pDispInfo->picture_index, DecodeStatus.decodeStatus);
    CUDAAPI_CALL_CHECK(cuvidUnmapVideoFrame(m_hDecoder, dpSrcFrame));
	
	return 1;
}

440287 is not outputting correctly and has been delayed for a long time before outputting:

pts=433087 picture_index=3 status=9
pts=434887 picture_index=4 status=9
pts=436687 picture_index=0 status=2
pts=438487 picture_index=1 status=2

...

pts=528487 picture_index=1 status=2
pts=530287 picture_index=0 status=2
pts=440287 picture_index=2 status=2

I also tried using CUVID to decode a-3.ts (ffmpeg -c:v h264_cuvid -i C:\a-3.ts -f mpegts C:\a-3-nvdec.ts) in FFmpeg, and the video can only output 468 frames; Try using NVDEC(ffmpeg -hwaccel nvdec -i C:\a-3.ts -f mpegts C:\a-3-nvdec.ts) and soft decoding to output a complete 471 frames. The main difference between them is that NVDEC does not use cuvidParseVideoData.

I think this is probably a problem with the parsing of the Video Codec SDK bitstream, or there are some fields on the code stream that are incompatible with the SDK.

2

h264_cuvid is to be deprecated anyway.

3

Thank’s for your reply @val.zapod.vz.

You means cuvidParseVideoData is deprecated also?

I want express is cuvidParseVideoData did not output correctly video sequence. And I want to prove that point with ‘ffmpeg -c:v h264_cuvid’ vs ‘ffmpeg -hwaccel nvdec’.

4

The stream is corrupt:

[h264 @ 0000012a4cb3ee00] concealing 1984 DC, 1984 AC, 1984 MV errors in P frames speed=4.37x
[h264 @ 0000012a4cb3f4c0] concealing 7744 DC, 7744 AC, 7744 MV errors in P frame
Downloads\a-3.ts: corrupt decoded frame in stream 0
Last message repeated 1 times
[h264 @ 0000012a4cb3f4c0] error while decoding MB 70 61, bytestream -6
[h264 @ 0000012a4cb3f4c0] concealing 819 DC, 819 AC, 819 MV errors in P frame
Downloads\a-3.ts: corrupt decoded frame in stream 0
[h264 @ 0000012a4caf5340] concealing 1515 DC, 1515 AC, 1515 MV errors in P frames speed=4.23x
Downloads\a-3.ts: corrupt decoded frame in stream 0
[h264 @ 0000012a4cb3f4c0] concealing 210 DC, 210 AC, 210 MV errors in P frame
[h264 @ 0000012a4cb43ec0] concealing 2643 DC, 2643 AC, 2643 MV errors in P frame
Downloads\a-3.ts: corrupt decoded frame in stream 0
Last message repeated 1 times
MD5=9f6dc190bd4456ab7a9d1e5738d8d9a0 0kB time=00:00:08.26 bitrate= 0.0kbits/s speed=4.09x
frame= 471 fps=198 q=-0.0 Lsize= 0kB time=00:00:09.44 bitrate= 0.0kbits/s speed=3.97x

5

Also you are mistaken, both h264_cuvid and -hwaccel cuda produce same size output: ffmpeg.exe -hwaccel cuda -i cacas.h264 cacsaeds1.yuv

and ffmpeg.exe -c:v h264_cuvid -i cacas.h264 cacsaeds.yuv

1 464 998 400 bytes.

But yeah, there is some md5 difference that there should not be. Using framemd5 we can see that the difference happens in all frames between 147 and 198 (that is counting from 0) and nowhere else.

ffmpeg.exe -c:v h264_cuvid -i cacas.h264 -f framemd5 -

ffmpeg.exe -hwaccel cuda -i cacas.h264 -f framemd5 -

6

ffmpeg -c:v h264_cuvid -i a-3.ts -map 0:0 -f framemd5 - > a-3-h264-cuvid.log output: a-3-h264-cuvid.log (37.2 KB)
ffmpeg -hwaccel cuda -i a-3.ts -map 0:0 -f framemd5 - > a-3-h264-nvdec.log output: a-3-h264-nvdec.log (37.5 KB)

ffmpeg version 5.0-essentials_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers

Yes that the difference happens in all frames between 147 and 198, and 297、298、299. And lossed 147、148、149 in a-3-h264-cuvid.log.

Thanks for your reminder, I have noticed some errors in the bitstream. But the output sequence is incorrect in my procedure (not just loss frame), and I still have no clue.

Maybe bypassing cuvidParseVideoData is a better for our business. (Usually we cannot determine the source of the bitstream)

7

ffmpeg version 5.0-essentials_build

That is too old. Very.

And lossed 147、148、149 in a-3-h264-cuvid.log.

No, it does not. Update ffmpeg.

Maybe bypassing cuvidParseVideoData is a better for our business. (Usually we cannot determine the source of the bitstream)

Again, newer ffmpeg produces same amount of frames.

8

Did you update ffmpeg and verify it does produces all the frame even if md5 of those is different?

9

Ping?