I had an issue where a h264 video with idr only frames was decoding just fine, but a h264 file with idr, b and p resulted in the typical h264 decode error garbage. After spending too much time I found the solution somewhere I didn’t expect.
I expected that the callbacks that you set on the CUVIDPARSERPARAMS should return 0 on success and < 0 on failure like most C/C++ functions do. And tbh, I expected that the parser would actually ignore the return values or would use them and returning an error would cause a failure. But I couldn’t find any info on this in the header or documentation.
It turns out this is not really the case. When my sequence callback returns 0 (as in success when following the convention that is used in C/C++) the h264 with idr only frames works fine. But this somehow causes h264 videos with other nals then idr to be decoded incorrectly.
So the fix was to return 1 from my sequence callback instead of 0.
@nvidia could you document this somewhere in the nvcuvid.h file?