I upgraded NVIDIA codec SDK from 11.1.5 to 12.1.14 (the latest I believe) recently and I think I found a “Memory hoarding” problem in SDK 12.1.14.
My application is an AVC/H.264 video player based on NVIDIA codec SDK and it has “loop play mode” feature.
When it reaches the end of a video clip, it seeks back to the first frame of the clip and continues playing infinitely.
After the upgrade, I’ve been having steady and limitless increase of memory size (RSS/RES: residential memory area) with my application in “loop play” mode.
Before the upgrade, the application’s memory size increases for a while, then hits a ceiling and stops increasing.
After having a hard time with debugging, I found the following facts.
- The increased memory area contains a huge array of SEI message data. In my case, they were repeated chunks of human-readable texts that describe encoding parameters, which are embedded at the top of AVC/H.264 stream data.
- The number of text chunks increase in proportion with “loop count”.
- With SDK 12.1.14, users can set a callback-function-pointer to pfnGetSEIMsg in set-up parameter structure CUVIDPARSERPARAMS given to cuvidCreateVideoParser().
- With SDK 11.1.5, fpnGetSEIMsg was not defined. It seems to be a new feature added in SDK 12.
- When a callback-function-pointer is set to pfnGetSEIMsg, the callback-function is called when the parser comes across SEI Messages or Metadata OBUs in a AVC/H.264 stream.
- The sample code in “Samples/NvCodec/NvDecoder/NvDecoder.cpp” specifies NULL for pfnGetSEIMsg if SEI messages are not necessary. Therefore, pfnGetSEIMsg is not a mandatory parameter and it can be left unspecified as NULL.
- When I specify a pointer to an empty function (which does nothing but returning 1) for pfnGetSEIMsg, the increase of memory size stops after hitting a ceiling, as it used to with SDK 11.1.5.
- When I set NULL to pfnGetSEIMsg, the size of memory increases without limit in “loop play mode”.
I suspect that the AVC/H.264 parser/decoder hoards SEI message data on heap memory without limit when it can’t call a callback-function (because pfnGetSEIMsg is NULL), let the function handle the messages, then free the memory area for the data on return from the function.
I think the parser should ignore all SEI messages and Metadata OBUs from the start when NULL is given to CUVIDPARSERPARAMS.pfnGetSEIMsg on cuvidCreateVideoParser().
The amount of hoarded memory is small (less than 1 KB per loop), but it can accumulate up to 100 MB when a short clip is kept loop-playing for several days (this is a typical use-case of my product).
This problem can be avoided by assigning a pointer to an empty function to pfnGetSEIMsg,
but fixing the problem will save future users from unexpected increase of memory size.
It would be nice if I could provide you a modified version of NVIDIA SDK sample code for easy reproduction, but I can’t install ffmpeg on my development environment for some reasons.
I believe you can easily reproduce the problem with AVC/H.264 MP4 files with some SEI messages embedded in the content or you can identify the cause of the problem if you have access to the source code of codec SDK 12.1.14.
I think this is a problem with AVC/H.264 parser and GPU has nothing to do with the problem,
but the following is my GPU information.
OS : Linux (Rocky 8)
Driver Version : 535.54.03
CUDA Version : 12.2
Product Name : Quadro RTX 4000 or RTX A4500
Thank you for reading this report.