TensorRt 6 - IExecutionContext->execute cause GPU memory leak

In sampleSSD project, which is provided in samples, I have noticed a GPU memory leak for about +10 mb for every call of execute.

My goal was to build an ANN and allow it to do multiple inferences as long as it is needed.
So I have edited the source code of sampleSSD.cpp to make such approach.

if (!sample.build())
    return gLogger.reportFail(sampleTest);

for (int i = 0; i < 100; ++i) // I've added the loop to do the multiple calls
    // original code
    if (!sample.infer()) // the nvinfer1::IExecutionContext::execute is called inside this function
        return gLogger.reportFail(sampleTest);

if (!sample.teardown())
    return gLogger.reportFail(sampleTest);

return gLogger.reportPass(sampleTest);

So after breaking down the problem I have noticed that the raising in GPU memory is coming from the command nvinfer1::IExecutionContext::execute.
Are there any suggestions how to avoid this problem or am I doing it wrong?


Hi nur.tsey and yfjiaren,

I’ve asked the engineering team to look into both of these cases. I will let you know when I hear back.

NVIDIA Enterprise Support

I have encountered the problem using:

I didn’t encounter the memory leak when using:

I have installed the same version of cuda cudnn tensorrt on linux, but it has memory leak as berfore.

@nilshinn, I have a task to test it on linux too, can you provide information on what version you have installed and you encountered the problem with? with cuda 10.1 or cuda 10.0?

I used cuda 10.1,

Just got an answer, they caught this in a similar bug and have fixed it for the next release. Thanks for pointing this out.

Also related to: https://devtalk.nvidia.com/default/topic/1065018/tensorrt/context-gt-setbindingdimensions-casing-gpu-memory-leak/?offset=7

When will the next version be released?
And what is the next release version?