TensorRt 6 - IExecutionContext->execute cause GPU memory leak

AI & Data Science Deep Learning (Training & Inference) TensorRT
#1

In sampleSSD project, which is provided in samples, I have noticed a GPU memory leak for about +10 mb for every call of execute.

My goal was to build an ANN and allow it to do multiple inferences as long as it is needed.
So I have edited the source code of sampleSSD.cpp to make such approach.

if (!sample.build())
{
    return gLogger.reportFail(sampleTest);
}

for (int i = 0; i < 100; ++i) // I've added the loop to do the multiple calls
{
    // original code
    if (!sample.infer()) // the nvinfer1::IExecutionContext::execute is called inside this function
    {
        return gLogger.reportFail(sampleTest);
    }
}

if (!sample.teardown())
{
    return gLogger.reportFail(sampleTest);
}

return gLogger.reportPass(sampleTest);

So after breaking down the problem I have noticed that the raising in GPU memory is coming from the command nvinfer1::IExecutionContext::execute.
Are there any suggestions how to avoid this problem or am I doing it wrong?

#2

https://devtalk.nvidia.com/default/topic/1065018/tensorrt/context-gt-setbindingdimensions-casing-gpu-memory-leak/

#3

Hi nur.tsey and yfjiaren,

I’ve asked the engineering team to look into both of these cases. I will let you know when I hear back.

Thanks,
NVIDIA Enterprise Support

#4

I have encountered the problem using:
cuda_10.0.130_411.31_win10
cudnn-10.0-windows10-x64-v7.6.3.30
TensorRT-6.0.1.5.Windows10.x86_64.cuda-10.0.cudnn7.6

I didn’t encounter the memory leak when using:
cuda_10.1.243_426.00_win10
cudnn-10.1-windows10-x64-v7.6.4.38
TensorRT-6.0.1.5.Windows10.x86_64.cuda-10.1.cudnn7.6

#5

I have installed the same version of cuda cudnn tensorrt on linux, but it has memory leak as berfore.

#6

@nilshinn, I have a task to test it on linux too, can you provide information on what version you have installed and you encountered the problem with? with cuda 10.1 or cuda 10.0?

#7

I used cuda 10.1,
cuda_10.1.243_418.87.00_linux.run
cudnn-10.1-linux-x64-v7.6.4.38.tgz
TensorRT-6.0.1.5.CentOS-7.6.x86_64-gnu.cuda-10.1.cudnn7.6.tar.gz
Thanks.

#8

Just got an answer, they caught this in a similar bug and have fixed it for the next release. Thanks for pointing this out.

Also related to: https://devtalk.nvidia.com/default/topic/1065018/tensorrt/context-gt-setbindingdimensions-casing-gpu-memory-leak/?offset=7

#9

When will the next version be released?
And what is the next release version?

#10