CUDA memory unavailable despite clearing cache and garbage

Accelerated Computing CUDA CUDA Programming and Performance
1

I am using AirSim with an ESPNet segmentation inference model on a GeForce RTX 2060 SUPER. It was working until CUDA started showing the OOM error. Despite nvitop and nvidia-smi showing free GPU memory, torch.cuda.mem_get_info() is showing that atleast 70% of my memory is filled (even going above 90% in sometimes).
I tried torch.cuda.empty_cache and gc.collect() but they aren’t helping me.

What can be the reason torch.cuda.mem_get_info() is showing filled memory? Please suggest some solutions to free up GPU memory.

image
image1079×1162 123 KB

2

my general suggestion for people asking pytorch questions here is that you may get better help by asking on a pytorch forum. This isn’t a pytorch forum. The control of CUDA memory in pytorch is handled by pytorch.

3

Sure, Robert, I’ve put it up on the PyTorch forum.
Here it is in case someone stumbles here: https://discuss.pytorch.org/t/cuda-memory-unavailable-despite-clearing-cache-and-garbage/186742