I’m fighting with an issue for a week - when I’m trying to load pytorch/tensorflow model the display suddenly turns off. I tried to enable terminal mode, to get more RAM, connected over ssh from my laptop and turned on memory monitoring. I found that a crash happens even when the occupied memory level is 2.5.
When the model was really big, I usually was getting Memory Error.
I have used different versions of:
pytorch - 1.0.0 … 1.3.0,
tensorflow - 1.12 … 2.0 (also tried to convert models to tensorRT)
CUDA - 10.0
I tried different models resnet50, resnet101
Don’t have an idea how to fix it, please help me