I’m trying to run a simple CNN Tensorflow model on my system Ubuntu 20.04 on laptop Razer 15 blade with an RTX 2060 Max-P with 6Gb GDDR6.
The model does not compile and I get the following error:
OP_REQUIRES failed at conv_ops_fused_impl.h:697 : Not found: No algorithm worked!
.
I have NVIDIA driver (460.32.3) installed by Ubuntu, with cuda compiler:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0
I also installed cudnn from cudnn-11.0-linux-x64-v8.0.5.39
.
Tensorflow installed is 2.4.1.
In my zshrc I have added:
export LD_LIBRARY_PATH=“$LD_LIBRARY_PATH:/usr/local/cuda/lib64”
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64
The only way to get things work is to add:
export TF_FORCE_GPU_ALLOW_GROWTH=‘true’
in my .zshrc, as suggested here.
This is really weird as I suppose that 6Gb of GDDR6 are enough for this CNN model.
I also tried to run ai-benchmark and I was not able to finish the tests while with the env variable set as shown above I get several warning regarding not having enough memory to allocate, but I’m able to end all tests.
If I use watch nvidia-smi
while running the notebook code from CNN model I noticed that with the env set the GDRR6 allocated is quite small while is near the limit of the available memory if the env variable is not set.
What’s the problem? Any suggestion to fix it?