We are attempting to train a network of knee MRI through Niftynet. It works quite well with CPU based training, although it takes a lot of time for each iteration. We have 2 GPUs available (2xGeForce RTX 2080) (0,1) with 8GB memory each.
We have the following CUDA version: nvcc: NVIDIA ® Cuda compiler driver
Copyright © 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:32_Central_Daylight_Time_2017
Cuda compilation tools, release 9.0, V9.0.176
Nevertheless, we are struggling with an error message. We encounter the following error message: “tensorflow/core/kernels/cuda_solvers.cc:803: cuBlas call error 13” - which as I understand is a general serialization error. It occurs at first while the shuffle buffer is filling.
Also: OP_REQUIRES failed at matrix_inverse_op.cc:191 : Internal: tensorflow/core/kernels/cuda_solvers.cc:803: cuBlas call failed status = 13
Has anyone else on the forum encountered a similar issue?