Tensorflow libcuda and Kernel version differ (CUDA_ERROR_SYSTEM_DRIVER_MISMATCH)

Hello I just started my first “big” project. Now i finished my model and wanted to start the training and i encountered some issues. First if I run an python file which includes Tensorflow i get the following error message and I cant find a way to solve it. Here are the outputs I get.

2022-07-28 13:32:43.097099: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_SYSTEM_DRIVER_MISMATCH: system has unsupported display driver / cuda driver combination
2022-07-28 13:32:43.097126: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: dennis-MS-7C91
2022-07-28 13:32:43.097132: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: dennis-MS-7C91
2022-07-28 13:32:43.097214: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 510.73.5
2022-07-28 13:32:43.097228: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 515.48.7
2022-07-28 13:32:43.097232: E tensorflow/stream_executor/cuda/cuda_diagnostics.cc:313] kernel version 515.48.7 does not match DSO version 510.73.5 -- cannot find working devices in this configuration
2022-07-28 13:32:43.097438: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

Im thankful for every helpful approach for my Problem thanks :)

Looks like you installed full “cuda” over an already installed driver instead of just “cuda-toolkit”. Please uninstall all cuda/nvidia packages, the cleanly install the driver and afterwards only cuda-toolkit.

1 Like