CUDA NOT WORKING

Accelerated Computing CUDA CUDA Setup and Installation
1

Im using a ppc64le pc with almalinux 8.5
This is my output of nvidia-smi :
±--------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.06 Driver Version: 545.23.06 CUDA Version: 12.3 |
|-----------------------------------------±---------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Tesla V100-SXM2-16GB Off | 00000004:04:00.0 Off | 0 |
| N/A 37C P0 51W / 300W | 0MiB / 16384MiB | 0% Default |
| | | N/A |
±----------------------------------------±---------------------±---------------------+
| 1 Tesla V100-SXM2-16GB Off | 00000004:05:00.0 Off | 0 |
| N/A 39C P0 53W / 300W | 0MiB / 16384MiB | 0% Default |
| | | N/A |
±----------------------------------------±---------------------±---------------------+
| 2 Tesla V100-SXM2-16GB Off | 00000035:03:00.0 Off | 0 |
| N/A 35C P0 54W / 300W | 0MiB / 16384MiB | 0% Default |
| | | N/A |
±----------------------------------------±---------------------±---------------------+
| 3 Tesla V100-SXM2-16GB Off | 00000035:04:00.0 Off | 0 |
| N/A 38C P0 51W / 300W | 0MiB / 16384MiB | 3% Default |
| | | N/A |
±----------------------------------------±---------------------±---------------------+

±--------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
±--------------------------------------------------------------------------------------+

this is my nvcc version :

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Sep__8_19:17:19_PDT_2023
Cuda compilation tools, release 12.3, V12.3.52
Build cuda_12.3.r12.3/compiler.33281558_0

I install driver and cuda with runfile, and I didn’t see anything wrong in /var/log/cuda-installer and nvidia-installer.

When i use cudaGetDeviceCount(&number) i get Initialation error while doing device query test i get :
cudaGetDeviceCount failed – CUDA driver and runtime may be mismatched
FORTRAN STOP

What I am doing wrong?

2

To me, this seems to be a pretty old driver. My system works on version 571.96.
I noticed in my first tests with CUDA that old drivers may prevent your software from actually utilizing the CUDA hardware. I suppose if you installed the driver included in the (also rather old) CUDA Toolkit 12.3 installer, this might be the problem.

Maybe the Driver Installation Guide will help.