CUDA 12.4 Compatibility with RTX 5070 Ti (Open Kernel Driver 570+)

Accelerated Computing CUDA CUDA Setup and Installation
,
1

Hello everyone.
I’m working with a NVIDIA RTX 5070 Ti graphic card, which as far as I can tell, is only supported by open-kernel drivers version 570 and above.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.133.07             Driver Version: 570.133.07     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5070 Ti     Off |   00000000:01:00.0  On |                  N/A |
|  0%   35C    P8              8W /  300W |      64MiB /  16303MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            2487      G   /usr/lib/xorg/Xorg                       53MiB |
+-----------------------------------------------------------------------------------------+

I need to deploy a computer vision project that is only compatible with cuda 12.4 or 12.6, however, as far as I can tell, all of the supported drivers for this graphic card are shipped with cuda 12.8+, and I’m not sure weather manually installing another cuda version along side 12.8 would be a good idea.

  1. Is CUDA 12.4 officially supported on Open Kernel drivers (>=570) with the RTX 5070 Ti?
    If not, is there a documented version matrix showing which user-space CUDA versions are compatible with which Open Kernel driver versions?
  2. What do you think is the best course of action I can take to tackle this problem?

Any official guidance, documentation, or driver/runtime version maps would be highly appreciated. I’m trying to understand if there’s any way to make CUDA 12.4-based binaries run on this system without rebuilding the full stack.

Thanks in advance!

2

There should be no problem, as driver versions are backwards compatible, within reason. See here for details.

3

Then how can I replace the default cuda 12.8 with 12.6? Should I go with the local (deb) installation?

4

If all you are doing is running Cuda binaries, you do not need any Cuda Toolkit installed. The driver you currently have will run binaries compiled for a minimum of Maxwell, on any Toolkit version from 6.5 onwards.

If you look at the table here under, “GPUs Supported”, you can see your 12.8 driver supports Compute Capability 5.0 (Maxwell)-12.0 (Blackwell). Support for Maxwell started with Cuda 6.5.

5

Reading the above I wonder if part of the OP’s difficulty is related to the way nvidia-smi tells him he has cuda version 12.8 even if he only installs the driver. This suggests he has to install another version of cuda, whereas as others have said, with care you can consider the driver version and Cuda version separately, to a degree.

If the project has been developed with 12.4 or 12.6 would including the ptx and relying on JIT compilation give him what he needs?

Out of curiosity would it perform approx as well as if developed with 12.9? (I know that’s probably a question that depends on too many questions about the nature of the project). Perhaps a more practical question would be: is it worth them giving it a go?

6

Yes, I perhaps unwisely, assumed that PTX would be included and certainly good practise to do so, for architectures beyond CC 9.0.