I am new here and interested in setting up a Linux desktop (OS?) with RTX 5090 to run mainly deep learning training application using Python. Any help on where to get started is appreciated, specifically for GPU drivers.
Are you still looking for a linux desktop to run your RTX 5090? Or have you already found one?
For what little it’s worth, I just got my 5090 running reasonably well in Ubuntu 24.04.2 LTS . I’ve got the open drivers installed with cuda toolkit and container toolkit. Ollama is running better than it did in wsl2 :) Happy to share the steps I took if it helps :)
Thank you for your response.
I am working to configure a Python environment where both PyTorch and TensorFlow can utilize my GPU (RTX-5090). While the PyTorch installation succeeds, TensorFlow consistently fails.
The error message indicates that TensorFlow lacks backend support for my GPU’s architecture. This seems to be an incompatibility issue with the required CUDA Toolkit version needed for my hardware.
Could you please advise on the correct installation procedure or the specific, compatible versions of TensorFlow, CUDA, and cuDNN I should be using? Not sure if this is what you are doing. Thanks.
Heya Andy :)
Sadly, I don’t know much of this landscape here yet, so please forgive my ignorance :) I’m open to suggestions and guidance :)
I’m working on something somewhat related: trying to get pytorch to compile from source for use with vllm compiled from source. Dependencies, dependencies… :)
Just a thought, the phrase “lacks backend support” makes me think of this. :
I’m running Ubuntu 24.04.2 currently. I’m working on an HP Omen 45L with an RTX 5090.
I’m in the process of verifying my cuda-toolkit-12-9, cudnn9-cuda-12, and nvidia-container-toolkit installs.
Last night, I compiled nearly all of the cuda-toolkit samples found here:
The only exceptions are the samples used for jetson and drive. For my process, I’m calling this good for the moment. I’m going to be working on the cudnn9-cuda-12 verification tonight. Hopefully I’ll make some progress :)
Following are my notes so far (sorry for the mess). I hope maybe there’s something in them that might be helpful.
# nvidia keyring
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
dpkg -i cuda-keyring_1.1-1_all.deb
# open driver
apt update && apt install nvidia-open -y
update-pciids
reboot
# cuda-toolkit
# followed: https://docs.nvidia.com/cuda/cuda-installation-guide-linux
apt install cuda-toolkit-12-9 -y
# verifying the install: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/#verify-the-installation
# cuda-samples build guide: https://github.com/NVIDIA/cuda-samples/blob/master/README.md#building-cuda-samples
# completed successfully
git clone https://github.com/NVIDIA/cuda-samples.git
cd cuda-samples
mkdir build && cd $_
# ### the following 5 commands were motivated from the errors I saw in the cmake output
export PATH=/usr/local/cuda-12.9/bin${PATH:+:${PATH}}
export export CMAKE_CUDA_ARCHITECTURES="120"
apt install freeglut3-dev -y
apt install libopenmpi-dev openmpi-bin -y
apt install libfreeimage-dev -y
# cmake error:
# -- Could NOT find NVSCI (missing: NVSCIBUF_LIBRARY NVSCISYNC_LIBRARY NVSCIBUF_INCLUDE_DIR NVSCISYNC_INCLUDE_DIR)
# -- NvSCI not found - will not build sample 'cudaNvSci'
# ### ignoring this. Search indicates NvSCI is not meant for desktop Linux systems. It’s part of the NVIDIA Drive and Jetson SDKs and is not included in standard CUDA or driver packages for x86.
# ### back to following: https://github.com/NVIDIA/cuda-samples/blob/master/README.md#building-cuda-samples
cmake ..
make -j$(nproc)
# ### everything seems good to here. Keeping above as is.
# ### The following seem to install correctly but still need testing/verification
apt install nvidia-container-toolkit -y
apt install cudnn9-cuda-12 -y
For grins, here’s the output from the “deviceQuery” sample I compiled last night:
root@omen:~/test/cuda-samples/build/Samples/1_Utilities/deviceQuery# ./deviceQuery
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "NVIDIA GeForce RTX 5090"
CUDA Driver Version / Runtime Version 12.9 / 12.9
CUDA Capability Major/Minor version number: 12.0
Total amount of global memory: 32087 MBytes (33645789184 bytes)
(170) Multiprocessors, (128) CUDA Cores/MP: 21760 CUDA Cores
GPU Max Clock rate: 2407 MHz (2.41 GHz)
Memory Clock rate: 14001 Mhz
Memory Bus Width: 512-bit
L2 Cache Size: 100663296 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total shared memory per multiprocessor: 102400 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 1536
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Managed Memory: Yes
Device supports Compute Preemption: Yes
Supports Cooperative Kernel Launch: Yes
Supports MultiDevice Co-op Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 2 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 12.9, CUDA Runtime Version = 12.9, NumDevs = 1
Result = PASS
If I run into anything interesting with the cudnn9 verification, I’ll mention it here. Cheers and good luck :)
If you’ve got some steps or a guide you’re following, I’d be happy to give it a shot and share what I get :)
I’ve had a bit of success with the cudnn verification. Following are my notes (continued from my previous notes)
# after: apt install cudnn9-cuda-12 -y
# verifying cudnn9: https://docs.nvidia.com/deeplearning/cudnn/installation/latest/linux.html#verifying-the-install-on-linux
# I was unable to locate a cudnn9-cuda-12 git repo. I wanted to see if they had samples/tests.
# I found the frontend repo and it does have samples :)
# I want to clone and build using the cudnn FE; it leverages the backend!
# this repo seems to be python based
mkdir python3.12-cudnn-frontent && cd $_
python3.12 -m venv .
source ./bin/activate
git clone https://github.com/NVIDIA/cudnn-frontend.git
cd cudnn-frontend
pip install -r requirements.txt
mkdir build && cd $_
cmake ..
# didn't see any errors
make -j$(nproc)
# all built minus some skips
# check ./bin for executables
# I had these:
# (python3.12-cudnn-frontend) root@omen:~/test/python3.12-cudnn-frontend/cudnn-frontend/build/bin# ll
# total 41852
# drwxr-xr-x 2 root root 4096 Jul 30 22:29 ./
# drwxr-xr-x 8 root root 4096 Jul 30 22:27 ../
# -rwxr-xr-x 1 root root 6964112 Jul 30 22:29 legacy_samples*
# -rwxr-xr-x 1 root root 24860472 Jul 30 22:29 samples*
# -rwxr-xr-x 1 root root 11014672 Jul 30 22:29 tests*
I ran the apps. They appear to be benchmarks for functionality/support. I got some fails but mostly successes :) Here is the summary output of running the “samples” executable:
===============================================================================
test cases: 84 | 71 passed | 6 failed | 7 skipped
assertions: 9460 | 9454 passed | 6 failed
Can provide more detailed output if you’re interested.