Nvidia driver issues on Ubuntu 24.04 and Cuda 12.6

I’ve spent the past several days trying to get the Nvidia drivers installed properly on my new GPU training rig. It uses 2x RTX 3090s bridged with NVLink. While I’ve used Linux quite extensively for work and school in the past, this is my first time setting up a linux box completely from scratch.

nvcc --version gives me the proper cuda information, but ever since I installed cuda, nvidia-smi has stopped working, and the nvidia driver isn’t working properly (only 1 monitor working, and terrible resolution/latency). I’ve followed just about every guide I can find on the internet to try and install the proper driver, but nothing ever seems to change anything. I’ve even started completely fresh with a brand new copy of Ubuntu 24.04 a couple of times and installed things in different orders to see if that makes a difference, but I always end up in exactly in the same spot.

Has anyone experienced anything similar, or perhaps knows the right questions to ask me to figure out what the issue is?

The next step I’m going to try is downgrading to Ubuntu 20.04 to see if that’s any smoother.