Driver issue with GPU and HWA in Ubuntu - Stuck at low resolution

I recently upgraded my Lenovo P700 workstation (that already had a Quadro NVS315 in it along with two Intel Xeon E5-2650V3 processors and 64GB of DDR4 RAM) with a Tesla K80 accelerator card I had lying around in order to program CUDA-based CFD simulations on it. I am aware that this card is intended to be used with servers but I am running a front intake fan plus a custom back exhaust to make sure it is properly cooled ($ nvidia-smi -q -d temperature for the temperatures).
After turning the Above 4G Decoding enabled option on in the BIOS under “Advanced/PCI Subsystem Settings”, the card was instantly accepted by the system running the latest drivers for the NVS315 and I could see both cards when calling
$ ubuntu-drivers devices
or
$ lspci | grep -i nvidia
After that I installed the CUDA V11.0 toolkit following the official manual Installation Guide Linux :: CUDA Toolkit Documentation on a fresh installation of Ubuntu 20.04. I now can compile code for compute capability 3.7
$ nvcc -gencode=arch=compute_37,code=sm_37 -o file file.cu
and it runs on the compute card and performs fine when launching with
$ ./hello
But after the installation and restart my resolution on the NVS315 would be stuck at 640x480, and the option of turning the resolution up again to 1920x1080 in the “Display settings/Resolution” is gone and I am not able to set a custom resolution with xrandr either How to Set A Custom Screen Resolution in Ubuntu Desktop | UbuntuHandbook. If I open “Software & Updates/Additional Drivers” it shows me both the GPU and the hardware accelerator. While I am able to choose between the nVidia drivers 435, 440 and 450 (currently selected) as well as Nouveau for the GK210GL [Tesla K80], the driver versions 390, 340.108 and Nouveau are greyed out for the GF119 [NVS 315] and instead “Continue using a manually installed driver” is selected. I assume it is using the driver that came with CUDA that is included in “Software & Updates/Other Software” located in “file:///var/cuda-repo-ubuntu2004-11-0-local /”, the version 450 that the Tesla is using as well.
I have re-installed the operating system with letting Ubuntu decide which driver version to use with both cards inserted. In such a case it instantly installs the 440 drivers and I end up with a 640x480 resolution by default. Then I have re-installed the system without installing additional drivers. The system uses Nouveau then and the resolution of the NVS315 is fine but it will also use Nouveau for the K80, of course nvcc will still compile but I won’t be able to launch any CUDA code on it. E.g. this small CUDA example https://developer.nvidia.com/blog/easy-introduction-cuda-c-and-c/ will return 2.0 instead of 0.0.
I tested installations from runfile, deb-local as well as using V10.1 from console with $ sudo apt-get install nvidia-cuda-toolkit, on fresh re-installed operating systems, all resulting in the same outcome.
Therefore I assume the issue in my case is the driver compability between the two. Is there a way to make these two drivers co-exist such that the NVS315 runs the 390 or 340 one and the Tesla the 450 one? Can I somehow set the NVS315 to use Nouveau instead and the K80 to stick with the 450 version of the nVidia one? Is there an option in Ubuntu that might force the “custom driver” to allow me to set higher resolutions?
Thanks in advance for your help!

You’re right that the Quadro NVS 315 is a legacy device and requires the release 390 series of drivers. Unfortunately you cannot install two different versions of the NVIDIA driver at the same time.

I don’t know if there’s a way to get the Nouveau driver to run only on one GPU. You’ll have to consult your Linux distribution on that.

1 Like

Thanks a lot for your answer. Are there any modern low-power alternatives (with no extra PCI-E power connector) that I could get instead of the NVS 315 and that would work with the 450 driver and the K80?

For anybody interested, what I did now was force Grub2 to use a higher frame buffer resolution.
Open the grub-settings file with
$ sudo gedit /etc/default/grub
uncomment the line #GRUB_GFXMODE=640x480 and modify it to
GRUB_GFXMODE=1920x1080
and potentially add
GRUB_GFXPAYLOAD_LINUX=keep
Then run
$ sudo update-grub
$ sudo reboot
For me it would push up the resolution to full-hd but you could see sort of scan-lines when switching a window, but maybe it works for you. I then simply used the VGA resolution option instead as proposed on this Ubuntu bug report by commenting again the inserted lines above and uncommenting the line GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" and modifying it to
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash vga=792"
where the code 792 corresponds to a resolution of 1024×768. I did not have luck with higher resolution codes such as 1600x1200, they again would suffer from the same bug.
Run
$ sudo update-grub
and restart again and you should have at least a system with a 1024x768 resolution.

While the fix I gave above works, its usability is limited. I ended up buying a newer generation graphics card, a Nvidia Quadro P620, which works together with the Tesla K80 without compatibility issues.