Dual GPU system in Ubuntu 16.04

Hi,

My desktop has:

  • 1 Intel integrated graphics card
  • 1 Tesla K40c
  • 1 Geforce 1060 6G

Everything running on Ubuntu Gnome 16.04. I would like use the Intel card to display and use Tesla and Geforce only for CUDA applications.
I followed many links, but I couldn’t come with a solution. My first issue is that even if I install my drivers via apt, or .deb or with the CUDA toolkit (from a clean OS install), “Additional Drivers” always use the nvidia driver only for one of the Nvidia cards (the other one keeps to Nouveau, even if a blacklisted it).
Several links in devtalk (most replied by txbob) says that a single driver can support both of these cards, but that does not seem to work. Also, strictly following the CUDA install guide seems to install it only for one of my devices, never for both of them.

Anyone has a clue?

I doubt this level of rigor will be actually necessary, but here’s how I would approach it:

  1. Familiarize yourself with the linux install guide: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
  2. Remove both NVIDIA GPUs from the system.
  3. Reinstall a clean copy of Ubuntu 16.04. Do not upgrade to the latest kernel, in fact do not upgrade anything, at all. If the kernel version is 4.10 or earlier, proceed. If it is 4.13, or later, stop, don’t use these instructions.
  4. Get the GUI working the way you want it.
  5. Remove nouveau entirely from the system:

sudo apt-get purge xserver-xorg-video-nouveau nouveau

  1. reboot
  2. remove nouveau from the initrd image:

sudo update-initramfs -u

6a. It’s probably a good idea at this point to force your Linux install to use the intel integrated graphics. To do so, it may be necessary to create a xorg.conf file that calls out the integrated graphics specifically. The exact method to do this will vary by system and linux distro, but see continuing thread below. If things don’t work in step 9, it’s possible that this step 6a was not done correctly.

  1. power down
  2. install the NVIDIA GPUs
  3. power up. verify that the GUI is still working correctly through the intel graphics. If it is not, stop, do not follow these instructions.
  4. get the CUDA 9.1 local runfile installer from http://www.nvidia.com/getcuda Do not get the network installer. Do not get any .deb installer.
  5. run the runfile installer using sudo, and append the following switch to the command line:

–no-opengl-libs

  1. During runfile install, if prompted to install any kind of openGL libs, select no. If prompted to install any 32-bit compatibility libs, select no. If prompted to make any changes to xorg, or run nvidia-xconfig, select no.
  2. Perform the necessary post-install actions covered in the linux install guide. (update PATH and LD_LIBRARY_PATH)
  3. reboot

That should work.

If things are working, and you at some time later choose to update or upgrade anything, and you don’t know what you are doing, the results may be not what you want. If you allow Ubuntu to install automatic updates or upgrades, that is a recipe for disaster. If you want to have things like automatic kernel upgrades, you’ll need to familiarize yourself a lot more with how the NVIDIA driver installation works, as well as related technologies like DKMS. Until I had things working, I would consider turning off any sort of automatic install or updating by Ubuntu, of anything, in step 2.

Hi,

Thanks for the quick reply. I managed to make it work with your help.

However I had to do some workarounds, mainly:

Regards,
Jean

Hi,

Just in case anyone stumbles upon this issue:
Once you have installed CUDA you can prevent kernel updates (that will probably break CUDA) using:
sudo apt-mark hold linux-image-generic linux-headers-generic
just change linux-image-generic and linux-headers-generic for the correct names.

Thanks, that is a good point. I was trying to make my instructions somewhat generic/easy/predictable, but there is a hole in them. Older linux distros generally created a xorg.conf and used that to define the display configuration. Once that is established, adding new graphics cards usually wouldn’t change things in my experience. Newer linux distros have various mechanisms to automatically run (and decide which display to use) even without a xorg.conf. This can obviously be affected/disturbed if you add new GPUs. The linux install may choose to do things different. Adding a proper xorg.conf will usually “pin this down”.

So for the benefit of future readers, I’ve added a step 6a to my instructions; they may be less likely to fall into this “hole”, even if they don’t read the whole thread.

Hi Robert,

I have been using this system for 1 year now. However I need openGL support now. There is a point in installing the drivers with no opengl support (step 11)? If so, why? Also, can I enable openGL support after the installation or should I reinstall the drivers to get that?

Thank you in advance,

The installation of the drivers with no opengl support is so that you will not disturb the X graphics stack associated with the intel integrated graphics. This is mentioned in the linux install guide if you read it carefully.

If you want OpenGL support, you should be prepared to switch your display to the NVIDIA GPU and stop using the intel integrated graphics.