How do I set one GPU for display and the other two GPUs for CUDA computing?

Hi, I’ve installed three GPUs on a machine running Ubuntu 16.04; and I’ve installed NVIDIA proprietary driver 375.39.

So now I have GPU 0 (TitanX); GPU 1 (GTX1050); GPU 2 (TitanX). By default, it select GPU 0 for display. How can I set GPU 1 for display instead? I want to use GPU 0 and GPU 2 for CUDA computing.

I think you have to set some options in /etc/X11/xorg.conf help you? Hopes the links is appropriate.

http://us.download.nvidia.com/XFree86/Linux-x86/381.09/README/editxconfig.html
http://us.download.nvidia.com/XFree86/Linux-x86/381.09/README/xconfigoptions.html
do a find (Option “MultiGPU” “string”)
do a find (Option “SLI” “string”)

Run lspci and find out your GPU PCI slot.

Then create /etc/X11/xorg.conf.d/nvidia.conf with this content:

Section "Device"
        Identifier      "Videocard0"
        BusID           "PCI:1:0:0" # This must be what your lspci command gave you
        Driver          "nvidia"
        VendorName      "NVIDIA"
EndSection

Then connect your monitor to this GPU and reboot your PC or press Ctrl + Alt + Backspace (not enabled on all distros).

Thank you, both sunbearc22 and birdie for providing help! birdie’s method works for me, except that for Linux 16.04, I should create the file in another directory: /usr/share/X11/xorg.conf.d/nvidia.conf

You can use either. /usr/share/X11/xorg.conf.d is for system software to install pieces of configuration data, and /etc/X11/xorg.conf and /etc/X11/xorg.conf.d/ are for the system administrator’s configuration files.

If you put your configuration files in /usr/share/X11/xorg.conf.d, you run the risk of them being overwritten by the package manager. Although the risk of that is probably small. :)

There’s a list of places the server looks for configuration files in the xorg.conf(5) man page.

Thank you Moderator! I see what you mean. However, Ubuntu 16.04 don’t have xorg.conf nor xorg.conf.d/ under the directory /etc/X11. Shall I mkdir /etc/X11/xorg.conf.d, and create the file /etc/X11/xorg.conf.d/nvidia.conf ?

You can just create /etc/X11/xorg.conf or mkdir /etc/X11/xorg.conf.d and put something inside of there. It doesn’t really matter either way.

I have the same issue as the original poster, but the solution given here does not work for me. I am running Linux Mint 18.3 which is based on Ubuntu 16.04. My Nvidia driver version is 384.130. I have two GTX1080’s (GPU0,GPU1) and one GT620 (GPU2) installed in an Asus X299 motherboard. I want to use the GT620 for display and the GTX1080’s for CUDA computing. I tried adding the BusID line for the GT620 (in decimal and hex) to both /etc/X11/xorg.conf and /usr/share/X11/xorg.conf.d/nvidia.conf. BIOS ignores the change and Linux Mint will not start at all. Nothing ever comes out of GPU2. Has something changed in the past year and a half that would require a different solution?
nvidia-bug-report.log.gz (213 KB)

This is still the correct way to make use of a specific gpu.
Please run nvidia-bug-report.sh as root and attach the resulting .gz file to your post. Hovering the mouse over an existing post of yours will reveal a paperclip icon.
https://devtalk.nvidia.com/default/topic/1043347/announcements/attaching-files-to-forum-topics-posts/

Using this xorg.conf should work in your case

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0"
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
EndSection

Section "Files"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Unknown"
    HorizSync       28.0 - 33.0
    VertRefresh     43.0 - 72.0
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusID          "PCI:179:0:0"
    Option         "AllowEmptyInitialConfiguration"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Thank you for the quick response. I tried your xorg.conf and got the same behavior - hang during startup, GPU2 silent, everything coming out GPU0. Is there anything else I can try?

Then you would have to create an nvidia-bug-report.log while it is hanging to see what is going on. If the console is inaccessible, use ssh.

Is there a way that I can use my 2nd GPU for display while having coolbits enabled on my 1st GPU for fan control? In order to enable manual fan control I need to explicitly enable coolbits on X11/xorg.conf for my 1st GPU but doing that will make Linux use it for display instead of my 2nd GPU.

I have two graphics cards gtx 1070 and gt 710 I have tried the solution suggested by user @birdie to no avail. I am running linux min 20. I can book into mint using the gt710, but it goes into “fallback mode”. Does the solution posted here still work? For more details please see my post:

https://unix.stackexchange.com/questions/640016/how-do-i-use-my-secondary-gpu-for-display-output-and-primary-only-for-computatio

Thank you for any help. I bought the GT 710 so that I can continue to use my computer while the gtx 1070 is training models using CUDA.

Yes, still should work.

Reading your stackexchange post… you used the same PCI ID for both devices. That cannot work.

Thank you for the prompt reply. In my Stack post I am detailing two separate attempts. From what I understand the Bus ID of my GT 710 is 5:0:0. Could you please explain what I have done incorrectly?

Thank you

That’s really not obvious in your post. For sure you cannot use the same PCI ID in two different device definitions.
Please post your complete xorg.conf here.
Also look for other .conf files in /etc/X11/xorg.conf.d, /etc/xorg.conf.d, /usr/etc/xorg.conf.d, /usr/lib/X11/xorg.conf.d, /etc/X11/xorg.conf.d-4 that might contain Screen, or nvidia configuration, which might interfere (some of those directories might not exist).

There are no other conf files in /etc/X11/ I had to make xorg.conf.d. I have checked in /usr/lib/X11/ as well and was not able to find any other xorg.conf files.

I found the solution. I had to switch MultiGPU to “on” in the “Screen” section of etc/X11/xorg.conf.

I had tried omitting this option before and it didnt work.

Just in case any one else has the same issue I will post my xorg.conf here:

# nvidia-settings: X configuration file generated by nvidia-settings
# nvidia-settings:  version 440.82

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "0"
EndSection

Section "Files"
    EndSection

Section "Module"
    Load           "dbe"
    Load           "extmod"
    Load           "type1"
    Load           "freetype"
    Load           "glx"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Philips PHL 325E1"
    HorizSync       114.0 - 114.0
    VertRefresh     48.0 - 75.0
    Option         "DPMS"
EndSection

#makes gtx 1070 work on display

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 1070"
EndSection

#added to make GT710 run my Display doesnt work
Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
BusID		    "PCI:5:0:0"
option		    "AllowEmptyInitialConfiguration"					
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device1"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "DFP-6"
    Option         "metamodes" "2560x1440_75 +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "on" #"Off" #CHANGE APPLIED HERE
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

EDIT: I forgot to say thank you for your help