Dual GPU problem with multiple displays in GNU/Linux

Graphics / Linux Linux
1

Hello Everyone,

I would like to ask for some help with my dual GPU setup with two identical GeForce® GTX 1650 WINDFORCE OC 4G.
I want to use them to display some static websites in six (3+3) 4k capable (Samsung) TV.
So I’m not interested in optimal performance or anything, I just want the cards to work with the 6 HDMI output.)

My specs in detail:

inxi -Fxz

	System:    Host: tv-mgmt Kernel: 4.15.0-54-generic x86_64 bits: 64 compiler: gcc v: 7.4.0 Console: tty 0 
		   Distro: Linux Mint 19.2 Tina base: Ubuntu 18.04 bionic 
	Machine:   Type: Desktop Mobo: ASUSTeK model: P7P55D v: Rev 1.xx serial: <filter> BIOS: American Megatrends v: 2101 
		   date: 10/20/2011 
	CPU:       Topology: Quad Core model: Intel Core i5 750 bits: 64 type: MCP arch: Nehalem rev: 5 L2 cache: 8192 KiB 
		   flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 21399 
		   Speed: 1204 MHz min/max: 1200/2668 MHz Core speeds (MHz): 1: 1204 2: 1204 3: 1204 4: 1204 
	Graphics:  Device-1: NVIDIA vendor: Gigabyte driver: nvidia v: 430.26 bus ID: 01:00.0 
		   Device-2: NVIDIA vendor: Gigabyte driver: nvidia v: 430.26 bus ID: 06:00.0 
		   Display: server: X.org 1.19.6 driver: none FAILED: nvidia tty: 120x25 
		   Message: Advanced graphics data unavailable in console for root. 
	Audio:     Device-1: Intel 5 Series/3400 Series High Definition Audio vendor: ASUSTeK driver: snd_hda_intel 
		   v: kernel bus ID: 00:1b.0 
		   Device-2: NVIDIA vendor: Gigabyte driver: snd_hda_intel v: kernel bus ID: 01:00.1 
		   Device-3: NVIDIA vendor: Gigabyte driver: snd_hda_intel v: kernel bus ID: 06:00.1 
		   Sound Server: ALSA v: k4.15.0-54-generic 
	Network:   Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet vendor: ASUSTeK M4A785/P7P55 
		   driver: r8169 v: 2.3LK-NAPI port: b800 bus ID: 02:00.0 
		   IF: enp2s0 state: up speed: 100 Mbps duplex: full mac: <filter> 
	Drives:    Local Storage: total: 111.79 GiB used: 6.40 GiB (5.7%) 
		   ID-1: /dev/sda vendor: OCZ model: VERTEX3 size: 111.79 GiB temp: 30 C 
	Partition: ID-1: / size: 108.56 GiB used: 6.40 GiB (5.9%) fs: ext4 dev: /dev/dm-0 
		   ID-2: swap-1 size: 976.0 MiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-1 
	Sensors:   System Temperatures: cpu: 35.0 C mobo: N/A 
		   Fan Speeds (RPM): N/A 
	Info:      Processes: 202 Uptime: 21h 19m Memory: 7.79 GiB used: 556.9 MiB (7.0%) Init: systemd runlevel: 5 
		   Compilers: gcc: 7.4.0 Shell: bash v: 4.4.20 inxi: 3.0.32

Both VGA working flawlessly by themselves, from any of the PCI-E slot.
Both can recognize 3 HDMI connected TV (4k res) and a DP connected DELL (1920x1080) monitor.
I could even make a VNC connection and do some remote work!

The main problem:
There is no scenario where all of the connected displays (6 TV + 1 monitor) are recognized at the same time.

sudo -H nvidia-settings # only one of the GPU-s are in the list

For the sake of simplicity I took the whole PC and connected 2 FHD (LG) monitor

  • instead of the 7 big screen - to test the multi GPU displaying. Sadly, without success.

I tried many distributions (Linux Mint 19.2, Manjaro 18.1.0, Ubuntu 19.04)
with different kernels (4.15, 4.19, 5.0) and desktop environments
(XFCE 4.12-, 4.14, Gnome 3.32, so lightdm and gdm were also tested).

More info:

lspci | grep -i nvidia

	01:00.0 VGA compatible controller: NVIDIA Corporation Device 1f82 (rev a1)
	01:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)
	06:00.0 VGA compatible controller: NVIDIA Corporation Device 1f82 (rev a1)
	06:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)

nvidia-smi -L

	GPU 0: GeForce GTX 1650 (UUID: GPU-...4ab3)

sudo dmesg |grep NVRM

	[   13.933222] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  430.26  Tue Jun  4 17:40:52 CDT 2019
	[   15.741299] NVRM: GPU at PCI:0000:06:00: GPU-20c4e4a3-d78c-3582-bb69-07264cbe7ddf
	[   15.741302] NVRM: GPU Board Serial Number: 
	[   15.741304] NVRM: Xid (PCI:0000:06:00): 61, 0cb5(2d50) 00000000 00000000
	[   68.666607] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[   68.666777] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
	[   74.318375] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[   74.318494] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
	[  681.394188] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[  681.394306] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
	[75788.064517] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[75788.064615] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1

nvidia-xconfig --query-gpu-info

	WARNING: Unable to use the nvidia-cfg library to query NVIDIA hardware.
	ERROR: Unable to query GPU information

nvidia-xconfig --enable-all-gpus --separate-x-screens

	Using X configuration file: "/etc/X11/xorg.conf".
	WARNING: Unable to use the nvidia-cfg library to query NVIDIA hardware.
	ERROR: Unable to determine number of GPUs in system; cannot honor '--enable-all-gpus' option.
	Backed up file '/etc/X11/xorg.conf' as '/etc/X11/xorg.conf.nvidia-xconfig-original'
	Backed up file '/etc/X11/xorg.conf' as '/etc/X11/xorg.conf.backup'
	New X configuration file written to '/etc/X11/xorg.conf'

xrandr
	Screen 0: minimum 8 x 8, current 1920 x 1080, maximum 32767 x 32767
	DP-0 disconnected (normal left inverted right x axis y axis)
	DP-1 disconnected (normal left inverted right x axis y axis)
	HDMI-0 disconnected (normal left inverted right x axis y axis)
	HDMI-1 disconnected (normal left inverted right x axis y axis)
	HDMI-2 connected 1920x1080+0+0 (normal left inverted right x axis y axis) 477mm x 268mm
	   1920x1080     60.00*+  59.94    50.00  
	   1680x1050     59.95  
	   1280x1024     75.02    60.02  
	   1280x720      60.00    59.94    50.00  
	   1152x864      75.00  
	   1024x768      75.03    60.00  
	   800x600       75.00    60.32  
	   720x576       50.00  
	   720x480       59.94  
	   640x480       75.00    59.94    59.93

Test results:

  1. With proprietary driver installed: (by default: 430.26, from ppa:graphics-drivers/ppa repo: 435.21, 390.77) the second monitor is fully blank.

    I tried…

    … to delete displays.xml:

    mv ~/.config/xfce4/xfconf/xfce-perchannel-xml/displays.xml ~/.displays.xml.bak

    …to modify (the appropriate) xorg.conf (in /etc/X11/ or xorg.conf.d or mhwd.d):

    Option    "Xinerama" "1"
Option    "MultiGPU" "On"
Option    "BaseMosaic" "On"

    …to blacklist nouveau:

    bash -c "echo blacklist nouveau > /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
bash -c "echo options nouveau modeset=0 >> /etc/modprobe.d/blacklist-nvidia-nouveau.conf"
rm -f /boot/initrd*
update-initramfs -c -k all
update-grub
  2. Without proprietary driver: same (empty second display). Interestingly with an auto-configured xorg.conf (X --configure provided me a basic config) it kinda' works: the second monitor has a desktop, but the resolution on both monitor are 640x480 and after opening the Display settings: it freezes. I could not even change to other virtual console to open another session or something. As for the resolution: the usual gtf; xrandr --newmode; xrandr --addmode commands dropped some errors, so I gave up.

After all this, I figured, why not try to merge
the Nvidia generated xorg.conf with the kinda’ working one, like:

Section "ServerLayout"
	    Identifier     "Layout0"
	    Screen      0  "Screen0" 0 0
	    Screen      1  "Screen1" LeftOf "Screen0"
	    InputDevice    "Keyboard0" "CoreKeyboard"
	    InputDevice    "Mouse0" "CorePointer"
	    Option         "Xinerama" "0"
	    Option "BlankTime" "0"
	    Option "StandbyTime" "0"
	    Option "SuspendTime" "0"
	    Option "OffTime" "0"
	EndSection

	Section "Files"
	EndSection

	Section "InputDevice"
	    # generated from default
	    Identifier     "Mouse0"
	    Driver         "mouse"
	    Option         "Protocol" "auto"
	    Option         "Device" "/dev/psaux"
	    Option         "Emulate3Buttons" "no"
	    Option         "ZAxisMapping" "4 5"
	EndSection

	Section "InputDevice"
	    # generated from default
	    Identifier     "Keyboard0"
	    Driver         "kbd"
	EndSection

	Section "Monitor"
	    # HorizSync source: edid, VertRefresh source: edid
	    Identifier     "Monitor0"
	    VendorName     "Unknown"
	    ModelName      "LG Electronics E2260"
	    HorizSync       30.0 - 83.0
	    VertRefresh     56.0 - 75.0
	    Option         "DPMS"
	EndSection

	Section "Monitor"
	    Identifier     "Monitor1"
	    VendorName     "Unknown"
	    ModelName      "LG Electronics E2260"
	    HorizSync       30.0 - 83.0
	    VertRefresh     56.0 - 75.0
	    Option         "DPMS"
	EndSection

	Section "Device"
	    Identifier     "Device0"
	    Driver         "nvidia"
	    VendorName     "NVIDIA Corporation"
	    BoardName      "GeForce GTX 1650"
	    BusID          "PCI:1:0:0"
	    Screen         0
	EndSection

	Section "Device"
	    Identifier     "Device1"
	    Driver         "nvidia"
	    VendorName     "NVIDIA Corporation"
	    BoardName      "GeForce GTX 1650"
	    BusID          "PCI:6:0:0"
	    Screen         1
	EndSection

	Section "Screen"
	    Identifier     "Screen0"
	    Device         "Device0"
	    Monitor        "Monitor0"
	    DefaultDepth    24
	    Option         "Stereo" "0"
	    Option         "nvidiaXineramaInfoOrder" "DFP-4"
	    Option         "metamodes" "nvidia-auto-select +0+0"
	    Option         "SLI" "Off"
	    Option         "MultiGPU" "Off"
	    Option         "BaseMosaic" "Off"
	    SubSection     "Display"
		Depth       24
	    EndSubSection
	EndSection

	Section "Screen"
	    Identifier     "Screen1"
	    Device         "Device1"
	    Monitor        "Monitor1"
	    DefaultDepth    24
	    Option         "Stereo" "0"
	    Option         "nvidiaXineramaInfoOrder" "DFP-4"
	    Option         "metamodes" "nvidia-auto-select +0+0"
	    Option         "SLI" "Off"
	    Option         "MultiGPU" "Off"
	    Option         "BaseMosaic" "Off"
	    SubSection     "Display"
		Depth       24
	    EndSubSection
	EndSection

It did not help.

Am I missing something? Should I use SLI for this kind of thing, or my xorg.conf is to blame?

Thanks in advance for any help.

nvidia-bug-report.log.gz (1.06 MB)

2

That won’t work. BaseMosaic on consumer Geforce type cards is limited to 3 displays so it’s useless. Buy Quadros.

3

Thank you for the answer, but are you sure about this? I mean, It worked with 4 display at once as I’ve stated earlier.
( “…Both can recognize 3 HDMI connected TV (4k res) AND a DP connected DELL (1920x1080) monitor…”)

4

Concerning BaseMosaic, yes, I’m sure.
Your post is a rather long read and it seems you tried a lot of stuff, I didn’t follow all of it. At some times, you had your monitors working with different screens, so all would work but you couldn’t move windows between them.
Since you don’t have any or much performance requirements, you should be able to use Xinerama to achieve this. You will lose compositing so you can’t start gnome since this relies on it. I changed your xorg.conf to something that might work:

Section "ServerLayout"
    	Identifier     "Layout0"
    	Screen      0  "Screen0" 0 0
    	Screen      1  "Screen1" LeftOf "Screen0"
    	Option         "Xinerama" "1"
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "LG Electronics E2260"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 75.0
    Option         "DPMS"
EndSection

Section "Monitor"
    Identifier     "Monitor1"
    VendorName     "Unknown"
    ModelName      "LG Electronics E2260"
    HorizSync       30.0 - 83.0
    VertRefresh     56.0 - 75.0
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 1650"
    BusID          "PCI:1:0:0"
EndSection

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 1650"
    BusID          "PCI:6:0:0"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
EndSection

Section "Screen"
    Identifier     "Screen1"
    Device         "Device1"
    Monitor        "Monitor1"
EndSection

Section "Extensions"
    Option "Composite" "Disable"
EndSection
5

Thank you for the .conf, but unfortunately it didn’t help: only one display at a time. I tried it with proprietary- and open-source driver too.
( Is it possible that this file: ~/.config/xfce4/xfconf/xfce-perchannel-xml/displays.xml has something to do with it?
I mean, there was a bug not so long ago, when it conflicted with the nvidia driver and I had to remove it to boot to the desktop so to speak.
Ref. link: [url]https://wiki.archlinux.org/index.php/Xfce[/url] )

Out of curiosity, I installed a Windows 10 (yes I know, shame on me!)
with the latest NVIDIA driver (436.48-desktop-win10-64bit-international-dch-whql.exe)
and guess what? It sucked even more. Both connected monitor was blank when I tried it.

I was lost, then all of a sudden it just… worked. I don’t know how, but after the third boot,
the driver installed on Windows recognized my second VGA with the second Monitor.

I’m tempted to quote Linus from June 14, 2012…

6

Please create a new nvidia-bug-report.log with the xorg.conf in place.

7

Here you go, and thanks again.

Ps.: the Windows “solution” only worked for the first reboot, then again: no login screen, nothing.
As if the monitors are just shutting down. This bugs me the most, that what I’m asking is obviously not an impossibility.
nvidia-bug-report-2.log.gz (1.06 MB)

8

You’re running into

[  118.849828] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1184)
[  118.849985] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1

meaning one of your gpus is possibly faulty. Please try reseating it, test it in another system, rma if issues persist.

9

Yes, I have seen this before:

sudo dmesg |grep NVRM

	[   13.933222] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  430.26  Tue Jun  4 17:40:52 CDT 2019
	[   15.741299] NVRM: GPU at PCI:0000:06:00: GPU-20c4e4a3-d78c-3582-bb69-07264cbe7ddf
	[   15.741302] NVRM: GPU Board Serial Number: 
	[   15.741304] NVRM: Xid (PCI:0000:06:00): 61, 0cb5(2d50) 00000000 00000000
	[   68.666607] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[   68.666777] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
	[   74.318375] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[   74.318494] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
	[  681.394188] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[  681.394306] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1
	[75788.064517] NVRM: GPU 0000:06:00.0: RmInitAdapter failed! (0x24:0x65:1119)
	[75788.064615] NVRM: GPU 0000:06:00.0: rm_init_adapter failed, device minor number 1

but the hardware is NOT faulty. I tried them one-by-one, as I’ve said:
“Both VGA working flawlessly by themselves, from any of the PCIe slot.”

Re-tested now - just to be sure - and attached the logs.
( Of course I had to use PCIe slot 1 on the mobo for both cards now, because if there is no attached device on that slot,
the second won’t work either. As far as I know, this is totally normal, depends on the bus.
When I tried to connect 3 TV, both slot worked, but only one at a time, no matter how I connected the HDMI cables.)

Should I try the whole thing with a newer motherboard, hence newer BIOS (UEFI, whatever)?

nvidia-bug-reportvga1pcie1.log.gz (1.07 MB)
nvidia-bug-reportvga2pcie1.log.gz (1.07 MB)

10

Didn’t notice the bios is from 2011. Then it’s probably the board that can’t reliably drive two turing gpus which require higher pcie quality than previous generations.

11

I did realize, when both VGA were attached, the boot time (and the time that it takes the grub menu(!) to appear) increased significantly.
I will try this whole thing again with a B360HD3 board (BIOS date: 03/14/2019) and report back.

12

That was it! Once I tried another - newer - hardware:

inxi -Fxz
System:
  Host: tv-mgmt Kernel: 4.15.0-54-generic x86_64 bits: 64 compiler: gcc 
  v: 7.4.0 Desktop: Xfce 4.12.3 Distro: Linux Mint 19.2 Tina 
  base: Ubuntu 18.04 bionic 
Machine:
  Type: Desktop System: Gigabyte product: B360 HD3 v: N/A serial: <filter> 
  Mobo: Gigabyte model: B360HD3 v: x.x serial: <filter> 
  UEFI [Legacy]: American Megatrends v: F13 date: 03/14/2019 
CPU:
  Topology: 6-Core model: Intel Core i5-8400 bits: 64 type: MCP 
  arch: Kaby Lake rev: A L2 cache: 9216 KiB 
  flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 33696 
  Speed: 800 MHz min/max: 800/4000 MHz Core speeds (MHz): 1: 800 2: 800 
  3: 800 4: 800 5: 800 6: 800 
Graphics:
  Device-1: NVIDIA vendor: Gigabyte driver: nvidia v: 435.21 bus ID: 01:00.0 
  Device-2: NVIDIA vendor: Gigabyte driver: nvidia v: 435.21 bus ID: 02:00.0 
  Display: x11 server: X.Org 1.19.6 driver: nvidia tty: N/A 
  OpenGL: renderer: GeForce GTX 1650/PCIe/SSE2 v: 4.6.0 NVIDIA 435.21 
  direct render: Yes 
Audio:
  Device-1: Intel Cannon Lake PCH cAVS vendor: Gigabyte 
  driver: snd_hda_intel v: kernel bus ID: 00:1f.3 
  Device-2: NVIDIA vendor: Gigabyte driver: snd_hda_intel v: kernel 
  bus ID: 01:00.1 
  Device-3: NVIDIA vendor: Gigabyte driver: snd_hda_intel v: kernel 
  bus ID: 02:00.1 
  Sound Server: ALSA v: k4.15.0-54-generic 
Network:
  Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet 
  vendor: Gigabyte driver: r8169 v: 2.3LK-NAPI port: 3000 bus ID: 04:00.0 
  IF: enp4s0 state: up speed: 100 Mbps duplex: full mac: <filter> 
Drives:
  Local Storage: total: 312.51 GiB used: 7.29 GiB (2.3%) 
  ID-1: /dev/sda vendor: Western Digital model: WD3200AAKS-00B3A0 
  size: 298.09 GiB 
  ID-2: /dev/sdb type: USB vendor: Kingston model: DataTraveler 2.0 
  size: 14.42 GiB 
Partition:
  ID-1: / size: 291.46 GiB used: 7.23 GiB (2.5%) fs: ext4 dev: /dev/dm-0 
  ID-2: swap-1 size: 980.0 MiB used: 0 KiB (0.0%) fs: swap dev: /dev/dm-1 
Sensors:
  System Temperatures: cpu: 54.0 C mobo: 27.8 C gpu: nvidia temp: 46 C 
  Fan Speeds (RPM): N/A gpu: nvidia fan: 0% 
Info:
  Processes: 217 Uptime: 16h 52m Memory: 31.35 GiB used: 2.26 GiB (7.2%) 
  Init: systemd runlevel: 5 Compilers: gcc: 7.4.0 Shell: bash v: 4.4.20 
  inxi: 3.0.32

… it worked out of the box. (With Xinerama set up via the nvidia-settings GUI.)

Thank you very much for helping me out. :)

Screenshot_2019-10-16_09-16-59.png
Screenshot_2019-10-16_09-16-59.png1166×1024 197 KB