Xorg configuration - two NVidia cards, xinerama breaks hardware acceleration, 390.25 current driver

Graphics / Linux Linux
1

Hey all. I’m having some trouble getting 4 monitors up and running on my system. I’ve got four 1920x1080 monitors, two each plugged in to a GTX 580 and GTX 570.

If I use Xinerama and a separate screen per monitor, the monitors will work, but hardware acceleration for Chrome and other various desktop components fails if it’s dragged outside of the originating desktop. Further, clicking and dragging windows with Xinerama turned on seems to move the window I am manipulating 3840 pixels to the right of where I’m holding the mouse.

If I don’t use Xinerama, the monitors attached to my secondary video card don’t have an X session associated with them. I can still move the mouse into those monitors, but I can’t drag any windows into those monitors.

When maximizing a window without xinerama enabled, it maximizes across the whole virtual desktop instead of stopping at the monitor boundaries, so I have to manually adjust with the edge of a window.

What I’d like: it would be nice if xinerama worked across all four monitors flawlessly, and I could do everything like I could in Windows.

What I’d settle for: two separate X servers running on two monitors, side-by-side each X server powered by its own video card.

You will find the bug script output attached.
nvidia-bug-report.log (572 KB)

2

First of all, you should remove/comment out the lines about twinview.
In case of xinerama, it depends on the nvidia driver finding both gpus ‘similar’ enough for having GLX on all displays, see:
[url]https://us.download.nvidia.com/XFree86/Linux-x86/340.98/README/xineramaglx.html[/url]
Enable xinerama and see if the mentioned error is showing up in /var/log/Xorg.1.log

In the second case, you already have a second screen on the second gpu. You can run applications or a window manager on that by prepending the right DISPLAY variable, like
DISPLAY=:1.1 openbox

3

Alright, I’ve done some further experimentation, and am reporting back. Thanks for your help - the twinview option didn’t appear to have any effect positive or negative. Is twinview still an acknowledged option? I seem to recall around 2008 that it was the “new hotness” until Xinerama was introduced.

I’d originally tried to get this monitor configuration working with a GTX 730 and a GTX 570, but those were reporting incompatibilities with each other. I’d assumed that it was due to a difference in processor architecture. I happened to have a GTX 580 sitting around, so I figured I’d try that out instead of the 730, and when I experienced issues with this configuration, I created the original post.

Using Gnome-flashback and Xinerama with two screens, the desktop seems to operate better than it had with the 730, and I can run GLX on both video cards, but there seems to be an issue with dragging windows around with two screens and 4 monitors: when I grabbed a window, the window would appear 3840 pixels to the right of my mouse cursor. This appears to only be a problem with Google Chrome. Everything else, xterm, steam games, whatever else, seemed to be able to be moved around without this problem.

I figured this was an issue with the screen size, so I reconfigured my xorg.conf with nvidia-settings to use one screen per monitor, instead of the original configuration of one screen per video card. This didn’t resolve many of the problems I’d been experiencing, and I remembered after looking at the Xorg.0.log that I should disable the compositing extension.

This seemed to do the trick with Metacity, but I wasn’t able to drag a chrome browser window outside of the screen it was originally created. So I tried out Compiz, which seemed to work a lot better! Sadly, Compiz requires compositing to be enabled, and seems to require a LOT of processing power to manage my desktop.

So I’m kind of stuck: I can use Metacity without compositing, which means I can only have chrome open on one screen, or I can use Compiz, which seems to still have some of the same window dragging issues I’d mentioned above (right one screen now, instead of two), and a massive loss of processing power to manage the window manager.

Any other suggestions? I’d like to use metacity since it seems like it uses a lot less processing power, but not being able to fully utilize the desktop for certain programs is unpalatable. Meanwhile, using Compiz seems like a good solution, but the usefulness of the desktop is limited due the amount of power required by Compiz. Further, it appears compiz requires compositing in order to operate properly, so I’m assuming that there’s an incompatibility with it and Xinerama.

You’ll find two bug reports attached: one with composite enabled, and one without. I have another GTX 580 which I am considering installing. I’m wondering if this is still an issue with a difference in cards. I suppose I could also run them in SLI. Not sure if this will be helpful for my situation though.

One thing I’d noticed about the whole “dragging windows around” situation: as I moved the chrome window around the desktops, it would appear to have a black screen for a moment, and the dragged window would appear to the right opf my curser by 3840 pixels for about 15 seconds. Then something happens, and everything seems to work properly. Is this some portion of Chrome being loaded into memory of the second video card?

Thanks for the help thus far!
nvidia-bug-report_with_composite.log (523 KB)
nvidia-bug-report_without_composite.log (776 KB)

4

As time flies by…you’re off by a decade;)
Xinerama was introduced in 1998, Twinview was announced in 2000, being the “new hotness” since Xinerama was kind of meh. Twinview was then superseded by randr 1.2, released in 2006, appearing first in xserver 1.3, 2007. So unless you’re running a 12 years old Xserver, Twinview is pointless and possibly introduces problems/incompatibilities.
Back to your issue, you’ve now pretty much measured out every downside of a Xinerama setup. First thing is compositing. When Xinerama is on, compositing is switched to software, done on CPU. That’s why compiz has such a great load. Besides, compositing+xinerama is unsupported in your nvidia setup since the driver is also poking glx into xinerama, so issues may appear.
Second thing is ‘Xinerama-awareness’ of the window manager, that’s why metacity doesn’t go well with it, it’s lacking it.
The only recent WMs with xinerama awareness and non-compositing would be xfce with effects turned off and fluxbox, I think. Maybe compiz with everything turned off works, too, when compositing enabled. YMMV.

Doesn’t sound good? Outlook is worse. It seems that Xserver 1.15, which you’re currently running, is the last which has xinerama on nvidia working. Recently, some people showed up in this forum whose xinerama setup broke when updating to 1.17,1.18 or 1.19. So with a newer graphics stack, you will probaly have to revert to PRIME, which will be hard to set up with two nvidias and the proprietary driver or revert to nouveau which has its own issues.

5

Wow. So this is it, huh?

Using nouveau is a non-option since it won’t work with steam (at least it wouldn’t last I tried).

So it sounds like Xinerama is a poor solution. What’s left? Buying NVS cards and using BaseMosaic? I feel like this isn’t something which should be that difficult to work out, but I didn’t realize how far the rabbit hole goes.

So what’s the good option? xrandr? Edit: Just realized that splitting DVI isn’t possible after looking further) finding a DVI splitter for dual link and removing a card? Just crying about it? This seems like an answer I should’ve recently stumbled upon after a week of trying to get this to work correctly…

6

BaseMosaic with non-Quadro cards is limited to 3 displays. So this won’t help with your setup.
So options are:

  • Try xinerama without compositing and xfce
  • Use two gtx580 with basemosaic and only 3 displays
  • Use one gtx580 and only 2 displays
  • Buy a new gfx card with 4 heads
  • Try to make prime work
7

Here’s a recipe to get prime working. I don’t know if that works with two nvidias using the proprietary driver, maybe nobody tried…
Remove

nomodeset

from kernel parameters.

Add/edit a file in /etc/modprobe.d containing

options nvidia-drm modeset=1

After that, run

sudo update-initramfs -u

After reboot, issuing

sudo cat /sys/module/nvidia_drm/parameters/modeset

should return ‘Y’

Use the following xorg.conf

Section "ServerLayout"
    Identifier     "layout"
    Screen      0  "master" 0 0
    Inactive       "slave"
EndSection

Section "Device"
    Identifier     "slave"
    Driver         "modesetting"
    Option         "AccelMethod" "none"
    BusID          "PCI:5:0:0"
EndSection

Section "Device"
    Identifier     "master"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusID          "PCI:3:0:0"
    Option         "AllowEmptyInitialConfiguration"
EndSection

Section "Screen"
    Identifier     "master"
    Device         "master"
    SubSection     "Display"
        Virtual     1920 1080
    EndSubSection
EndSection

When you start X, you should have a desktop on two displays, the other two should be dark.
open a terminal, issue

xrandr --listproviders

this should return two providers

Providers: number : 2
Provider 0: id: 0x2b5 cap: 0x1, Source Output crtcs: 0 outputs: 0 associated providers: 1 name:NVIDIA-0
Provider 1: id: 0x45 cap: 0x2, Sink Output crtcs: 3 outputs: 3 associated providers: 1 name:modesetting

If both are there, issue

xrandr --setprovideroutputsource modesetting NVIDIA-0
xrandr --auto

…and hope the other two displays come alive.
If both providers show up but the displays don’t come alive, your xserver might be too old. Then try the same with xserver 1.17.4 and up.

1 Like
8

Wow, thanks for all the help. I’m going to give the prime drivers a shot next week when I have some time.

Taking a look at the NVidia documentation, it appears that basemosaic and slimosaic are only useful for NVS and Quadro cards, but you’re saying that I can use basemosaic with two 580s. I had assumed that mosaic wouldn’t work or be useful for this configuration. Would using an SLI link for two GTX 580 cards have any difference in desktop rendering?

I feel like this is a limitation artificially imposed by NVidia to push NVS/Quadro. Is this the case?

9

Alright! I just tried this out without upgrading to Prime, since you’d said no one has attempted it to your knowledge.

One note: the screen’s virtual size needed to be corrected to fit the two monitors, so it wasn’t working upon boot. Needed to have SubSection “Display” virtual setting set to 3840 1080. It didn’t work at first because it was set to 1920 1080, which caused X to freak out.

modeset does return ‘Y’:

root@nope:~/nvidia_log_output# sudo cat /sys/module/nvidia_drm/parameters/modeset
Y

xrandr --listproviders does show two cards:

root@nope:~/nvidia_log_output# xrandr --listproviders
Providers: number : 2
Provider 0: id: 0x2a7 cap: 0x1, Source Output crtcs: 2 outputs: 5 associated providers: 0 name:NVIDIA-0
Provider 1: id: 0x44 cap: 0x0 crtcs: 2 outputs: 3 associated providers: 0 name:modesetting

however, there’s an error in the next command you gave:

root@nope:~/nvidia_log_output# xrandr --setprovideroutputsource modesetting NVIDIA-0
X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  140 (RANDR)
  Minor opcode of failed request:  35 ()
  Value in failed request:  0x44
  Serial number of failed request:  16
  Current serial number in output stream:  17

I’m presuming this is due to the 390.25 drivers.

You will find the bug report attached. I may try out Prime, but would like to see if you have any ideas before continuing.

EDIT: Just upgraded to 390.42 with an attempt prior to going to Prime. Same error as above.
nvidia-bug-report_corrected_virtual_size.log (484 KB)

10

About SLI status and BaseMosaic limitations:
https://devtalk.nvidia.com/default/topic/1000833/linux/unity-doesn-t-start-on-ubuntu-16-04-when-sli-on/2
https://devtalk.nvidia.com/default/topic/579449/?comment=3975202

The previous config already was PRIME, it’s nothing you install but configure. It’s included in the kernel gpu drivers and the Xserver. That was the first attempt with nvidia prop. driver alone. This doesn’t work because the nvidia kernel driver is lacking an implementation. So the capability shown by the modesetting driver is:

cap: 0x0

needed is:

cap: 0x2, Sink Output

So now it gets tricky. The second attempt would be to use a split setup, the master driven by the nvidia proprietary driver, the slave by the nouveau kernel driver.
find/edit/add a file in /etc/modprobe.d containing

options nvidia NVreg_AssignGpus="0:03:00.0"

run

sudo update-initramfs -u

reboot. Stop the Xserver.

sudo lspci -v -d 10de:* | grep "Kernel driver"

should return 4 lines, two with snd_hda_intel, one with nvidia and one empty.
Run

sudo modprobe nouveau

then again

sudo lspci -v -d 10de:* | grep "Kernel driver"

The empty line should have turned to nouveau.
Then start the Xserver with the same xorg.conf from post #7, run

xrandr --listproviders

Check if both providers are there and the capability of the modesetting driver is correct, then try

xrandr --setprovideroutputsource modesetting NVIDIA-0
xrandr --auto
11

Okay! Getting close… I think?

I switched to the 370.23 driver, assuming that was the first issue which was getting in the way.

First, I presume that nouveau should still be blacklisted in modprobe, so I’ve left that as follows:

root@nope:~# cat /etc/modprobe.d/blacklist-nouveau.conf 
blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0

Without blacklisting, the xorg.conf you’d recommended above doesn’t boot properly because it’s referring to the nvidia driver when nouveau loads.

I still have nomodeset removed from the linux kernel grub.cfg option.

Just want to make sure there’s no kernel issues going on here, so I am going to put make sure you realize that I am working with 4.4.0-116-generic:

Linux nope 4.4.0-116-generic #140~14.04.1-Ubuntu SMP Fri Feb 16 09:25:20 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

EDIT: I did add a modprobe.d rule as follows, and did run update-initramfs -u afterward:

root@nope:~$ cat /etc/modprobe.d/nvidia-add.conf 
options nvidia NVreg_AssignGpus="0:03:00.0"

I seem to recall reading something where a kernel beyond 4.5 was required, but am unsure if this applies to my situation or not.

anyway, I’m seeing some inconsistencies with the lspci command return. There’s no “blank” line for the “Kernel driver” line in the card which is disabled. Here’s the return on that command:

root@nope:~# lspci -v -d 10de:*
03:00.0 VGA compatible controller: NVIDIA Corporation GF110 [GeForce GTX 580] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: eVga.com. Corp. Device 1580
	Physical Slot: 3
	Flags: bus master, fast devsel, latency 0, IRQ 58
	Memory at 9c000000 (32-bit, non-prefetchable) 
	Memory at 80000000 (64-bit, prefetchable) 
	Memory at 88000000 (64-bit, prefetchable) 
	I/O ports at 4000 
	[virtual] Expansion ROM at 9d000000 [disabled] 
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100] Virtual Channel
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: nvidia

03:00.1 Audio device: NVIDIA Corporation GF110 High Definition Audio Controller (rev a1)
	Subsystem: eVga.com. Corp. Device 1580
	Physical Slot: 3
	Flags: bus master, fast devsel, latency 0, IRQ 19
	Memory at 9d080000 (32-bit, non-prefetchable) 
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Kernel driver in use: snd_hda_intel

05:00.0 VGA compatible controller: NVIDIA Corporation GF110 [GeForce GTX 580] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: eVga.com. Corp. Device 1580
	Physical Slot: 1
	Flags: fast devsel, IRQ 11
	Memory at 9a000000 (32-bit, non-prefetchable) [disabled] 
	Memory at 90000000 (64-bit, prefetchable) [disabled] 
	Memory at 98000000 (64-bit, prefetchable) [disabled] 
	I/O ports at 3000 [disabled] 
	Expansion ROM at 9b000000 [disabled] 
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100] Virtual Channel
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>

05:00.1 Audio device: NVIDIA Corporation GF110 High Definition Audio Controller (rev a1)
	Subsystem: eVga.com. Corp. Device 1580
	Physical Slot: 1
	Flags: bus master, fast devsel, latency 0, IRQ 17
	Memory at 9b080000 (32-bit, non-prefetchable) 
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Kernel driver in use: snd_hda_intel

notice in the 05:00.0 that there isn’t a line about which kernel driver is used. Is this due to my kernel version? I would like to upgrade to Ubuntu 16.04 on this machine, but I can no longer run VMware Workstation 11 when upgrading past kernel 4.5, which limits me to to Ubuntu 14.04. :(

Running modprobe nouveau seems to work alright, and nouveau is loaded, but running lspci -v -d 10de:* still results in the output above.

listing providers still appears to return 1:

root@nope:~# xrandr --listproviders
Providers: number : 1
Provider 0: id: 0x279 cap: 0x1, Source Output crtcs: 2 outputs: 5 associated providers: 0 name:NVIDIA-0

and setprovideroutputsource can’t find the modesetting device.

root@nope:~# xrandr --setprovideroutputsource modesetting NVIDIA-0
Could not find provider with name modesetting

If this is a kernel mismatch issue, I understand, and will probably be looking for a different video device.

I’ll be honest, I’d be fine with settling for three video outputs at this point. It seems like two is the greatest amount I can get working properly with two video cards, unless I want to use compositing, which eats up a massive amount of processor power (previously discussed).

Thanks again for your help. This has been quite educational.

You will find my bugreport attached.
nvidia-bug-report.log (946 KB)

12

Yes, the nouveau blacklist hast to stay so it doesn’t autoload on boot.
Everything went as planned up until modprobing nouveau, it didn’t bind to the second nvidia gpu where no Kernel driver is displayed.
So maybe give it a little push:
Same procedure, but after modprobe nouveau, check if the file
/sys/bus/pci/drivers/nouveau/bind
exists, then issue
sudo tee /sys/bus/pci/drivers/nouveau/bind <<< 0000:05:00.0
and check with lspci if the driver got bound to the second gpu.
Edit: maybe use “0000:05:00.0”

13

Continuing on, modprobe nouveau worked… sort of.

lsmod shows as follows:

root@nope:~# lsmod
Module                  Size  Used by
nouveau              1482752  0 
mxm_wmi                16384  1 nouveau
wmi                    20480  2 mxm_wmi,nouveau
video                  40960  1 nouveau
ttm                    94208  1 nouveau
i2c_algo_bit           16384  1 nouveau
ipt_MASQUERADE         16384  3 
nf_nat_masquerade_ipv4    16384  1 ipt_MASQUERADE
iptable_nat            16384  1 
nf_nat_ipv4            16384  1 iptable_nat
nf_nat                 24576  2 nf_nat_ipv4,nf_nat_masquerade_ipv4
nf_conntrack_ipv4      16384  2 
nf_defrag_ipv4         16384  1 nf_conntrack_ipv4
xt_conntrack           16384  1 
nf_conntrack          106496  5 nf_nat,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4
ipt_REJECT             16384  2 
nf_reject_ipv4         16384  1 ipt_REJECT
ip6table_filter        16384  0 
ip6_tables             28672  1 ip6table_filter
ebtable_nat            16384  0 
ebtables               36864  1 ebtable_nat
vmnet                  53248  13 
vmw_vsock_vmci_transport    32768  0 
vsock                  36864  1 vmw_vsock_vmci_transport
vmw_vmci               65536  1 vmw_vsock_vmci_transport
vmmon                  86016  0 
xt_CHECKSUM            16384  2 
iptable_mangle         16384  1 
xt_comment             16384  6 
xt_tcpudp              16384  10 
bridge                126976  0 
stp                    16384  1 bridge
llc                    16384  2 stp,bridge
iptable_filter         16384  1 
ip_tables              24576  3 iptable_filter,iptable_mangle,iptable_nat
x_tables               36864  12 ip6table_filter,xt_comment,xt_CHECKSUM,ip_tables,xt_tcpudp,ipt_MASQUERADE,xt_conntrack,iptable_filter,ebtables,ipt_REJECT,iptable_mangle,ip6_tables
autofs4                40960  0 
bnep                   20480  2 
rfcomm                 69632  12 
nvidia_drm             53248  1 
nvidia_modeset        774144  4 nvidia_drm
nvidia              11907072  70 nvidia_modeset
snd_hda_codec_hdmi     53248  8 
snd_hda_codec_realtek    90112  1 
drm_kms_helper        151552  2 nouveau,nvidia_drm
applesmc               20480  0 
snd_hda_codec_generic    73728  1 snd_hda_codec_realtek
drm                   360448  6 ttm,drm_kms_helper,nouveau,nvidia_drm
cp210x                 24576  0 
input_polldev          16384  1 applesmc
snd_usb_audio         176128  2 
snd_hda_intel          40960  3 
usbserial              53248  1 cp210x
intel_powerclamp       16384  0 
snd_hda_codec         135168  4 snd_hda_codec_realtek,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_intel
snd_usbmidi_lib        36864  1 snd_usb_audio
fb_sys_fops            16384  1 drm_kms_helper
input_leds             16384  0 
snd_hda_core           73728  5 snd_hda_codec_realtek,snd_hda_codec_hdmi,snd_hda_codec_generic,snd_hda_codec,snd_hda_intel
btusb                  45056  0 
coretemp               16384  0 
btrtl                  16384  1 btusb
snd_hwdep              16384  2 snd_usb_audio,snd_hda_codec
syscopyarea            16384  1 drm_kms_helper
btbcm                  16384  1 btusb
sysfillrect            16384  1 drm_kms_helper
kvm_intel             172032  0 
snd_pcm               106496  5 snd_usb_audio,snd_hda_codec_hdmi,snd_hda_codec,snd_hda_intel,snd_hda_core
btintel                16384  1 btusb
snd_seq_midi           16384  0 
sysimgblt              16384  1 drm_kms_helper
bluetooth             520192  25 bnep,btbcm,btrtl,btusb,rfcomm,btintel
snd_seq_midi_event     16384  1 snd_seq_midi
kvm                   544768  1 kvm_intel
snd_rawmidi            32768  2 snd_usbmidi_lib,snd_seq_midi
snd_seq                69632  2 snd_seq_midi_event,snd_seq_midi
snd_seq_device         16384  3 snd_seq,snd_rawmidi,snd_seq_midi
snd_timer              32768  2 snd_pcm,snd_seq
irqbypass              16384  1 kvm
snd                    81920  23 snd_hda_codec_realtek,snd_usb_audio,snd_hwdep,snd_timer,snd_hda_codec_hdmi,snd_pcm,snd_seq,snd_rawmidi,snd_hda_codec_generic,snd_usbmidi_lib,snd_hda_codec,snd_hda_intel,snd_seq_device
soundcore              16384  1 snd
lpc_ich                24576  0 
shpchp                 36864  0 
ioatdma                53248  0 
i5500_temp             16384  0 
i7core_edac            24576  0 
edac_core              53248  2 i7core_edac
dca                    16384  1 ioatdma
mac_hid                16384  0 
parport_pc             36864  0 
ppdev                  20480  0 
lp                     20480  0 
parport                49152  3 lp,ppdev,parport_pc
hid_generic            16384  0 
e1000e                237568  0 
firewire_ohci          40960  0 
usbhid                 49152  0 
hid                   118784  2 hid_generic,usbhid
firewire_core          65536  1 firewire_ohci
ptp                    20480  1 e1000e
pata_acpi              16384  0 
crc_itu_t              16384  1 firewire_core
pps_core               20480  1 ptp
fjes                   28672  0

I can see nouveau is being loaded:

root@nope:~# lsmod |grep -i nouveau
nouveau              1482752  0 
mxm_wmi                16384  1 nouveau
wmi                    20480  2 mxm_wmi,nouveau
video                  40960  1 nouveau
ttm                    94208  1 nouveau
i2c_algo_bit           16384  1 nouveau
drm_kms_helper        151552  2 nouveau,nvidia_drm
drm                   360448  6 ttm,drm_kms_helper,nouveau,nvidia_drm

Sadly, no directory in /sys/bus/pci/drivers/ for nouveau is created.

from dmesg:

[   15.919006] NVRM: The NVIDIA probe routine was not called for 2 device(s).
[   15.919007] NVRM: This can occur when a driver such as: 
[   15.919007] NVRM: nouveau, rivafb, nvidiafb or rivatv 
[   15.919007] NVRM: was loaded and obtained ownership of the NVIDIA device(s).
[   15.919010] NVRM: Try unloading the conflicting kernel module (and/or
[   15.919010] NVRM: reconfigure your kernel without the conflicting
[   15.919010] NVRM: driver(s)), then try loading the NVIDIA kernel module
[   15.919010] NVRM: again.
[   15.919014] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  370.23  Mon Aug  8 18:02:36 PDT 2016
[   16.998549] NVRM: Your system is not currently configured to drive a VGA console
[   16.998552] NVRM: on the primary VGA device. The NVIDIA Linux graphics driver
[   16.998554] NVRM: requires the use of a text-mode VGA console. Use of other console
[   16.998555] NVRM: drivers including, but not limited to, vesafb, may result in
[   16.998557] NVRM: corruption and stability problems, and is not supported.

I can’t create a directory in /sys/bus/pci/drivers/

root@nope:/sys/bus/pci/drivers# mkdir nouveau
mkdir: cannot create directory ‘nouveau’: Operation not permitted

Of course I can’t tee into a file/directory which doesn’t exist :/

To sum up: it looks like nouveau isn’t creating a directory in /sys/bus/pci/drivers, so I can’t add anything in there. Not sure if this is a modprobe issue, or a kernel issue.

14

Yep, sad and bit odd that no sysfs files get created. But thanks anyway for at least trying, otherwise still nobody would know the obstacles. I’ll investigate the co-existance of nouveau and the proprietary driver a bit deeper when time permits. Up till then, you’re left with the known options less one.;)

15

Alright, that’s no big deal.

In the mean time, you’d mentioned that I could use the following configuration:

I now presently have two GTX580 cards installed, with an SLI bridge joining them. I understand that SLI is quite possibly broken, but I’d be interested in getting basemosaic working if it will properly support 3 monitors with compositing and hardware acceleration.

Not sure if you know, but the 580GTX does have two DVI outputs and an HDMI output, but I’m seeing reports that only two of these outputs can be active at the same time. I’m wondering if you have a good example of a basemosaic configuration I could try to utilize in the meantime.

Thanks!

16

GTX580 is a Fermi, those only have two heads (display engines), so even if it has more connectors it can only drive two monitors directly attached. It’s just an extra, to have a choice which connector fits best with an available monitor.
BaseMosaic would work with a very simple xorg.conf just containing

Section "Device"
    Identifier     "Nvidia"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusID          "PCI:3:0:0"
    Option         "AllowEmptyInitialConfiguration"
    Option         "BaseMosaic" "true"
EndSection

In case you want to try SLI with it, replace ‘Option “BaseMosaic” “true”’ with
Option “SLI” “MOSAIC”
Indeed, only one gpu has to be configured.

17

Alright, so I tried as you’d said above - just a simple xorg.conf with basemosaic listed for a single device.

Of course, I forgot to remove the following, so only one device was initially detected:

root@nope:~$ cat /etc/modprobe.d/nvidia-add.conf 
options nvidia NVreg_AssignGpus="0:03:00.0"

which was throwing errors about only one card being detected.

Current configuration:

  • nomodeset removed
  • 2x GTX580s are now detected
  • nvidia driver 370.23
  • Ubuntu 14.04.1
  • Kernel Version 4.4.0-116-generic x86_64

After removing the nvreg_assignGPUsthat and rebooting, I’m seeing the following error in Xorg.0log:

[    45.916] (EE) NVIDIA(GPU-0): Failed to find a valid Base Mosaic configuration.
[    45.916] (EE) NVIDIA(GPU-0): Invalid Base Mosaic configuration 1 of 1:
[    45.916] (EE) NVIDIA(GPU-0): GPUs:
[    45.916] (EE) NVIDIA(GPU-0):     1) NVIDIA GPU at PCI:3:0:0
[    45.916] (EE) NVIDIA(GPU-0):     2) NVIDIA GPU at PCI:5:0:0
[    45.916] (EE) NVIDIA(GPU-0): Errors:
[    45.916] (EE) NVIDIA(GPU-0):     - Unknown error
[    45.916] (WW) NVIDIA(GPU-0): Failed to find a valid Base Mosaic configuration for the
[    45.916] (WW) NVIDIA(GPU-0):     NVIDIA graphics device PCI:3:0:0. Please see Chapter 28:
[    45.916] (WW) NVIDIA(GPU-0):     Configuring SLI and Multi-GPU FrameRendering in the README
[    45.916] (WW) NVIDIA(GPU-0):     for troubleshooting suggestions.

It appears that without some sort of basemosaic configuration, this isn’t going to automatically configure itself.

What is that configuration supposed to look like to get these two GTX580 cards working with 3 monitors? You’ll find the bug report attached.
nvidia-bug-report.log.gz (100 KB)

18

“Unknown Error”, always helpful;)
I think you also have to turn off modeset=1, so remove also that file in /etc/modprobe.d and run sudo update-initramfs -u afterwards.
For a setup-hint, add to the device section:

Option        "MetaModes" "GPU-0.DFP-0: 1920x1080 +0+0, GPU-0.DFP-2:  1920x1080 +1920+0, GPU-1.DFP-0: 1920x1080 +3840+0"
19

Okay, removing

options nvidia-drm modeset=1

and updating initramfs.

Same issue - no valid metamode BaseMosaic configuration.

Current xorg.conf:

ection "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "HP E221"
    HorizSync       24.0 - 94.0
    VertRefresh     50.0 - 76.0
    Option         "DPMS"
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor1"
    VendorName     "Unknown"
    ModelName      "HP E221"
    HorizSync       24.0 - 94.0
    VertRefresh     50.0 - 76.0
    Option         "DPMS"
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor2"
    VendorName     "Unknown"
    ModelName      "HP E221"
    HorizSync       24.0 - 94.0
    VertRefresh     50.0 - 76.0
    Option         "DPMS"
EndSection

Section "Monitor"
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor3"
    VendorName     "Unknown"
    ModelName      "HP E221"
    HorizSync       24.0 - 94.0
    VertRefresh     50.0 - 76.0
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 580"
    BusID          "PCI:3:0:0"
EndSection

Section "Device"
    Identifier     "Device1"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 580"
    BusID          "PCI:5:0:0"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "BaseMosaic" "True"
    Option         "MetaModes" "GPU-1.TV-2: 1920x1080 +0+0, GPU-0.TV-0:  1920x1080 +1920+0, GPU-1.TV-2: 1920x1080 +3840+0"
#    Option         "nvidiaXineramaInfo" "FALSE"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

nvidia-query-info lists:

root@nope:~# nvidia-xconfig --query-gpu-info
Number of GPUs: 2

GPU #0:
  Name      : GeForce GTX 580
  UUID      : GPU-5cbd392c-7fc6-23f1-ab15-9261c1ed9f76
  PCI BusID : PCI:3:0:0

  Number of Display Devices: 2

  Display Device 0 (TV-0):
      EDID Name             : HP E221
      Minimum HorizSync     : 24.000 kHz
      Maximum HorizSync     : 94.000 kHz
      Minimum VertRefresh   : 50 Hz
      Maximum VertRefresh   : 76 Hz
      Maximum PixelClock    : 170.000 MHz
      Maximum Width         : 1920 pixels
      Maximum Height        : 1080 pixels
      Preferred Width       : 1920 pixels
      Preferred Height      : 1080 pixels
      Preferred VertRefresh : 60 Hz
      Physical Width        : 500 mm
      Physical Height       : 290 mm

  Display Device 1 (TV-2):
      EDID Name             : HP E221
      Minimum HorizSync     : 24.000 kHz
      Maximum HorizSync     : 94.000 kHz
      Minimum VertRefresh   : 50 Hz
      Maximum VertRefresh   : 76 Hz
      Maximum PixelClock    : 170.000 MHz
      Maximum Width         : 1920 pixels
      Maximum Height        : 1080 pixels
      Preferred Width       : 1920 pixels
      Preferred Height      : 1080 pixels
      Preferred VertRefresh : 60 Hz
      Physical Width        : 500 mm
      Physical Height       : 290 mm

GPU #1:
  Name      : GeForce GTX 580
  UUID      : GPU-4afb5f0b-c4fb-be37-9573-99aac2abd504
  PCI BusID : PCI:5:0:0

  Number of Display Devices: 2

  Display Device 0 (TV-0):
      EDID Name             : HP E221
      Minimum HorizSync     : 24.000 kHz
      Maximum HorizSync     : 94.000 kHz
      Minimum VertRefresh   : 50 Hz
      Maximum VertRefresh   : 76 Hz
      Maximum PixelClock    : 170.000 MHz
      Maximum Width         : 1920 pixels
      Maximum Height        : 1080 pixels
      Preferred Width       : 1920 pixels
      Preferred Height      : 1080 pixels
      Preferred VertRefresh : 60 Hz
      Physical Width        : 500 mm
      Physical Height       : 290 mm

  Display Device 1 (TV-2):
      EDID Name             : HP E221
      Minimum HorizSync     : 24.000 kHz
      Maximum HorizSync     : 94.000 kHz
      Minimum VertRefresh   : 50 Hz
      Maximum VertRefresh   : 76 Hz
      Maximum PixelClock    : 170.000 MHz
      Maximum Width         : 1920 pixels
      Maximum Height        : 1080 pixels
      Preferred Width       : 1920 pixels
      Preferred Height      : 1080 pixels
      Preferred VertRefresh : 60 Hz
      Physical Width        : 500 mm
      Physical Height       : 290 mm

You will find error log attached.

nvidia-bug-report.log.gz (105 KB)

20

There’s really no config magic involved. It’s only turning it on or off, so the Device snippet I posted should be enough, anything else is driver internal.
Maybe it’s confused about the video bridge installed, so instead of basemosaic, use
Option “SLI” “MOSAIC”
or remove the bridge and try again with BaseMosaic.
If it doesn’t works at all, that’s a driver bug.