Need help with PRIME render offload

I saw the announcement about PRIME render offload in 435.17, and wanted to try it. I’m on Arch Linux, so I can’t use the PPA containing Xorg server with the needed patches, so I compiled Xorg from master branch. However I had an error about ABI incompatibility.

[   433.440] This server has a video driver ABI version of 25.0 that this
driver does not officially support.  Please check
http://www.nvidia.com/ for driver updates or downgrade to an X
server with a supported driver ABI.

After adding the “IgnoreABI” flag, now I get

[   433.440] (WW) NVIDIA: This driver was compiled against the X.Org server SDK from commit 29a8ba
a031a87ef44d1e5320ecec5015d26fd385 and may not be compatible with the final version of this SDK.

I then compiled and installed Xorg at the mentioned commit 29a8baa0, but it still didn’t remove the ABI mismatch or the compatibility warning. I have a few lines of “NVIDIA(G0)”, which is a good sign according to the README. However that ends with

[   433.552] (==) NVIDIA(G0): Depth 24, (==) framebuffer bpp 32
[   433.552] (==) NVIDIA(G0): RGB weight 888
[   433.552] (==) NVIDIA(G0): Default visual is TrueColor
[   433.552] (==) NVIDIA(G0): Using gamma correction (1.0, 1.0, 1.0)
[   433.552] (**) NVIDIA(G0): Enabling 2D acceleration
[   433.552] (EE) NVIDIA(G0): GPU screens are not yet supported by the NVIDIA driver
[   433.552] (EE) NVIDIA(G0): Failing initialization of X screen

And in xrandr --listproviders I only see the Intel iGPU.

Providers: number : 1
Provider 0: id: 0x45 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 3 outputs: 3 associated providers: 0 name:modesetting

I’m not sure if this is because I didn’t compile the correct version of Xorg, and/or my Xorg configuration. Is it possible to provide some instructions on how the Ubuntu ppa package is built, for other distros to follow?

Here’s my nvidia-bug-report.log. Thanks for any help you could provide.

I’m on Gentoo so patching is a breeze. To avoid the ABI mismatch just take the current xserver 1.20 sources (I used 1.20.5) and apply the five patches. I’m attaching them shortly.

offloadpatches.zip (6.68 KB)

OK, I seem to be getting much closer. I compiled and installed the server-1.20-branch, and replaced my configuration to be similar to this post. Now everything works, but only after I either logout from KDE and back in, or restart SDDM (display manager).

Before I do that, there’s this error in Xorg.0.log:

[     5.426] (EE) Screen 1 deleted because of no matching config section.
[     5.426] (II) UnloadModule: "nvidia"

And xrandr lists only the iGPU.

Please attach the full xorg logs from before and after X restart.

Before X restart: https://gist.github.com/2201035474a06582fb5c6a8eb9faee2e
After X restart: https://gist.github.com/c53c10a5e0090e84ce03b09372cee672

This looks like on first start the nvidia drm device is missing.
1.start

[     3.927] (II) xfree86: Adding drm device (/dev/dri/card0)

2.start

[   197.329] (II) xfree86: Adding drm device (/dev/dri/card0)
[   197.329] (II) xfree86: Adding drm device (/dev/dri/card1)

Maybe some timing problem with driver loading, don’t know what to make of it right now.
Please create a new nvidia-bug-report.log, maybe something is visible in dmesg.

Maybe it helps to preload the kernel driver. Try this:
Create /etc/systemd/system/load-nvidia.service

[Unit]
Description=Load Nvidia driver
Before=display-manager.service

[Service]
Type=oneshot
ExecStart=/sbin/modprobe nvidia

[Install]
WantedBy=display-manager.service

and enable it using
sudo systemctl enable load-nvidia
make sure /sbin/modprobe is the correct path. Then reboot.

Preloading the kernel driver worked!

I spoke too soon. With the preload service enabled, I still have the problem some of the time after a reboot. I changed the command to “/usr/bin/sh -c ‘/usr/bin/modprobe nvidia; sleep 1’”, and now it seems to work consistently.

Any idea why this would be needed?

Here’s the nvidia-bug-report.log when I don’t preload: https://gist.github.com/84d7316b14cd13d4daffb4695530dea1

Thanks for all the help!

I got PRIME Offload to work on Arch Linux by building the default Xorg 1.20.5 with the 5 added patches and using nvidia-beta-dkms from AUR. Didn’t need any kernel module (re)loading.

Note that nvidia-utils(-beta) installs a file at “/usr/share/X11/xorg.conf.d/10-nvidia-drm-outputclass.conf” which breaks PRIME output, so I just cleared the file and set it to immutable to prevent overwriting (a more proper solution would be to ignore it in pacman.conf):

sudo truncate -s0 /usr/share/X11/xorg.conf.d/10-nvidia-drm-outputclass.conf
sudo chattr +i /usr/share/X11/xorg.conf.d/10-nvidia-drm-outputclass.conf

Then just edit /etc/X11/xorg.conf to include the ModulePaths:

Section "ServerLayout"
        Identifier "layout"
        Option "AllowNVIDIAGPUScreens"
EndSection

Section "Device"
        Identifier "intel"
        Driver "modesetting"
EndSection

Section "Device"
        Identifier "nvidia"
        Driver "nvidia"
EndSection

Section "OutputClass"
        Identifier "intel"
        MatchDriver "i915"
        Driver "modesetting"
EndSection

Section "OutputClass"
        Identifier "nvidia"
        MatchDriver "nvidia-drm"
        Driver "nvidia"
        ModulePath "/usr/lib/nvidia/xorg"
        ModulePath "/usr/lib/xorg/modules"
EndSection

I noticed that OutputClass and Device sections must be specified for both GPUs, otherwise I ended up staring at a black screen. YMMV.

hagabaka, looking at this:

[     3.927] (II) xfree86

your system is mighty fast, and if the new nvidia driver now takes a bit longer to initialize, the Xserver doesn’t see the device. Don’t know if Arch has the possibility to embed the driver in the initrd so it’s available at an earlier point to avoid this.

I do have the system configured to include the nvidia modules in initramfs, but I probably forgot to do it after upgrading the nvidia driver, and the package manager hook to automatically do it didn’t kick in for some reason. I just rebuilt the image and rebooted 3 times without the preload service, and it has been working. Thanks!