Cannot get nvidia driver to work with two RTX A6000

I have a system with Rocky 8 Linux installed and the nvidia drivers from the RPM repo. It has two RTX A6000’s but we are using only one monitor on one card.

cat /proc/driver/nvidia/version

NVRM version: NVIDIA UNIX x86_64 Kernel Module 530.30.02 Wed Feb 22 04:11:39 UTC 2023
GCC version: gcc version 8.5.0 20210514 (Red Hat 8.5.0-16) (GCC)

I cannot get any configuration in /etc/X11/xorg.conf to work for this. If I try the following from “nvidia-xconfig --sli=mosaic”:

Section "Device"
        Identifier  "Videocard0"
        Driver      "nvidia"
        VendorName  "NVIDIA Corporation"
        BoardName   "RTX A6000" # 51:00.0
        BusID       "PCI:81:0:0"
        Option      "AllowEmptyInitialConfiguration" "on"
EndSection

Section "Device"
        Identifier  "Videocard1"
        Driver      "nvidia"
        VendorName  "NVIDIA Corporation"
        BoardName   "RTX A6000" # 9c:00.0
        BusID       "PCI:156:0:0"
        Option      "AllowEmptyInitialConfiguration" "on"
EndSection

Section "Screen"
    Identifier     "Default Screen"
    Device         "Videocard0"
    DefaultDepth    24
    Option         "BaseMosaic" "Off"
    Option         "AllowEmptyInitialConfiguration" "on"
    Option         "SLI" "mosaic"
    SubSection     "Display"
        Depth       24
        Modes      "nvidia-auto-select"
    EndSubSection
EndSection

The X server fails to start with

[ 69515.336] (**) NVIDIA(0): Depth 24, (--) framebuffer bpp 32
[ 69515.336] (==) NVIDIA(0): RGB weight 888
[ 69515.336] (==) NVIDIA(0): Default visual is TrueColor
[ 69515.336] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[ 69515.336] (**) NVIDIA(0): Option "SLI" "mosaic"
[ 69515.336] (**) NVIDIA(0): Option "BaseMosaic" "Off"
[ 69515.336] (**) NVIDIA(0): Option "AllowEmptyInitialConfiguration" "on"
[ 69515.336] (**) NVIDIA(0): NVIDIA SLI Mosaic mode selected.
[ 69515.336] (**) NVIDIA(0): Enabling 2D acceleration
[ 69515.336] (II) Loading sub module "glxserver_nvidia"
[ 69515.336] (II) LoadModule: "glxserver_nvidia"
[ 69515.336] (II) Loading /usr/lib64/xorg/modules/extensions/libglxserver_nvidia.so
[ 69515.340] (II) Module glxserver_nvidia: vendor="NVIDIA Corporation"
[ 69515.340]    compiled for 1.6.99.901, module version = 1.0.0
[ 69515.340]    Module class: X.Org Server Extension
[ 69515.340] (II) NVIDIA GLX Module  530.30.02  Wed Feb 22 03:51:52 UTC 2023
[ 69515.340] (II) NVIDIA: The X server supports PRIME Render Offload.
[ 69515.340] (WW) NVIDIA(GPU-0): The "BaseMosaic" X configuration option should be used for
[ 69515.340] (WW) NVIDIA(GPU-0):     this configuration instead of the "SLI" "Mosaic" option.
[ 69515.340] (WW) NVIDIA(GPU-0):     The "BaseMosaic" option will be assumed.  Please see
[ 69515.340] (WW) NVIDIA(GPU-0):     Chapter 31: Configuring SLI and Multi-GPU Mosaic and
[ 69515.340] (WW) NVIDIA(GPU-0):     Appendix B: X Config Options for more information.
[ 69515.340] (II) NVIDIA(GPU-0): NVIDIA SLI enabled.
[ 69515.340] (EE) NVIDIA(GPU-0): Failed to select a display subsystem.
[ 69515.340] (EE) NVIDIA(GPU-0): Only one GPU will be used for this X screen.
[ 69515.340] (EE) NVIDIA(GPU-0): The NVIDIA graphics device PCI:81:0:0 is part of an active SLI
[ 69515.341] (EE) NVIDIA(GPU-0):     configuration and is currently unavailable for single GPU
[ 69515.341] (EE) NVIDIA(GPU-0):     rendering.  Please see Chapter 31: Configuring SLI and
[ 69515.341] (EE) NVIDIA(GPU-0):     Multi-GPU Mosaic in the README for troubleshooting
[ 69515.341] (EE) NVIDIA(GPU-0):     information.
[ 69515.341] (EE)  *** Aborting ***
[ 69515.341] (EE) NVIDIA(0): Failing initialization of X screen
[ 69515.341] (II) UnloadModule: "nvidia"
[ 69515.341] (II) UnloadSubModule: "glxserver_nvidia"
[ 69515.341] (II) Unloading glxserver_nvidia
[ 69515.341] (II) UnloadSubModule: "wfb"
[ 69515.341] (II) UnloadSubModule: "fb"
[ 69515.341] (**) NVIDIA(G0): Depth 24, (--) framebuffer bpp 32
[ 69515.341] (==) NVIDIA(G0): RGB weight 888
[ 69515.341] (==) NVIDIA(G0): Default visual is TrueColor
[ 69515.341] (==) NVIDIA(G0): Using gamma correction (1.0, 1.0, 1.0)
[ 69515.341] (**) NVIDIA(G0): Option "SLI" "mosaic"
[ 69515.341] (**) NVIDIA(G0): Option "BaseMosaic" "Off"
[ 69515.341] (**) NVIDIA(G0): Option "AllowEmptyInitialConfiguration" "on"
[ 69515.341] (**) NVIDIA(G0): NVIDIA SLI Mosaic mode selected.
[ 69515.341] (WW) NVIDIA(G0): SLI is available only on screen 0.  Disabling SLI.
[ 69515.341] (EE) NVIDIA(G0): Only one X screen is supported when SLI is enabled.  Disabling
[ 69515.341] (EE) NVIDIA(G0):     this screen.
[ 69515.341] (EE) NVIDIA(G0): Failing initialization of X screen
[ 69515.341] (II) UnloadModule: "nvidia"
[ 69515.341] (II) UnloadSubModule: "wfb"
[ 69515.341] (II) UnloadSubModule: "fb"
[ 69515.341] (EE) Screen(s) found, but none have a usable configuration.
[ 69515.341] (EE)
Fatal server error:
[ 69515.341] (EE) no screens found(EE)

Trying with

    Option         "BaseMosaic" "Off"
    Option         "SLI" "off"

and

    Option         "BaseMosaic" "On"
    Option         "SLI" "off"

and

    Option         "BaseMosaic" "Off"
    Option         "SLI" "Single"

and

    Option         "BaseMosaic" "On"
    Option         "SLI" "on"

all fail as well.

I tried configuring for just the one GPU and setting ProbeAllGpus and AutoAddGpu off but it still gives me “part of active SLI and is currently unavailable for single GPU” error and fails.