GPU1 not showing display in a dual gpu system

I have 2 GPUs, RTX 3090 & RTX 3090Ti in the system with 3090 connected on slot1 and 3090 on slot2.
They both are detected and working fine. However, gpu1 (RTX 3090Ti) doesn’t show the display when connected to the HDMI. 3090 works fine when connected. I am using only 1 monitor and connecting to 1 gpu at a time.
I am able to get the display from both of them in Windows, but not in ubuntu.
This is what I see:
nvidia-smi -L
GPU 0: NVIDIA GeForce RTX 3090 (UUID: some id)
GPU 1: NVIDIA GeForce RTX 3090 Ti (UUID: some id)
How can I switch the GPUs? I have the pcie4 slots in 2x8 configuration when both slots are occupied, so that couldn’t be an issue.

Hello @net.lazuli.0m and welcome to the NVIDIA developer forums!

Ubuntu with a Gnome Desktop usually uses X11 as the Display server. That in turn uses a default configuration which always uses the first available GPU to probe for monitors. After it is configured this way you will need to manually change these configuration files to switch output to a different GPU, there is no automatic port probing like in Windows.

One way of changing to the other GPU is to modify the /etc/X11/xorg.conf file. Normally that is created during the installation process of the NVIDIA drivers, but you can also manually create it by running sudo nvidia-xconfig. In that file you will see a Device section which, in your case, might already contain a BusID entry with a Hex number like this:

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BusID          "PCI:1:0:0"

If there is no BusID you will need to add it. To find out what BusID each of your GPUs has, check the output of lspci:

lspci | grep VGA

This will list the PCI BusIDs for your GPUs. Then replace the BusID in the xorg.conf file with the GPU you would like to use as Display output.

To properly understand configurations you should have a look at the xorg.conf man pages, specifically the DEVICE section.

There are more complex setups possible where you can assign specific Screens (Displays) to specific GPUs.

I hope this helps!