Guidance on setting up dual GPU setup for compute

My attempts to set up a dual GPU system for inference utilizing dual Nvidia cards, even with just compute, have failed.

Both cards (RTX 4070 Ti 16 GB + RTX 3060 12 GB) work fine individually, but in a dual-GPU setup `nvidia-smi` will only ever show one card at a time. The second card consistently fails to initialize with “RmInitAdapter failed.” Clean OS, multiple boards tested, same result. 3060 VBIOS dump looks suspiciously small (~157 KB vs expected ~512 KB+).

System specs:

- CPU: Intel Core i9-13900

- RAM: 128 GB DDR5

- GPUs: NVIDIA RTX 4070 Ti Super 16 GB (primary), NVIDIA RTX 3060 12 GB (secondary), also tested with K80 in the past

- Motherboards tested: MSI Z790 Carbon Wi-Fi, Gigabyte Z790 Aorus Elite AX

- Storage: Multiple NVMe SSDs, plus 1 TB SATA SSD pulled from an old laptop for clean OS test

- OS: Linux Mint 22 XFCE (Ubuntu 24.04 base), also tested Ubuntu/Mint fresh installs

- Kernel: 6.8.x generic

- Display: Samsung Q90 OLED 65"

- PSU: Adequate for dual-GPU setup

- BIOS: IGFX set as primary display, Above 4G Decoding enabled, ReBAR enabled, PCIe slots forced to Gen4, ASPM disabled

What I’ve tried:

- Fresh OS installs on a completely clean SSD that never had NVIDIA before

- NVIDIA driver installs (mostly version 550 from Ubuntu repos)

- CUDA toolkit installs (12.1 and 12.5 from NVIDIA repos)

- Python 3.10.14 via pyenv and deadsnakes PPA for compatibility with AI repos

- Verified nouveau blacklisting and modeset settings

- Tried slot configurations (x16 + x8, swapped cards between slots)

- Each card works fine **on its own**: if only the 3060 is present, `nvidia-smi` sees it; if only the 4070 Ti is present, `nvidia-smi` sees it

- Problem is specifically **dual-GPU binding**: driver never brings both up at once

PCI devices (from lspci):

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 4070 Ti] [10de:2484]

02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [GeForce RTX 3060] [10de:2504]

Error logs (dmesg/journalctl):

nvidia: module verification failed: signature and/or required key missing - tainting kernel

nvidia-drm[GPU ID 0x0000:02:00.0]: Failed to allocate NvKmsKapiDevice

nvidia-drm[GPU ID 0x0000:02:00.0]: Failed to register device

NVRM: GPU 0000:02:00.0: RmInitAdapter failed!

VBIOS notes:

- Dumped 3060 VBIOS from /sys/bus/pci (Linux method)

- File size was only ~157 KB (normal would be ~512 KB–1 MB)

- Could suggest VBIOS corruption or incomplete ROM

- However, since the 3060 works fine as the only card in the system, the card is not dead — the issue appears only when both GPUs are installed

Summary:

- Both GPUs visible on PCIe bus

- Each works perfectly by itself

- Dual-GPU setup: only one card ever initializes; the other fails with `RmInitAdapter failed`

- Tested clean SSD, clean OS, multiple boards, drivers, toolkits, same result

- Strong suspicion of either a VBIOS/firmware conflict on the 3060 or a resource binding issue with dual-GPU configs. At this point after several months of on and off attempts any advice would help. Thanks in advance. -rio

What’s more is that I’ve only been using Linux for maybe 3 years and I never looked back but I’ve never reached out on a forum for help with the stuff before. I usually just figured it out by exhausting web search but I’m at my wit’s end with his. So before I buy a bigger card or a WS mobo and tossing in the towel, I figured I’d come here as the last resort and maybe get some really good advice. I appreciate you guys.