Overclocking more than 1 nvidia GPU problem Ubuntu (PCI x1) Asrock

Hello everybody.

From few weeks I am trying to overclock my Nvidia 3090 gpus. I was installed ubuntu, install drivers etc. Everything (mining) was work stable, but i have only 107Mh per gpus. I tried to make some overclocking but after few hours i was able to overclock only one card (main one).

My configuration:

  • MB: Asrock H110 Pro BTC+
  • GPUS: 5x RTX3090
  • System: Ubuntu 20.04
  • Nvidia Drivers: 470.63.01

One GPU is connected to PCI x16 - screen is connected to this GPU (only this GPU i could overclk), rest is connected to PCI x1. (I was also tryed to connect screen to integrated GPU or others, mining is working but ovclk also do not)

Here’s what i did:

    • Set up coolbits to 26 (i wase used command: ‘nvidia-xconfig --cool-bits=28‘ and also change xorg.conf adding option coolbit to section ‘screen’ an ‘device’ )
  • -Set nvidia-xconfig --enable-all-gpus
  • -Set persistence mode (nvidia-smi -pm 1)
  • -Set PowerMizer mode = 1 via nvidia-settings
  • -Set the "GPUGraphicsClockOffset[3] and GPUMemoryTranserRateOffset[3] (but it was doesnt work – I had to use :

Everything is working only for GPU0, for other GPUS nothing happend.

I also run GUI i go to nvidia-x-settings to try change value on graphics mode. I can make it only for GPU0 – on other GPUs this performance level option is anavialable.

I tried to connect screen to other GPU – no changes

What is wrong? Why i cant overclock other GPUS connected to PCI x1?

My actual xorg.conf:

# nvidia-xconfig: X configuration file generated by nvidia-xconfig

# nvidia-xconfig: version 470.63.01

Section "ServerLayout"

Identifier "Layout0"

Screen 0 "Screen0" 0 0

Screen 1 "Screen1" RightOf "Screen0"

Screen 2 "Screen2" RightOf "Screen1"

Screen 3 "Screen3" RightOf "Screen2"

Screen 4 "Screen4" RightOf "Screen3"

InputDevice "Keyboard0" "CoreKeyboard"

InputDevice "Mouse0" "CorePointer"

EndSection

Section "Files"

EndSection

Section "InputDevice"

# generated from default

Identifier "Mouse0"

Driver "mouse"

Option "Protocol" "auto"

Option "Device" "/dev/psaux"

Option "Emulate3Buttons" "no"

Option "ZAxisMapping" "4 5"

EndSection

Section "InputDevice"

# generated from default

Identifier "Keyboard0"

Driver "kbd"

EndSection

Section "Monitor"

Identifier "Monitor0"

VendorName "Unknown"

ModelName "Unknown"

Option "DPMS"

EndSection

Section "Monitor"

Identifier "Monitor1"

VendorName "Unknown"

ModelName "Unknown"

Option "DPMS"

EndSection

Section "Monitor"

Identifier "Monitor2"

VendorName "Unknown"

ModelName "Unknown"

Option "DPMS"

EndSection

Section "Monitor"

Identifier "Monitor3"

VendorName "Unknown"

ModelName "Unknown"

Option "DPMS"

EndSection

Section "Monitor"

Identifier "Monitor4"

VendorName "Unknown"

ModelName "Unknown"

Option "DPMS"

EndSection

Section "Device"

Identifier "Device0"

Driver "nvidia"

VendorName "NVIDIA Corporation"

BoardName "NVIDIA GeForce RTX 3090"

BusID "PCI:1:0:0"

Option "Coolbits" "28"

EndSection

Section "Device"

Identifier "Device1"

Driver "nvidia"

VendorName "NVIDIA Corporation"

BoardName "NVIDIA GeForce RTX 3090"

BusID "PCI:2:0:0"

Option "Coolbits" "28"

EndSection

Section "Device"

Identifier "Device2"

Driver "nvidia"

VendorName "NVIDIA Corporation"

BoardName "NVIDIA GeForce RTX 3090"

BusID "PCI:3:0:0"

Option "Coolbits" "28"

EndSection

Section "Device"

Identifier "Device3"

Driver "nvidia"

VendorName "NVIDIA Corporation"

BoardName "NVIDIA GeForce RTX 3090"

BusID "PCI:6:0:0"

Option "Coolbits" "28"

EndSection

Section "Device"

Identifier "Device4"

Driver "nvidia"

VendorName "NVIDIA Corporation"

BoardName "NVIDIA GeForce RTX 3090"

BusID "PCI:8:0:0"

Option "Coolbits" "28"

EndSection

Section "Screen"

Identifier "Screen0"

Device "Device0"

Monitor "Monitor0"

DefaultDepth 24

Option "AllowEmptyInitialConfiguration" "True"

Option "ConnectedMonitor" "DFP-0"

Option "Interactive" "False"

Option "Coolbits" "28"

SubSection "Display"

Depth 24

EndSubSection

EndSection

Section "Screen"

Identifier "Screen1"

Device "Device1"

Monitor "Monitor1"

DefaultDepth 24

Option "AllowEmptyInitialConfiguration" "True"

Option "ConnectedMonitor" "DFP-0"

Option "Interactive" "False"

Option "Coolbits" "28"

SubSection "Display"

Depth 24

EndSubSection

EndSection

Section "Screen"

Identifier "Screen2"

Device "Device2"

Monitor "Monitor2"

DefaultDepth 24

Option "AllowEmptyInitialConfiguration" "True"

Option "ConnectedMonitor" "DFP-0"

Option "Interactive" "False"

Option "Coolbits" "28"

SubSection "Display"

Depth 24

EndSubSection

EndSection

Section "Screen"

Identifier "Screen3"

Device "Device3"

Monitor "Monitor3"

DefaultDepth 24

Option "AllowEmptyInitialConfiguration" "True"

Option "ConnectedMonitor" "DFP-0"

Option "Interactive" "False"

Option "Coolbits" "28"

SubSection "Display"

Depth 24

EndSubSection

EndSection

Section "Screen"

Identifier "Screen4"

Device "Device4"

Monitor "Monitor4"

DefaultDepth 24

Option "AllowEmptyInitialConfiguration" "True"

Option "ConnectedMonitor" "DFP-0"

Option "Interactive" "False"

Option "Coolbits" "28"

SubSection "Display"

Depth 24

EndSubSection

EndSection

nvidia-smi:


+-----------------------------------------------------------------------------+

| NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4 |

|-------------------------------+----------------------+----------------------+

| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |

| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |

| | | MIG M. |

|===============================+======================+======================|

| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |

| 30% 29C P8 23W / 350W | 168MiB / 24267MiB | 0% Default |

| | | N/A |

+-------------------------------+----------------------+----------------------+

| 1 NVIDIA GeForce ... Off | 00000000:02:00.0 Off | N/A |

| 30% 27C P8 12W / 350W | 5MiB / 24268MiB | 0% Default |

| | | N/A |

+-------------------------------+----------------------+----------------------+

| 2 NVIDIA GeForce ... Off | 00000000:03:00.0 Off | N/A |

| 0% 26C P8 15W / 350W | 5MiB / 24268MiB | 0% Default |

| | | N/A |

+-------------------------------+----------------------+----------------------+

| 3 NVIDIA GeForce ... Off | 00000000:06:00.0 Off | N/A |

| 30% 26C P8 14W / 350W | 5MiB / 24268MiB | 0% Default |

| | | N/A |

+-------------------------------+----------------------+----------------------+

| 4 NVIDIA GeForce ... Off | 00000000:08:00.0 Off | N/A |

| 0% 24C P8 30W / 390W | 5MiB / 24268MiB | 0% Default |

| | | N/A |

+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+

| Processes: |

| GPU GI CI PID Type Process name GPU Memory |

| ID ID Usage |

|=============================================================================|

| 0 N/A N/A 1335 G /usr/lib/xorg/Xorg 79MiB |

| 0 N/A N/A 1555 G /usr/bin/gnome-shell 86MiB |

| 0 N/A N/A 1864 G /usr/bin/nvidia-settings 0MiB |

| 1 N/A N/A 1335 G /usr/lib/xorg/Xorg 4MiB |

| 2 N/A N/A 1335 G /usr/lib/xorg/Xorg 4MiB |

| 3 N/A N/A 1335 G /usr/lib/xorg/Xorg 4MiB |

| 4 N/A N/A 1335 G /usr/lib/xorg/Xorg 4MiB |

+-----------------------------------------------------------------------------+

Blockquote

Thanks in advance for help.

Hi jakub9,
I wish I had a solution. I am having the same problem. Four card, can only OC the first one. I can, however, control fan speeds on all 4, but no Cor clock or Mem speed changes. No error code, they just don’t register. I tried a previous driver, and both by using apt in ubuntu and also by downloading directly from nvidia but same results.

Have had other boxes with ubuntu as well, working fine for several cards. So, if I find the solutions I will post here.

Solved it on my end. I had been trying several BIOS settings on my Gigabyte H410M H and could not get more than gpu 0 to react to OC’ing. Then I decided to reset ALL BIOS settings to default, and it worked. I don;t know what caused it but it’s ok now, with same version of nvidia drivers and ubuntu as you.

Shut down to install one more fan. On boot, can NO longer OC GPU’s other than the first one (I have four in this rig). Suspecting mobo HW problem.

—Reinstalling Ubuntu, then several hours testing, then finally and CSM Support OFF in BIOS fixed it.