RTX 5070 Ti Linux 590.48.01: Multi-monitor DSC failure with 3 high-res monitors, one random monitor no signal, workaround enable_overlay_layers=N

Graphics / Linux Linux
1

Without any nvidia kernel module parameters, at the full resolution of my displays, one random monitor has no signal and turns off after modesetting.

Failing monitor configuration, one random of the 3 fails to have signal:

  1. 7680x2160@60Hz, Odyssey G9 57” DisplayPort 2.1
  2. 7680x2160@60Hz, Odyssey G9 57” DisplayPort 2.1
  3. 1920x1080@60Hz, TCL TV HDMI 2.1

Working monitor configuration for all 3

  1. 5120x1440@120Hz, Odyssey G9 DisplayPort 2.1
  2. 5120x1440@120Hz, Odyssey G9 DisplayPort 2.1
  3. 3840x2160@144Hz, TCL TV HDMI 2.1

They all work at their maximum resolution and refresh rates after setting the following nvidia_modset kernel module parameter:

noname@devbox ~ $ cat /etc/modprobe.d/custom.conf
options nvidia-modeset debug=4 enable_overlay_layers=N

So I am filing a bug report because the GPU does not work at the advertised maximum resolutions and requires kernel module modifications. In addition there are lots of module errors and warnings.

I’ve provided the journalctl log part for how nvidia-modeset behaves when switching from a lower working resolution to the maximum resolution without kernel parameters

journalctl-7680x2160-failing.log (8.6 KB)

I’ve also uploaded the full journalctl where I switched from working to non-working several times, manifesting many errors like “AuxChCtl Failing, if a device is connected you shouldn’t be seeing this” or “Got short pulse after an unplug”, as:

journalctl.log (253.6 KB)

I’ve also uploaded nvidia-bug-report.log.gz generated after switching from working to non-working a few times without kernel module parameters.

nvidia-bug-report.log.gz (1.1 MB)

Thank you for the help.

2

If you have DSC enabled on the two G9s it’s somewhat expected that a third screen will not work as NVIDIA’s GPUs treat DSC as the equivalent of two monitors.

What happens if you make sure none of them use DSC by limiting to SDR, 8bpc and avoiding Game Mode?

3

Thank you for the reply but we would like an official response from NVIDIA since large monitors are becoming standard these days. I should add that it works in Windows without kernel module modifications or issues, it’s unclear if windows disables overlay layers automatically or something.

And it should be within specification of the outputs, especially in my tests with:

  • 60Hz or even120Hz since it’s not 8K. (specification is 8K 165Hz)
  • 8-bit colors (specification is 12-bit)
  • no HDR (Specification is 8K 165Hz 12-bit HDR)

This all indicates that it’s a linux module bug.

My preferred configuration that works fine in Windows or if I set enable_overlay_layers=N in Linux is:

  1. 7680x2160@120Hz, Odyssey G9
  2. 7680x2160@120Hz, Odyssey G9
  3. 3840x2160@144Hz, TCL TV

This is the equivalent of 5 4K monitors since 7680x2160 is dual 4K.

4

Ah, hadn’t realised you’d already tested on Windows. Best of luck!

If, by any chance, you don’t get an official response on here, it may be worth cross-posting a bug report on NVIDIA Open GPU Kernel Modules - Issues (GitHub) as an extra way to gain visibility, since 50-series with 590.xx drivers will definitely be using said module and its associated GSP firmware and you’ve already confirmed things work as expected with proprietary drivers on Windows.