Display modes above 2560x1440p@120hz with HDR enabled cause flickering/corruption within gamescope-session

Since the release of 555.42.02, I have been thoroughly testing gamescope compatibility with the nvidia-drm driver and have had a few difficulties. The main issue I’m experiencing has to do with HDR colorspace specifically within embedded gamescope or “gamescope-session”, which is responsible for providing a “Steam Deck” like desktop session for gaming.

As a preliminary point of reference, all of this has been done on gamescope versions past this PR which fixes gamescope-session on nvidia-drm: Disable use of IN_FENCE_FD if it fails by cubanismo · Pull Request #1330 · ValveSoftware/gamescope · GitHub

If I enable HDR within this session and the resolution is anything above 2560x1440p@120hz, I experience severe display corruption as recorded in my clips section below and on this GitHub issue: [NVIDIA] 555 Driver Status · Issue #1320 · ValveSoftware/gamescope · GitHub

It should also be noted that this happens:
On Arch Linux + Fedora
Across both the NVIDIA .run driver installs and packaged versions of the driver for both Arch and Fedora

If I use nested gamescope on my Plasma 6 desktop, HDR works very well with no issues all the way up to higher resolutions of my displays, 3440px1440p@144hz via DisplayPort (175hz seems broken on 555 with my specific monitor but that’s a separate issue) and 3840x2160p@120hz via HDMI 2.1, but trying to achieve the same in embedded gamescope (via TTY) or dedicated gamescope-session has been impossible for me.

Is anyone else experiencing similar issues? This is a very hyperspecific issue topic so I’m not sure how many people have been regularly testing gamescope/gamescope-session on nvidia.

Related media:

nvidia-bug-report.log.gz (747.7 KB)

4 Likes

This issue is still present as of 560.35.03 with both the open and proprietary modules across 3 monitors (only one connected at a time) and both DisplayPort and HDMI 2.0 + 2.1.


the issue on a separate monitor, a G95NC, at 7680x2160p@120hz w/ HDR enabled

1 Like

Yup, can confirm the same issue happens here when starting gamescope on the Nvidia GPU in my laptop.

I’m seeing this on 565.77, single monitor (LG C2).

fwiw this is also being tracked as a gamescope issue after someone reported it separately here: [Nvidia] Black screen when HDR option is enabled · Issue #1593 · ValveSoftware/gamescope · GitHub

this bug still occurs on a new card (5090), new cable (DisplayPort 2.1), and a new monitor (LG 5K2K 45" OLED Gaming Monitor - 45GX950A-B | LG USA). If you toggle HDR within gamescope-session on a NVIDIA gpu, it corrupts your entire display with flickering artifacts:

nvidia-bug-report.log.gz (622.6 KB)

WARNING: FLASHING LIGHTS

That clip was recorded at my monitor’s native resolution of 5120x2160p after toggling HDR from off to on.

Hi @matt-schwartz
I have filed a bug 5240452 internally for tracking purpose.

1 Like

@amrits thank you! I’ve put together a more comprehensive walk-through for this bug report below, including clearer steps for how to reproduce it in a couple different ways.

Current System Information

System:
  Host: blackwell Kernel: 6.14.3-2-cachyos arch: x86_64 bits: 64
  Desktop: KDE Plasma v: 6.3.4 Distro: CachyOS
CPU:
  Info: 8-core model: AMD Ryzen 7 9800X3D bits: 64 type: MT MCP cache:
    L2: 8 MiB
  Speed (MHz): avg: 3834 min/max: 603/5272 cores: 1: 3834 2: 3834 3: 3834
    4: 3834 5: 3834 6: 3834 7: 3834 8: 3834 9: 3834 10: 3834 11: 3834 12: 3834
    13: 3834 14: 3834 15: 3834 16: 3834
Graphics:
  Device-1: NVIDIA GB202 [GeForce RTX 5090] driver: nvidia v: 575.51.02
  Display: wayland server: X.org v: 1.21.1.16 with: Xwayland v: 24.1.6
    compositors: 1: Gamescope 2: kwin_wayland driver:
    gpu: nvidia,nvidia-nvswitch resolution: 5120x2160~165Hz
  API: EGL v: 1.5 drivers: nouveau,nvidia,swrast
    platforms: gbm,wayland,x11,surfaceless,device
  API: OpenGL v: 4.6.0 compat-v: 4.5 vendor: nvidia mesa v: 575.51.02
    renderer: NVIDIA GeForce RTX 5090/PCIe/SSE2
  API: Vulkan v: 1.4.309 drivers: nvidia surfaces: xcb,xlib,wayland
  Info: Tools: api: clinfo, eglinfo, glxinfo, vulkaninfo
    de: kscreen-console,kscreen-doctor gpu: nvidia-settings,nvidia-smi
    wl: wayland-info x11: xdpyinfo, xprop, xrandr

Display: 45" OLED Gaming Monitor - 45GX950A-B. However, the display model or connector should not really matter as long as it supports a high enough resolution (preferably 4K) as well as HDR.

I have reproduced this issue before on a 3070TI laptop when my MUX switch was set to dGPU only (so no iGPU was present on the system at the time), as well as on a 4090 desktop card and now my 5090, so card choice should not matter either.

Steps to reproduce:

The easiest way to reproduce this issue is going to be using gamescope-session, which is a standalone compositor that utilizes Gamescope’s DRM backend without another compositor running. If you’re on Arch Linux, a convenient method of setting this up is by using gamescope-session-git and gamescope-session-steam-git on the AUR. Other distros may also have their own equivelant packages. Once those are installed, you can then log out of your desktop session and log into the Wayland session titled “Steam Big Picture session”.

Once you are logged into the session, here are the steps to reproduce the flickering issue:

  1. Open the Left side menu with Ctrl + 1 or Xbox Guide Button on a controller and go to Settings → Display.
  2. Turn off “Automatically Select Resolution” for your display, and select native resolution.
  3. Open the right side menu with Ctrl + 2 or Xbox Guide Button + A and go to the submenu with the lightning bolt.
  4. Select “Advanced View”, and then flip the toggle for “Enable HDR”

Your display should have started flickering either after Step 2, or after completing Step 4. If it didn’t, try toggling HDR a couple more times.

Alternative method:

You can do this same general process manually via commandline without installing anything except for Steam and gamescope. I would not recommend this because you are unable to switch the resolution that gamescope is rendering at on the fly, but I’ve still been able to reproduce the issue when using this method:

  1. From a standalone TTY console (Ctrl + Alt + F5 for ex.), run gamescope -f -h <monitor native height> -w <monitor native width> -r <display refresh rate> --mangoapp --adaptive-sync -- steam -steamdeck -steamos3 -steampal -gamepadui.
  2. Open the Left side menu with Ctrl + 1 or Xbox Guide Button on a controller and go to Settings → Display.
  3. Scroll all the way to the bottom, and flip the toggle for Enable HDR.

Please feel free to ping me if you run into trouble along the way and I’m happy to assist further.

Additional info:

new nvidia-bug-report.sh from within gamescope-session when it was flickering:

nvidia-bug-report.log.gz (681.5 KB)

I’ve tried every module option for every nvidia module and none have made a difference fwiw.

Seems this is still a issue, except I’m using HDMI2.1

Interesting finding here, if I use Sunshine’s KMS capture from gamescope’s DRM backend, the visual corruption on my display is not visible in the captured image even while the image on my monitor is a flickering mess. Seems like maybe an issue between the display <—> GPU in that case?

Do you have the vk-hdr-layer installed? HDR will not work without it because the Nvidia drivers don’t have the required Vulkan extensions yet.

On another note, have you tried it through the Wine Wayland driver? Does it happen there too?

the vk-hdr-layer interferes with gamescope’s WSI layer so no, I don’t have it installed. HDR works fine other than this in gamescope so I’m not sure where you got the impression HDR will not work without that layer.

I do not use Wine Wayland, I only stick to Valve Proton versions.

Yea, looks like Gamescope uses its own. Sorry don’t use gamescope because it isn’t needed anymore for HDR unless you use it for something else.

I am talking about Proton and using the Wayland driver. If you have for instance Proton-GE or Proton-CachyOS, launch using this

PROTON_ENABLE_WAYLAND=1 PROTON_ENABLE_HDR=1 ENABLE_HDR_WSI=1 %command%

You don’t use gamescope anymore for HDR, but I certainly do because it’s the most stable implementation (and is the compositor I actively work on). I really do not care to try proton/wine wayland until it’s stable and in a Valve Proton version, so this bug report stands.

It is in a Valve’s proton by default even back on Proton 9. You can activate it by unsetting the DISPLAY variable. Anyway, just trying to help out.