I picked up a new monitor and cannot run it in its highest mode on my RTX 4060 via DisplayPort. When switching to 3840x2160 @ 240Hz in Xorg, the mode switch occurs and I can see the mouse cursor updating at 240Hz but the desktop is completely hung with the previous image contents.
This happens on Xorg with both Gnome Shell (tested 45.6 & 46.3.1) and KDE Plasma (tested 6.1.3) and eventually the display switches back to the previous mode. On Gnome I then get the “Oh no, something has gone wrong” screen and must logout. On KDE I get a notification that desktop effects were restarted due to graphics reset. If I drop the resolution to 2560x1440 and then set 240Hz it works properly so it seems the culprit is DSC. I also tested both desktops on Wayland and 3840x2160 @ 240Hz works properly so whatever the problem is, it’s specific to setting a DSC mode on Xorg. Are there any known issues with this?
For my drivers, I tested on both 555.58.02 and 560.28.03 but they behave the same. I also tried with/without GSP firmware and compiling with open/closed kernel modules, fbdev=0 & 1, all with no effect. My system is Gentoo Linux running on kernel 6.10.0 (previously 6.9.9). I’m not seeing any errors of note in journald or the kernel logs, except after the mode switch I see a single instance of:
/usr/libexec/gdm-x-session[25924]: (WW) NVIDIA: Wait for channel idle timed out.
On KDE I also see multiple instance of this:
kwin_platform_x11_standalone: Bogus refresh rate -202034
Any advice or ideas to further debug this issue are appreciated.