I’ve been looking into it a bit more and it seems to be exactly the same issue sowed in DisplayPort DSC 4K@240Hz flickering artifacts
apparently, he was able to fix it by disabling the “allow flipping” option in the nvidia control panel.
So I tried it in the X session and indeed, it seems to fix the issue. But the X session in laggy and full of tearing. Not to mention some of the newest functions of KDE do not work properly there.
I’ve been looking to see if there was a way to disable that “flipping” option in wayland but found none.