Strange issue with using RTX 3070

I recently decided to move from amd to nvidia and so far it’s been extremely smooth sailing and butter until I found some issue(that is highly untrackable, but easy to reproduce) when you open a native vulkan app(only retroarch and serious sam 4 has done this so far from my testing and serious sam 4 using dxvk and vkd3d are just fine) it makes the entire system freeze for a good minute or so before going smooth again and to end up freezing again in a bit. So I tested with many compositors and desktop environments some have issues with nvidia xorg config file I make it disables gpu hardware acceleration, some work with it, but I think it messes up the x server and makes the app freeze but it does it regardless of that unless the compositor strictly overrides the app itself.

I’ve been trying to track it to no avail, dmesg is clean, running through terminal both steam and retroarch shows absolutely nothing so I have no idea how I can diagnose what is happening if anyone could assist.

System specs:
4x8GB DDR4 2400mhz
Intel Core i7 7700 @ 4.2ghz
MSI Nvidia Geforce RTX 3070 8GB Ventus 2x

Kernel: 5.9.12 zen-kernel
Arch Linux
Running latest nvidia stable driver 455.45.01
EDIT: Running KDE Plasma 5.20
EDIT2: Happens only when they are in borderless or fullscreen.

Please excuse me for the double post. I have tried debugging this and to no avail.
journalctl has absolutely no trace of nvrm or any indication of a crash or anything.
dmesg also shows absolutely nothing.
The freeze CAN BE reproduced with RTX 3xxx cards, BUT NOT with RTX 2xxx or GTX 1xxx(had a few friends to try it out). It happens only on KDE desktop environment, I’ve tried changing to xfce, gnome, the trigger of retroarch vulkan api crash doesn’t happen at all, serious sam 4 using native vulkan api just hangs/freezes for no reason on any desktop environment.
What I noticed about KDE and retroarch is that if I disable Allow Flipping it wouldn’t trigger the freeze IF compositor is disabled, but I can let compositor on top of retroarch and also the freeze is gone. Like I said above happens only with RTX 3xxx cards and not with any other nvidia gpu. It seems to be somehow related to the widely known Quadro GPUs problem and the suggestion to disable Allow Flipping.