[BUG] Extremely slow system when running FHD@120Hz refresh rate, but Nouveau drivers work fine

I spent the past few days trying to set a 1920x1080@120Hz resolution on proprietary Nvidia drivers.
The mode is in EDID, the exact same modeline (the exact timings and clocks) works with Nouveau drivers and in Windows.

I can apply the modeline but it just freezes for a few seconds, my mouse disappears, the system runs extremly slow.

I run Arch Linux with the 390.138 drivers using dkms, installed from the aur.

Here is the log: journalctl -b-1 output after freeze reboot

htop is reporting a 100% usage on only one CPU core on the X server process, so it is not a performance issue.
I could ssh into the PC that is how I got that info.

About 5 minutes after I tried to switch to a bspwm desktop and launch a terminal, it actually switched and launched the terminal, I could type into the terminal.
So I tried moving the mouse just to see if it appears or something happens, and it froze again.

If I work very slowly and only with the keyboard, after like half an hour I finally get to a firefox window, and it indeed looks like 120Hz when scrolling.
But when I switch windows or desktops it just basically freezes for a few minutes. It is unusable, but at least we know the refresh rate can work with proprietary nvidia drivers.

No errors in Xorg log started with verbose log mode.

Any help appreciated, thank You.