Screen freezing after prime-select nvidia

I have an NVIDIA + AMD GPU laptop running Ubuntu 22.04. The NVIDIA driver is installed, and nvidia-smi works, but I am encountering issues with PRIME switching and DRM modesetting. Specifically, I see repeated errors in dmesg like:
[drm:nv_drm_master_set [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000100] Failed to grab modeset ownership
The system seems to struggle with modesetting, and I suspect a conflict between the AMDGPU and NVIDIA drivers.

Here are what I have tried

  1. prime select to nvidia
  2. Enabled DRM modeset by adding:
    GRUB_CMDLINE_LINUX_DEFAULT="nvidia-drm.modeset=1 nvidia-drm.fbdev=1"

I would like to use nvidia mainly in my ubuntu to run my program, but after prime select my screen become freezing very often. So that’s the main problem that I want to fix.

I’m using NVIDIA GeForce RTX 3050 Ti and processor is Ryzen9 59000HX on Asus ROG strix G15
The driver that I installed is 535.183.01, I installed it from sudo apt install nvidia-driver-535

Thank you for your help!