Unbinding gpu from nvidia driver hangs forever

Edit: I think i’ve found the problem, read the edit at the bottom.

Context:

I have 2 nvidia GPUs; 2070 super and 3060. I’m trying to unbind the 3060 from the nvidia drivers and use the vfio drivers on it.

In trying to unload the nvidia drivers, I used the command echo '0000:05:00.0' > /sys/bus/pci/drivers/nvidia/unbind as root. However, this simply hangs with no output and causes 1 thread to pin at 100% (ctrl+c doesn’t work either)

The freeze also happens when both lsof /dev/nvidia0 and lsof /dev/nvidia1 show that nothing is running on them (I stopped display-manager.service to test this)

other info

I’m running with nvidia_drm.modeset=1.
0000:05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [GeForce RTX 3060] [10de:2544] (rev a1)
/dev/nvidia0 is the 3060 (i think)
Nvidia driver version 550.78

on a sidenote, what’s the method I should be using to specify which GPU to run a given application? (for applications using vulkan and opengl)

Edit:

I’ve found this post Can't rebind GPU with 'driverctl' if system booted with GPU attached to nvidia driver - #4 by generix which says that I have to have modeset set to 0, but I’m running on a plasma wayland session which seems to require modeset=1 according to every source (notably Plasma/Wayland/Nvidia - KDE Community Wiki).

Is there no way to have dynamic unbinding of the gpu? Is there something like a method to have modeset set to 0 on the secondary nvidia GPU only?

AFAIK unbinding is not supported when you’re using kernel modeset.