You’re running into a bug of the nvidia driver with the display’s firmware resulting in some incorrect EDID.
Please see this thread for a workaround:
https://forums.developer.nvidia.com/t/rtx-2080-super-gigabyte-aorus-fi27q-driver-510-54-maximum-screen-resolution-640x480-via-dp-0/210786/2?u=generix
Basically downgrade to 470, extract edid, upgrade to latest driver then use CustomEDID option.