TL;DR It seems that NVIDIA CPU UEFI Firmware Update Tool fixed the problem for me together with temporary change of PCIe speed to 4.0.
I have no way to confirm if that is the step that did the trick for me, but after being unable to get the card to work with Ubuntu 24.04.3, I installed Windows 11, downloaded NVIDIA drivers and tried to update the firmware with the tool I mentioned. The tool reported “Unsupported GPU” error. The problem did not get fixed.
Then, I ran into this article: RTX 5090 PCIe Compatibility Issues: Complete Fix Guide in 2026?
I changed the PCIe speed in the bios from AUTO to 4.0 wherever option 5.0 was also available. With that change I ran the firmware update tool again, which this time showed that the gpu is supported and did the update successfully.
After that I could revert the change in bios and the gpu works correctly with both windows 11 and ubuntu 24.