XCP-NG Ubuntu VM error Quadro P400

Hello I currently have a problem having a Quadro P400 card work in a Ubuntu 20.04 VM.
I would like to refer to this forum where I am stating all the current findings regarding this issue.

I have this issue for quite some time already and I have no idea on how to resolve this. On Proxmox this works flawlessly but on XCP-NG it does not…

This is the dmesg error I am getting:

[ 25.789595] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 25.789622] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 25.794920] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 25.794943] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 38.508573] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 38.508614] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 38.514513] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 38.514542] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 1254.128346] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 1254.128378] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 1254.134808] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 1254.134834] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 9042.371773] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 9042.371831] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 9042.380194] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x22:0x56:667)
[ 9042.380241] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0

Neither can I find anything about this on the interwebs…

I guess xcp-ng is not a supported hypervisor for passthrough by the nvidia driver. Since it also seems to lack the possibility to hide the hypervisor from the vm, I suspect this won’t work.

Aha…
Well I just bought a Quadro M4000 to see if that works, and it did on one VM but not on the other. On the other VM I am getting this:

[ 51.736135] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x25:0x40:1250)
[ 51.736192] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0
[ 52.723450] NVRM: GPU 0000:00:06.0: GPU has fallen off the bus.
[ 52.925616] NVRM: A GPU crash dump has been created. If possible, please run
NVRM: nvidia-bug-report.sh as root to collect this data before
NVRM: the NVIDIA kernel module is unloaded.
[ 53.128444] NVRM: GPU 0000:00:06.0: RmInitAdapter failed! (0x24:0xffff:1220)
[ 53.128499] NVRM: GPU 0000:00:06.0: rm_init_adapter failed, device minor number 0

It fell of the bus, so I am wondering in this case what it means as that is the only thing that is not 100% clear to me. Could this be an power issue? If so then the 2x molex to 6 pin converter could be broken… which I just bought as well…