Issue with pci passthrough in ESXi and with ubuntu

We try to passtrhough a P620 nvidia gpu in an ubuntu 20.04 VM
The GPU is already listed as PCI device, but the driver seems to have a problem with the passtrough
the log e I get is

Jan 10 14:39:06 hostname /usr/lib/gdm3/gdm-x-session[1318]: (EE) NVIDIA(GPU-0): The NVIDIA GPU at PCI:11:0:0 is not supported by the 470.94
Jan 10 14:39:06 hostname /usr/lib/gdm3/gdm-x-session[1318]: (EE) NVIDIA(GPU-0): NVIDIA driver.
Jan 10 14:39:06 hostname /usr/lib/gdm3/gdm-x-session[1318]: (EE) NVIDIA(GPU-0): Failed to initialize the NVIDIA graphics device!
Jan 10 14:39:06 hostname /usr/lib/gdm3/gdm-x-session[1318]: (EE) NVIDIA(0): Failing initialization of X screen
Jan 10 14:39:06 hostname /usr/lib/gdm3/gdm-x-session[1318]: (II) UnloadModule: “nvidia”

interestingly enough, the mentioned PCI device is the passthough device, not the GPU

00:11.0 PCI bridge: VMware PCI bridge (rev 02)
0b:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P620] (rev a1)

via dmesg you see this:

[ 69.248302] NVRM: GPU 0000:0b:00.0: RmInitAdapter failed! (0x22:0x56:667)
[ 69.248356] NVRM: GPU 0000:0b:00.0: rm_init_adapter failed, device minor number 0

Because of this article *Bug* 470.42.01-1 dGPU can not be initialized - #12 by entropie1983 I installed the driver manual, but this did not help

Xorg uses decimal values. 0xb=11dec
Try this:

thanks for your answer, unfortunally it does not change anything regarding the errors :(

Under normal circumstances, the parameter is not needed since you have a quadro.
I’ve seen the 470.94 driver failing in a pass-through scenario before, please check if an ealier (or later) driver works.

I tried all the drivers shown by device drivers and also manually 470 and 495 (as recommended by the nvidia site), but is does not help
I also added bug report for deb 470 and the manual installers
nvidia-bug-report-470-manual.log.gz (80.8 KB)
nvidia-bug-report-495-manual.log.gz (85.0 KB)
nvidia-bug-report-495-apt.log.gz (108.8 KB)