Hi,
I desperately need help.
After a very long wait and many emails, Supermicro suddenly released a new BIOS that allowed for a smooth GPU transition on my Supermicro AS-2015CS server.
I want to achieve desktop acceleration using an Nvidia L40 graphics card. I mainly want to encode Windows RDP by pushing the entire card to a guest host. I don’t want vGPU, just a raw PCI passthrough.
As a hypervisor, I use Proxmox 8.1.4, which is on Debian 12. It looks like I have everything configured correctly, and yet the graphics card drivers display error 10 - This device cannot be started (screen attached).
My configuration is:
- proxmox 8.1.4 with kernel 6.5.11-8-pve
- AMD Epyc processor
- Nvidia L40 graphics card with ROM 95.02.39.00.01
- booting takes place in UEFI mode,
- in /etc/kernel/cmdline I added iommu=pt initcall_blacklist=sysfb_init realloc=off video=vesafb:off video=efifb:off video=simplefb:off nofb nomodeset kvm.ignore_msrs=1 vfio-pci.ids=10de:26b5 , 10de:22ba,10de:169d
- in /etc/modules I placed: vfio vfio_iommu_type1 vfio_pci
- in /etc/modprobe.d/kvm.conf I placed: kvm options ignored_msrs=1
- I made a blacklist in /etc/modprobe.d/blacklist.conf: nouveau nvidia nvidiafb nvidia_drm
I’ve gotten to the point where it looks like GPU Passthrough is almost completely working. Using the NVIDIA display mode selector, I can read and change operating modes in Windows (guest machine) - if I go to compute mode (displaymodeselector.exe --gpumode compute) the card is seen correctly and the driver loads without errors, but I don’t need this mode.
- when switching to graphics mode (displaymodeselector.exe --gpumode graphics), the driver displays error 10. (it does not matter which mode: 256MB/8GB BAR) i get error 10 in windows driver
According to the video - https://youtu.be/5nLhKhnQRjo?si=uMMKlL8dxoGurNez&t=295 (4:55) the gpu can be switched to graphics mode and you need to install RTX drivers - that’s what I did, I installed clean windows, I downloaded the drivers from A6000, it recognized the card as L40, but error number 10 is still there.
the same error also occurs in data center driver.
DataCenter: 538.15-datacenter-tesla-desktop-winserver-2019-2022-dch-international
RTX: RTX_551.52-nvidia-rtx-winserv-2019-2022-64bit-international-dch-whql
According to the instructions in the PDF zip file NVIDIA Display Mode Selector Tool | NVIDIA Developer, you should also change the graphics drivers using the command: C:\Windows\System32>nvidia-smi.exe -fdm 0
and it gives me an error:
NVIDIA-SMI crashed because it couldn’t communicate with the NVIDIA driver. Make sure the latest NVIDIA driver is installed and running. This can also happen if a non-NVIDIA GPU is acting as the primary display and the NVIDIA GPU is in WDDM mode.
What am I doing wrong? why doesn’t the card want to work in RTX mode?
Attached are some screenshots,
dmesg from the host computer returns no errors.
I really count on your help.
more screens: https://cloud.elektrolift.pl/index.php/s/eMSC3AKMkgLTzWY