Ubuntu 24.04 Desktop VM using llvmpipe for applications instead of nvidia drivers

My Ubuntu 24.04 Desktop VM with nvidia-drivers-590-open, xorg/X11, and xrdp, is using llvmpipe for applications instead of nvidia drivers. nvidia-smi works, the drivers are loaded, and the kernel modules are loaded. Non-graphics based programs can use the GPU such as pytorch, but no graphical applications are using then nvidia drivers to render graphics.