Can't use nvidia-smi on VM

I’m using a VM with Ubuntu 22.04 and trying to do GPU pass-through.
The drivers are detected and in use however when I use the command

nvidia-smi

I get the following output

Failed to initialize NVML: Driver/library version mismatch

With the error

NVRM: API mismatch: the client has the version 520.61.05, but
               NVRM: this kernel module has the version 470.161.03.  Please
               NVRM: make sure that this kernel module and all NVIDIA driver
               NVRM: components have the same version.

However, it appears I only have 470 nvidia-drivers. What is going on? Everytime I purge the nvidia driver, it appears some random version again. Am I missing something here?

PS: I need max nvidia-driver-470 since I’m using a legacy graphic card GT 740M

dpkg -l | grep nvidia
ii  libnvidia-cfg1-470:amd64              470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA binary OpenGL/GLX configuration library
ii  libnvidia-common-470                  470.161.03-0ubuntu0.22.04.1             all          Shared files used by the NVIDIA libraries
rc  libnvidia-compute-418-server:amd64    418.226.00-0ubuntu5~0.22.04.1           amd64        NVIDIA libcompute package
ii  libnvidia-compute-470:amd64           470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA libcompute package
ii  libnvidia-decode-470:amd64            470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-egl-wayland1:amd64          1:1.1.9-1.1                             amd64        Wayland EGL External Platform library -- shared library
ii  libnvidia-encode-470:amd64            470.161.03-0ubuntu0.22.04.1             amd64        NVENC Video Encoding runtime library
ii  libnvidia-extra-470:amd64             470.161.03-0ubuntu0.22.04.1             amd64        Extra libraries for the NVIDIA driver
ii  libnvidia-fbc1-470:amd64              470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-gl-470:amd64                470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD
ii  libnvidia-ifr1-470:amd64              470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA OpenGL-based Inband Frame Readback runtime library
ii  nvidia-compute-utils-470              470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA compute utilities
ii  nvidia-dkms-470                       470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA DKMS package
ii  nvidia-driver-470                     470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA driver metapackage
ii  nvidia-kernel-common-470              470.161.03-0ubuntu0.22.04.1             amd64        Shared files used with the kernel module
ii  nvidia-kernel-source-470              470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA kernel source package
ii  nvidia-prime                          0.8.17.1                                all          Tools to enable NVIDIA's Prime
ii  nvidia-settings                       510.47.03-0ubuntu1                      amd64        Tool for configuring the NVIDIA graphics driver
ii  nvidia-utils-470                      470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA driver support binaries
ii  screen-resolution-extra               0.18.2                                  all          Extension for the nvidia-settings control panel
ii  xserver-xorg-video-nvidia-470         470.161.03-0ubuntu0.22.04.1             amd64        NVIDIA binary Xorg driver

Additionally, I find some files with this 520 version. Not sure what is going on :/

sudo find /usr/lib -iname "*nvidia*520*"
/usr/lib/x86_64-linux-gnu/libnvidia-compiler.so.520.61.05
/usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.520.61.05
/usr/lib/x86_64-linux-gnu/vdpau/libvdpau_nvidia.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.520.61.05
/usr/lib/x86_64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.520.61.05
/usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.520.61.05
/usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.520.61.05
/usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-encode.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.520.61.05
/usr/lib/x86_64-linux-gnu/libnvidia-tls.so.520.61.05
/usr/lib/libnvidia-gtk3.so.520.61.05
/usr/lib/libnvidia-gtk2.so.520.61.05

Nvidia-smi on the host or guest?

Can’t run nvidia-smi on my vm. Additionally I’m using proxmox on my host but my vm runs on Ubuntu 22.04

Hi I have similar issue described here: RTX 6000 Ada Linux driver crash
Looking for suggestions but also my info might help