VGPU-KVM and CUDA Driver Version mismatch

When I install, cuda is not available anymore and it complains about driver version mismatch.

So which CUDA package I have to install with VGPU KVM driver?

In all you documentation there is no clear statement about that. Maybe you add that to documentation, after you found out.

Following: CUDA Compatibility :: NVIDIA Data Center GPU Driver Documentation you need CUDA 12.2

Guess you never tried that by yourself.

| NVIDIA-SMI 535.54.06 Driver Version: 535.54.06 CUDA Version: N/A |
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
| 0 Tesla P40 Off | 00000000:0A:00.0 Off | Off |
| N/A 36C P0 46W / 250W | 0MiB / 24576MiB | 1% Default |
| | | N/A |

Tried 12.2 and 12.2.1

CUDA is always N/A with VGPU KVM driver.

What exactly are you going to achieve?
Why do you expect CUDA with the host driver? For a VM you should install the guest driver:

This is only for KVM VMs with mdev device(s).

Yes this works as expected.

But I want to run CUDA on host and in containers (lxc/docker) too.

And containers for sure use the kernel module from the host, so it is also impossible to use cuda in containers with the vgpu KVM driver.