P2P transfers inside a VM

Accelerated Computing CUDA CUDA Programming and Performance
1

I’m using 8x RTX A6000 cards in a 2 CPU Gigabyte G482-Z52 server.
On the host, I can do p2p transfers at ~50GB/s with a connectivity matrix of all 1’s, however inside a VM with all GPUs passed through, I can’t do P2P transfers (connectivity matrix all 0’s besides the diagonal). amd_iommu=on and iommu=pt are set as kernel parameters on the host, and IOMMU is enabled in BIOS, is there anything else I need to set to enable P2P transfers inside a VM?

Thanks in advance!

2

The curious thing is that before I was able to do P2P transfers inside a VM with these, I suspect something changed with newer drivers.