How does the mlx5 driver use the IOVA generated by IOMMU as DMA address?

I have enabled IOMMU on the physical machine. I expect the RDMA NIC to use the IOVA allocated by the IOMMU module for DMA after enabling IOMMU. However, in reality, the RDMA NIC does not use the IOVA for DMA

I found through reading the kernel source code that ib_dma_map_sgtable_attrs() is called in ib_umem_get to obtain the DMA address for each scatterlist (sg) entry. During the process of obtaining the DMA address, it first checks whether dev->ops is empty, and then decides whether to assign the DMA address to IOVA. Currently, it was debugged that dev->ops is empty at this point, and the DMA address is not assigned to IOVA.

Does the mlx5 driver perform any other actions (such as assigning dev->ops) when obtaining the struct device *dev to be able to use IOMMU?

RDMA not good support on IOMMU so far. Bugs and performance issue on each vender CPU.

The current RDMA driver cannot use the IOVA provided by IOMMU for DMA?

if you run with IOMMU need add iommu=pt

I have added iommu=pt intel_iommu=on and I have ensured that the physical machine has enabled IOMMU. However, it seems that the RDMA driver is still not using the IOVA provided by IOMMU for DMA (just like I described above). I don’t know how to make the RDMA driver use IOVA for DMA successfully…If anyone knows, please tell me.