ATS PRI and PASID capabilities support on ConnectX-7

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

I am working with ConnectX-7, after updating firmware version to 28.37.1014 and doing below step
mlxconfig -d /dev/mst/mt4129_pciconf0 set ATS_ENABLED=1
I see ATS capability listed in lscpi output but I don’t see ATS requests raised by the device, as I do not see PCIe RC PMU(Performance monitor unit) ATS counters getting incremented. Could someone confirm device supports ATS requests.

And, also I do not see PRI and PASID capabilities listed in the device’s config space, should I need to enable something more to support these capabilities?

For your information I am using the latest Kernelv6.3 which has software support to these PCIe capabilities.

Thanks,
Anil

2

Hello,

It seems that you enabled it properly on the adapter, so please check the BIOS configuration or check with the server vendor.

This are the needed commands to enable the ATS:

  1.   mlxconfig -d <PCI ID> set ATS_ENABLED=1

  2.   power cycle / reboot.

to check that its enabled:
3. mlxconfig -d q | grep ATS_ENABLED
you should see this:
· mlxconfig -d /dev/mst/mt4123_pciconf0 q | grep ATS_ENABLED
ATS_ENABLED True(1)
to see it in the config space:
4. lspci -s -vvvxxx | grep ATS

you should see this:

· Capabilities: [480 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable-, Smallest Translation Unit: 00

If all the above is set then you should check the BIOS and kernel.

Best Regards,
Viki

3

Thanks for your reply @vikiz.

After doing steps 1,2.
I see ATS capability got enabled on the card. and have confirmed as suggested.

#mlxconfig -d /dev/mst/mt4129_pciconf0 q | grep ATS_ENABLED
ATS_ENABLED True(1)

#lspci -s 000b:01:00.0 -vvv | grep ATS
Capabilities: [480 v1] Address Translation Service (ATS)
ATSCap: Invalidate Queue Depth: 00
ATSCtl: Enable+, Smallest Translation Unit: 00

After enabling ATS ATSCtl: Enable+, I do not see ATS requests raised by the device, I have confirmed this by monitoring PCIe RC PMU(Performance monitor unit) ATS counters. Kindly please confirm firmware version 28.37.1014 have ATS support.

And even after enabling the ATS capability mentioned above, I do not see PRI and PASID capabilities listed in the device’s config space. Should I need to enable something more? Linux software stack expects the endpoint device capability to be listed in first place so that it will enable and use these capabilities.

Thanks,
Anil

4

Any update on this request?

5

Hi,
I see no further support here, can you tag/assign this ticket to right forum/technical team or guide me how to reach them. We bought couple of ConnectX-7 cards for ATS/PRI capabilities, we would need support to enable and use those capabilities.

Thanks,
Anil