Do all NICs in the Mellanox ConnectX-6 DX portfolio enable Intel's DDIO?

Infrastructure & Networking Software And Drivers Mellanox OFED
, , ,
1

From the product brief, I see only some models offer Socket Direct which help enable DDIO on multi socket and bypass UPI/QPI.

I can’t find if the NIC I have (MCX623106AN-CDAT) supports Intel’s DDIO on the same socket.

Is there a way to toggle this feature on and off?

Thanks,

Anirudh

2

Hi,

You are right, this feature is used only when the card is a socket direct model.

Intel:

In general, Intel DDIO technology can enable NIC to do I/O operation directly to the CPU’s cache, and not to main memory as usually done.

RDMA goal is to do I/O with no CPU intervention (to/from host main memory). By definition cache is HW shared resource, for multiple processes.

Hence using both features can lead to potential vulnerabilities like information disclosure.

Mellanox:

On ConnectX-4Lx/ConnectX-5 we do not allow accessing our internal caches directly from the wire. This means that this vulnerability is not related to our products.

More theoretical info on potential attack:

There are two attacks vector:

  1. Using the NIC RDMA as an attack vector to perform side channel attack on Intel DDIO. This is very possible attack scenario, depends on the application in use.
  2. In this scenario the attack is on the server and not on our product.
  3. Using the NIC RDMA as an attack vector to perform side channel attack on BlueField L3 cache. External software does not have direct access to the ARM memory. The assumption we made that only cloud provider can run the applications, and ARM memory and caches are dedicated to the cloud provider and are not shared with users. In other words, application running on ARM are trusted, managed by a trusteed entity like cloud provider.
  4. This can be mitigated in BlueField3 when L3 cache will probably be per cluster and not shared.

I cannot find any other inforation related to the usage of DDIO by other cards model

Regards

Marc

3

Thanks for the speedy response Marc!

You mention the vulnerability on ConnectX-4Lx/ConnectX-5

Does this apply to the ConnectX-6 DX as well, assuming a non-socket direct model?

Can you point me to a resource which documents this as well?

Regards,

Anirudh