How to configure Partition Key (PKey) to isolate different tenants' traffic on Mellanox M8790 switches in an IB network?

Hello ,The current IB network environment consists of 2 M8700 switches and 10 M8790 switches, forming 4 spines and 8 leaf networks. Is there any official documentation or configuration examples for Partition Key that I can refer to for learning?

I have made the relevant configurations, but the isolation is not working as expected.

cat /etc/opensm/partitions.conf

Storage=0x8020,indx0,ipoib,defmember=full:0xe41d2d0300cac9f6,0xe41d2d0300cac9f7 ;
hpc=0x8021,indx1,ipoib,defmember=limited:0x946dae03009cc5b4,0x946dae03009cf751 ;

smpquery PkeyTable -D 0

0: 0x8020 0xffff 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
8: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
16: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
24: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
32: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
40: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
48: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
56: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
64: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
72: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
80: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
88: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
96: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
104: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
112: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
120: 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000
128 pkeys capacity for this port

https://docs.nvidia.com/networking/display/mlnxofedv23100550/nvidia+sm#src-2396915000_NVIDIASM-FileFormat

https://docs.nvidia.com/networking/display/mlnxofedv23100550/ip+over+infiniband+(ipoib)#src-2396915069_safe-id-SVBvdmVySW5maW5pQmFuZChJUG9JQiktU3ViLWludGVyZmFjZXM

1 Like

Thank you very much for your assistance. I am now able to configure the SM to achieve isolation between different tenants. I would like to know how to make a host running opensm become the SM in an IB network. Also, if the current SM host in the IB network fails, how can I make another host in the IB network become the SM host? I look forward to your response. Thank you!