Hello , we have SX6036 Mellanox IB switch.
Active Image: PPC_M460EX 3.4.2008 2015-06-12 11:48:41 ppc
Dafault Firmware: 9.3.3180
We wish to create 2 separate partitions per 2 HPC clusters connected to one physical IB switch. There are two HPC clusters, one for TEST and second for PROD. The idea is to hide nodes from one cluster from another. We have tried plenty of configuration possibilities but we are always allowed to ibping from one cluster member to another cluster member.
Current partition configuration:
show ib partition
Default
PKey = 0x7FFF
defmember = full
ipoib = yes
members
GUID=‘ALL’ member=‘full’
prod
PKey = 0x7FF3
ipoib = yes
members
GUID=‘94:6d:ae:03:00:49:c1:ac’ member=‘default’
GUID=‘94:6d:ae:03:00:49:c1:b0’ member=‘default’
GUID=‘94:6d:ae:03:00:49:c1:d4’ member=‘default’
GUID=‘94:6d:ae:03:00:49:c2:94’ member=‘default’
GUID=‘94:6d:ae:03:00:4b:a4:7e’ member=‘default’
GUID=‘94:6d:ae:03:00:4b:a4:82’ member=‘default’
GUID=‘94:6d:ae:03:00:4b:a4:86’ member=‘default’
GUID=‘94:6d:ae:03:00:4b:a4:8a’ member=‘default’
GUID=‘94:6d:ae:03:00:4b:a4:8e’ member=‘default’
GUID=‘b8:3f:d2:03:00:69:a5:2a’ member=‘default’
GUID=‘b8:3f:d2:03:00:69:af:1a’ member=‘default’
GUID=‘b8:3f:d2:03:00:69:af:9a’ member=‘default’
GUID=‘e8:eb:d3:03:00:ec:10:c0’ member=‘default’
test
PKey = 0x7FF2
ipoib = yes
members
GUID=‘e4:1d:2d:03:00:11:14:51’ member=‘default’
GUID=‘ec:0d:9a:03:00:15:c3:71’ member=‘default’
We are not sure if “Default” partition should stay, when we have to assign members to “Full”, when to “Limited”. Were the nodes need to be members of Default and dedicated “test” or “prod” or should be a member of only one partition?
The SM is running on IB switch and on one of the nodes per cluster.
Should we also add IB switch GUID to both partitions?
What do the “Default Member” option on partitions level and “Membership” inside the partition (options are: Default, Limited, Full, Both).
Thanks in Advance for your support.
Regards
Tomek