Limited and full members on default partition pkey cannot communicate?

I’m configuring a very simple network with two connectx-2 hosts and an unmanaged switch.

If I set the default partition in partitions.conf as “Default=0x7fff, ipoib : ALL=full,SELF=full;”

everything works, but if I set ALL to limited I can no longer do IP pings or ibpings with the two hosts!

What am I doing wrong?

“cat /sys/class/infiniband/mlx4_0/ports/1/pkeys/* |grep -v 0000” shows “0xffff” in the host running opensm and “0x7fff” on the client host.

ibhosts shows both nodes in both servers:

Ca : 0x0002c9xxxxxxxxxxx ports 2 “vps HCA-1”

Ca : 0x0002c9xxxxxxxxxxx ports 2 “srv0 HCA-1”

Just noticed the opensm log gets flooded with the following message when trying to ping the limited “vps” from the full “srv0”:

Mar 13 10:35:54 375510 [3649700] 0x01 → log_trap_info: Received Generic Notice type:2 num:259 (Bad P_Key (switch external port)) Producer:2 (Switch) from LID:2 TID:0x0000000000000138

Hi,

I see that you are using old products of Mellanox (ConnectX and ConnectX-2)

For further information, please refer :

https://docs.mellanox.com/pages/viewpage.action?pageId=12013510

Thanks,

Samer