VPI Configuration issue on SX6036G - Proxy-ARP does not come up

I try to configure a factory-resetted Mellanox SX6036SX. I need the VPI functionality, therefore I try to setup Proxy-ARP. The software versions are the following:

MLNX-OS 3.6.8010

PPC_M460EX 3.6.8010 2018-08-20 18:04:16 ppc

I configure the switch as described in the manual like the following:

ibsw2 [standalone: master] (config) # system profile vpi-single-switch

switch-6314f4 [standalone: master] (config) # no ip routing

% Unrecognized command “routing”.

Type “no ip ?” for help.

switch-6314f4 [standalone: master] (config) # no ip igmp snooping

switch-6314f4 [standalone: master] (config) # no ib sm

switch-6314f4 [standalone: master] (config) # vlan 100

switch-6314f4 [standalone: master] (config vlan 100) # exit

switch-6314f4 [standalone: master] (config) # ip proxy-arp

switch-6314f4 [standalone: master] (config) # interface proxy-arp 1

switch-6314f4 [standalone: master] (config interface proxy-arp 1) # ip address 10.0.53.199

switch-6314f4 [standalone: master] (config interface proxy-arp 1) # ip netmask /24

switch-6314f4 [standalone: master] (config interface proxy-arp 1) # ip vlan 100

switch-6314f4 [standalone: master] (config interface proxy-arp 1) # ip pkey 0x7fff

switch-6314f4 [standalone: master] (config interface proxy-arp 1) # no shutdown

switch-6314f4 [standalone: master] (config interface proxy-arp 1) # show interfaces proxy-arp 1

After the last command, the configuration of the proxy-arp should be displayed, however, it is not. When looking at the web frontend, the checkbox “IP Proxy-ARP” is checked, however, all other options are greyed out (see screenshot).

I attached the resulting configuration of the system to this message.

Even worse, if I reboot the switch with this configuration, parts of the web interface (System / Ports) do not work (see screenshot). Moreover, all Infiniband ports are dead. If I disable the checkbox “IP Proxy-ARP” and reboot, the system is functional again.

I also had a look at the log (see attached file). In line 8180 one can see the command to enable the Proxy-ARP. Then in line 8258 something seems to go wrong:

Nov 19 19:37:42 switch-6314f4 prad[5352]: TID 1230881936: [prad.NOTICE]: prad_update_virtual_port: Enable swid=2 dev=1 lcl=66 idx=130 gda=false

Nov 19 19:37:42 switch-6314f4 prad[5352]: TID 1230881936: [prad.NOTICE]: prad_exec_enable, IN

Nov 19 19:37:42 switch-6314f4 prad[5352]: TID 1230881936: [prad.NOTICE]: prad_get_port_guid not ready, try 0

[…]

Nov 19 19:29:40 switch-6314f4 prad[5352]: TID 1230881936: [prad.ERR]: prad_get_port_guid(), prad_logic.c:86, build 1: prad_get_port_guid timed out waiting for valid /smad/state/swid_guid_map/2/port_guid

Nov 19 19:29:40 switch-6314f4 prad[5352]: TID 1230881936: [prad.ERR]: prad_exec_enable(), prad_logic.c:1477, build 1: failed to read guid, error is 14031

The log entry “Enable swid=2” seems strange to me: I have only one switch in this configuration?

Does anyone have a clue what’s happening here? Is this a configuration issue, software bug or hardware issue?

Best Regards,

Hermann

log(1).txt (550 KB)

config-6036G-211119.txt (1.2 KB)

This is an unusual condition. I see a generic “internal error”. It seems the software/config got corrupted. My recommendation cold boot by unplugging/replugging power cables. If that does not help, try the procedure below:

Unplug port cables, save the VPI licence (copy and paste to a file for backup - seen with “show licence”), and reset the configuration with “reset factory keep-basic”. Then after it reboots, reinstall the licence, mgmt IP/mask, and reconfigure the switch entirely from the CLI (not web gui). I believe that will work.

Use this link for configuration instructions:

https://support.mellanox.com/s/article/howto-configure-infiniband-gateway-ha--proxy-arp-x

FYI: Once the configuration looks normal and the “show” commands work again, keep in mind the proxy-arp interface, in order to go UP/Active, will need at least one active Eth link, and an IB link to a cluster that has an Subnet Manager running in it.

Thank you for your quick answer - however, it did not help:

  • I restarted the switch (unplug / replug power cables)
  • I did a factory reset as described above

Nothing changed the behaviour of the switch regarding the proxy.

So I can only suspect that this is some kind of firmware issue?

I’d therefore like to try another firmware. However, I found no way to download / get any new or different firmware for this device - does anyone know how to do that?

Best Regards,

Hermann