Infiniband MTU value cannot be changed

Infrastructure & Networking Software And Drivers Mellanox OFED
1

type : CX516A
OS : CentOS 7.6
I have the following configuration and I want to set the MTU value of IB to 65520 but it is not updated.

CONNECTED_MODE described in ifcfg-ib0 is set to YES.
/etc/sysconfig/network-scripts/ifcfg-ib0
CONNECTED_MODE=yes
TYPE=InfiniBand
MTU=65520
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
IPADDR=
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ib0
UUID=b1f0e139-55a4-4df3-b860-78806d60c84d
DEVICE=ib0
ONBOOT=yes

openibd opensmd service is enabled.

When the ibstat command is executed, port1 is displayed as two.

ibstat

CA ‘mlx5_0’
CA type: MT4119
Number of ports: 1
Firmware version: 16.33.1048
Hardware version: 0
Node GUID: 0x1070fd0300242dd0
System image GUID: 0x1070fd0300242dd0
Port 1:
State: Down
Physical state: Disabled
Rate: 40
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x00010000
Port GUID: 0x1270fdfffe242dd0
Link layer: Ethernet
CA ‘mlx5_1’
CA type: MT4119
Number of ports: 1
Firmware version: 16.33.1048
Hardware version: 0
Node GUID: 0x1070fd0300242dd1
System image GUID: 0x1070fd0300242dd0
Port 1:
State: Down
Physical state: Disabled
Rate: 10
Base lid: 65535
LMC: 0
SM lid: 0
Capability mask: 0xa651e84a
Port GUID: 0x1070fd0300242dd1
Link layer: InfiniBand

The IP address can be set, but the MTU value cannot be changed and remains “4092”.

ifconfig

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet <> netmask <> broadcast <>
inet6 fe80::6d9:f5ff:febc:6e26 prefixlen 64 scopeid 0x20
ether 04:d9:f5:bc:6e:26 txqueuelen 1000 (Ethernet)
RX packets 24239 bytes 1982868 (1.8 MiB)
RX errors 0 dropped 1448 overruns 0 frame 0
TX packets 32145 bytes 38036388 (36.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x90120000-9013ffff

enp24s0f0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet <> netmask <> broadcast <>
ether 10:70:fd:24:2d:d0 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ib0: flags=4099<UP,BROADCAST,MULTICAST> mtu 4092
inet <> netmask <> broadcast <>
Infiniband hardware address can be incorrect! Please read BUGS section in ifconfig(8).
infiniband 00:00:0A:77:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 txqueuelen 256 (InfiniBand)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 1000 (Local Loopback)
RX packets 52 bytes 4311 (4.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 52 bytes 4311 (4.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Can you tell us if there are other setting points?

Regards,
Chikara.

2

Correction of card model number.
CX556A-ECAT

3

Hi
In order to set ConnectX-5 card as connected mode, please disable the Enhanced IPoIB
To disable Enhanced IPoIB:

  1. Disable “ipoib_enhanced”:
    #vi /etc/modprobe.d/ib_ipoib.conf

Add the following entry:
options ib_ipoib ipoib_enhanced=0

  1. Stop and start openibd service:
    #/etc/init.d/openibd stop
    #/etc/init.d/openibd start

  2. Verify parameter is disabled: (The value should be 0)
    #cat /sys/module/ib_ipoib/parameters/ipoib_enhanced

  3. Verify the mode:
    #cat /sys/class/net/ib0/mode

4

zhang4-san ,
MTU update is available.
ib0: flags=4099<UP,BROADCAST,MULTICAST> mtu 65520

I appreciate your cooperation,It was very helpful.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.