Hi I am using mellanox connect-x5 nic card and connect to IB-switch of exadata.
But connected only IB-SDR(10G) .
We try to change port speed in IB switch of commands, but port down…
switch commands ( no change IB card auto-negotiation state in RHEL OS )
ibportstate 1 22 speed 4
ibportstate 1 22 reset
Please help me, how can i complete this problem.
Additional information below it.
Server
HPE DL380 gen10
OS
REHL7.6
Mellanox card
HPE InfiniBand FDR/Ethernet 40/50Gb 2-port 547FLR-QSFP Adapter
IB Switch
Sun Datacenter Infiniband switch 36 (Support speed SDR,DDR,QDR) (Exadata system ib switch)
CARD INFO
/]# ibstat
hca_id: mlx5_0
transport: InfiniBand (0)
fw_ver: 16.26.1040
node_guid: b883:03ff:ff64:28e4
sys_image_guid: b883:03ff:ff64:28e4
vendor_id: 0x02c9
vendor_part_id: 4119
hw_ver: 0x0
board_id: HPE0000000022
phys_port_cnt: 1
Device ports:
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 4096 (5)
sm_lid: 1
port_lid: 14
port_lmc: 0x00
link_layer: InfiniBand
ibportstate 14 1
CA/RT PortInfo:
Port info: Lid 14 port 1
LinkState:…Active
PhysLinkState:…LinkUp
Lid:…14
SMLid:…1
LMC:…0
LinkWidthSupported:…1X or 4X
LinkWidthEnabled:…1X or 4X
LinkWidthActive:…4X
LinkSpeedSupported:…2.5 Gbps or 5.0 Gbps or 10.0 Gbps
LinkSpeedEnabled:…2.5 Gbps or 10.0 Gbps
LinkSpeedActive:…2.5 Gbps
LinkSpeedExtSupported:…14.0625 Gbps
LinkSpeedExtEnabled:…14.0625 Gbps
LinkSpeedExtActive:…No Extended Speed
Mkey:…
MkeyLeasePeriod:…60
ProtectBits:…0
MLNX ext Port info: Lid 14 port 1
StateChangeEnable:…0x00
LinkSpeedSupported:…0x01
LinkSpeedEnabled:…0x01
LinkSpeedActive:…0x00
OOOSLMask:…0x0000
#mlxlink -d 5d:00.0
…
State : Active
Speed : IB-SDR
width : 4X
Supported Info
Enabled Link Speed : 0x00000007 (QDR,DDR,SDR)
Supported Cable Speed : 0x0000000f (FDR10,QDR,DDR,SDR)
Hello Gram,
There are a few things you can try to do to to get QDR speeds.
1.make sure QDR speeds are enabled on the switch by typing interface
ib <interface#> speed sdr ddr qdr fdr10 fdr force
configure write (to save the changes)
- On a server try using ibportstate sdr espeed 40
More information on ibportstate can be found here: https://linux.die.net/man/8/ibportstate
Hope this helps.
Thanks and regards,
~Mellanox Technical Support