Cable or OpenSM problem?

Infrastructure & Networking Adapters and Cables
1

I bought QSFP+ AOC 40G 850nm AQOA9N13ADLN0723 cable. According to the specification it is InfiniBand 4x DDR/QDR compliant. Unfortunately after

ibportstate --Direct 0 1 enable

I get: state: Down, and Physical state: PortConfigurationTraining or Disabled alternately. After

ibportstate --Direct 0 1 width 1

(on both ends) i get Physical state: LinkUp but State: Initializing. OpenSM is running on one of the nodes (no switch). And in the OpenSM log:

OpenSM 3.3.23

osm_vendor_init: 1000 pending umads specified

Entering DISCOVERING state

osm_vendor_bind: Mgmt class 0x81 binding to port GUID 0x117500006f916e

osm_vendor_bind: Mgmt class 0x03 binding to port GUID 0x117500006f916e

osm_vendor_bind: Mgmt class 0x04 binding to port GUID 0x117500006f916e

osm_vendor_bind: Mgmt class 0x21 binding to port GUID 0x117500006f916e

osm_opensm_bind: Setting IS_SM on port 0x00117500006f916e

SM port is down

SM port is up

log_send_error: ERR 5411: DR SMP Send completed with error (IB_TIMEOUT) – dropping

Method 0x1, Attr 0x11, TID 0x12a8

Received SMP on a 1 hop path: Initial path = 0,1, Return path = 0,0

sm_mad_ctrl_send_err_cb: ERR 3113: MAD completed in error (IB_TIMEOUT): SubnGet(NodeInfo), attr_mod 0x0, TID 0x12a8

sm_mad_ctrl_send_err_cb: ERR 3120: Timeout while getting attribute 0x11 (NodeInfo); Possible mis-set mkey?

Entering MASTER state

Is it cable compatibility or OpenSM problem?

2

Hello Zbuntowany,

Thank you for posting your inquiry on the NVIDIA Networking Community.

Based on the provided cable information, we cannot determine if the cable is working. The cable you are using is not a supported NVIDIA InfiniBand cable, so functionality cannot be confirmed by NVIDIA Networking.

Based on the OpenSM.log, the SM is running properly and it is in Master State so functioning properly.

We recommend to use a supported NVIDIA networking cable. Several cables we have are backwards compatible with QDR.

Thank you and regards,

~NVIDIA Networking Technical Support