Hi, I looked around quite a while but my problem is obviously rare:
On a system attached to two IB fabrics (“subnets”) via mlx5_0 and mlx5_1, resp., how can I reach un-managed switches in the Fabric mlx5_1 is connected to by mlxconfig?
I can get the switches and other devices of that secondary fabric by
$ mst ib add mlx5_1
into the /dev/mst directory, but mlxconfig appears to not find these in the IB network:
$ mst ib add mlx5_1
$ mst status |grep Quantum
/dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0029,mlx5_1,
/dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x002A,mlx5_1,
/dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0064,mlx5_1,
/dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0092,mlx5_1,
/dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0116,mlx5_1,
/dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x011C,mlx5_1,
$ mlxconfig -d /dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0064 q SPLIT_MODE
ibwarn: [12274] _do_madrpc: recv failed: Connection timed out
ibwarn: [12274] mad_rpc_rmpp: _do_madrpc failed; dport (Lid 100)
ibwarn: [12274] _do_madrpc: recv failed: Connection timed out
ibwarn: [12274] mad_rpc_rmpp: _do_madrpc failed; dport (Lid 100)
FATAL - crspace read (0xf0014) failed: Success
-E- Failed to identify the device
$ mlxconfig -d /dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0064,mlx5_1 q SPLIT_MODE
ibwarn: [18228] _do_madrpc: recv failed: Connection timed out
ibwarn: [18228] mad_rpc_rmpp: _do_madrpc failed; dport (Lid 100)
FATAL - crspace read (0xf0014) failed: Success
-E- Failed to identify the device
$ mlxconfig -d /dev/mst/SW_MT54000_Quantum_Mellanox_Technologies_lid-0x0064,mlx5_1, q SPLIT_MODE
ibwarn: [18476] _do_madrpc: recv failed: Connection timed out
ibwarn: [18476] mad_rpc_rmpp: _do_madrpc failed; dport (Lid 100)
FATAL - crspace read (0xf0014) failed: Success
-E- Failed to identify the device
mlxconfig has no option/command-line switch to determine on which fabric it should look for the specified LID …
Acessing switches in the default (first) fabric behind mlx5_0 works ok.
Uwe