sx1012 link issue

system · October 14, 2015, 8:37pm

Hi,

So I have two new sx1012 connected to each other with fiber from two different buildings. They both have the same transceivers:

[standalone: master] # show interfaces ethernet transceiver

Port 1/1 state

identifier : QSFP+

cable/ module type : Optical cable/ module - LR4

ethernet speed and type : 40GBASE - LR4

vendor : OPNEXT, INC.

supported cable length : 10000m SMF

part number : TRQ5B20MN-LF100W

revision : X1

serial number : Z13J00069

[standalone: master] # show interfaces ethernet 1/12

Eth1/12

Admin state: Enabled

Operational state: Down

Description: N\A

Mac address: e4:1d:2d:39:3b:54

MTU: 1500 bytes(Maximum packet size 1522 bytes)

Flow-control: receive off send off

Actual speed: 40 Gbps

Width reduction mode: disabled

Switchport mode: access

MAC learning mode: Enabled

Last clearing of “show interface” counters : Never

60 seconds ingress rate: 0 bits/sec, 0 bytes/sec, 0 packets/sec

60 seconds egress rate: 0 bits/sec, 0 bytes/sec, 0 packets/sec

Rx

0 packets

0 unicast packets

0 multicast packets

0 broadcast packets

0 bytes

0 error packets

0 discard packets

Tx

0 packets

0 unicast packets

0 multicast packets

0 broadcast packets

0 bytes

0 discard packets

We have measured the optical power levels on the fibers and check out fine but I don’t get a connection. The port “operational state” shows as “Down” and the LED light doesn’t come on. I’m wondering if that transceiver is compatible even though it recognizes it?

Any help or suggestions would be greatly appreciate to troubleshoot that issue.

Thank you,

Daniel

rdarbha · October 15, 2015, 12:01pm

Ophir and Anand,

the fiber is 7918 feet long.

Daniel

rdarbha · October 15, 2015, 5:55pm

Ophir,

So, if all my ports are already ethernet and there is no specific settings that I need to configure, on my ethernet ports connected point to point with fiber, for me to see some traffic between them that falls back to some issue with the fiber or transceiver that don’t seem to be supported by the sx1012.

Does anyone know about any third party vendors (finisar FTLX1471D3BCL Duplex SFP with Mellanox adapter) that would work with the sx1012?

Thank you,

Daniel

rdarbha · October 15, 2015, 2:20am

Hi Anand,

If use 1012, which ports support Mellanox LR4 module? Thanks a lot.

ophirm · October 15, 2015, 4:52pm

All ports are ethernet,

this command is applicable only for Gateway systems (you select which port is Ethernet and which ports are infiniBand)

Ophir.

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

ophirm · October 14, 2015, 10:45pm

what is the length of the fiber?

anand6 · October 15, 2015, 2:09pm

Wei

We dont support third party optics for LR4 specifically. The Finisar module may or may not work.

anand6 · October 14, 2015, 11:03pm

We dont support third party LR4 or SR4 modules. That doesn’t mean we do anything to prevent it from working but it may not work. If you provide us the length of the fiber in between, we can suggest a Mellanox optical module.

rdarbha · October 15, 2015, 2:11am

Hi Anand,

Here is the link for LR4 support:

LR4 Support on Mellanox Switches https://community.mellanox.com/s/article/lr4-support-on-mellanox-switches

It says:

Here is the list of Mellanox switches and ports that supports LR4 module (Long Reach - QSFP), OPN: MC2210511-LR4 up to 10Km:

SX1036 use ports 1,3,33,35 only.
SX1024 use ports 50,52,54,56,58,60 only.
SX1012 all ports
SX1710 all ports
TX6000/6100 all long-haul ports

Can the Switch support Finisar LR4 QSFP module?

colinm · November 17, 2015, 3:53pm

Situation: Link is not UP on each end, when port speed is 40G Ethernet.

Two SX1012 switches connected by ~2500 meters of single mode fiber. Transcievers are QSFP+ LR4 40GE OPNEXT. Models:

TRQ5B20MN-LF069

TRQ5B20MN-LF100

These transceivers support this ~2500 meter cable length.

Light readings verified and acceptable on both ends of the cable. They are within proper range, not saturating (too hot) and not below sensitivity threshold (too weak).

Switch software is latest version - 3.4.3002.

Switch configurations are proper.

Root cause of link issue: The OPNEXT transceivers aren’t working, at 40GE, in these Mellanox switches, with this fiber length.

Fix:

Mellanox recommends using this transceiver:

Module QSFP MC2210511-LR4 Active Optical Module 40GigE+FDR10

10km SM

wavelengths: 1270, 1290, 1310 and 1330nm,

Standard MSA Singlemode Transceiver for 10Gigabit Ethernet and 40Gigabit Ethernet with LC connector.

Use with MetroX MTX6XX0 long haul platforms, SX1012, SX1710, SX1036/1700 ports 1,3,33,35 and SX1024 ports 50,52,54,56,58,60.

Mellanox guaranties this module functionality only in a Mellanox End-to-End solution.

Mellanox doesn’t officially support third party LR4 or SR4 modules, though third party transceivers may or may not work in various solutions.

Note: Due to a hardware limitation, SwitchX-2 units do support LR-4 modules, but SwitchX models do not. These switches referenced in this discussion are in fact SwitchX-2 models.

Additionally, LR4 modules will be supported by ConnectX-4 adapters, and by the Spectrum family of switches (starting next software release with ETA December 2015).

anand6 · October 15, 2015, 2:10pm

See the post above. All ports on an SX1012 support a Mellanox LR4 module.

rdarbha · October 15, 2015, 4:03pm

Addition question about that connection: Is there some mode that I need to configure on the port to be aware that it is a Ethernet connection and not IB?

Thank you,

Daniel

ophirm · October 15, 2015, 4:05pm

You need to configure the port type to be ethernet or InfiniBand.

See here, step 7.

HowTo Configure InfiniBand Gateway HA (Proxy ARP) https://community.mellanox.com/s/article/howto-configure-infiniband-gateway-ha--proxy-arp-x

rdarbha · October 15, 2015, 4:43pm

Ophir,

That command from step 7 “show ports type” or “port…” is not recognized on my sx1012. I ran some of the other ones that are mentioned in that link that work:

[standalone: master] (config) # show system capabilities

Ethernet: Supported, Full L2, L3

Ethernet Max licensed speed: 40Gb

[standalone: master] (config) # show system profile

eth-single-switch

[standalone: master] (config) # show port

% Unrecognized command “port”.

Type “show ?” for help.

[standalone: master] (config) # show ports

% Unrecognized command “ports”.

Type “show ?” for help.

The only option that ‘port’ gives me is “load-balance”

tuxnoel · February 4, 2020, 11:21am

to unlock non-Mellanox SFPs use command like:

fae cable-stamping-unlock eth_sfp_25g