Is SN3700 support Auto-Negotiation under 200GE speed mode?

Hi,
Our group has a SN3700 switch which support 200GE speed.
When auto-negotiation is enabled in the 200GE rate mode, the SN3700 fails to set up a link with the peer.
However, the same peer can link up when auto-negotiation is enabled in the 100GE rate mode.

Therefore, does the SN3700 support auto-negotiation at the rate of 200 GE?
By the way, our switch firmware version is:
Version summary: X86_64 3.9.1014 2020-08-05 18:06:58 x86_64

Looking forward to your reply.

Hello,
Yes, the SN3700 supports 200GE ports and will by default auto-negotiate with the highest speed possible. Since the port is coming up at 100GE, I would think either the peer side, or the modules/cable being used, or some configuration is limited at 100G. We would need to know more about the peer switch/port, modules/cable used, port config and see some L1 output to check further.
On a side note, the SN3700C (not SN3700) only supports 100GE.
Regards,
Eric

Hello,
We did several comparative experiments, as shown in the following figure.

  1. NIC1 200G AN - SN3700 200G AN : failed;
  2. NIC2 200G AN - SN3700 200G AN : failed;
  3. switch 2 200G AN - SN3700 200G AN : failed;
    As you can see in the picture:
  4. NIC1 200G AN -NIC2 200G AN : success;
  5. NIC1 200G AN -switch 2 200G AN : success;
  6. NIC2 200G AN - switch 2 200G AN: success;

In the preceding tests, we used the same 200G -1M-copper cable.

Above are some of the experiments we have done, which I think will help to analyze this problem.
Looking forward to receiving your new reply.

Hey There,
Thanks for the information. Can you please help me with the following questions?
AN failed means link is coming up at 100G (not link is down)?
What OS and version is running on the 3700?
What make/model of cable are we using?
May I ask what the Other Switch is (assuming it is not a SN3700)?
Does link come up at 200G if we configure the port speed at 200G (ie. no AN)?
Thanks!
Regards,
Eric

Hi Eric,
Sorry for replying you for so long time.
Here are my answers for your questions:

  1. AN do link down under 200G(it seems SN3700 try to link up with peer again and again);
  2. OS and version: X86_64 3.9.1014 2020-08-05 18:06:58 x86_64;
  3. cable model: 04051357_NDAAFJ-H205_Amphenol-5m-cable;
  4. Other Switch is do not a SN3700;
  5. SN3700 can link come up at 200G if we fixed the port speed at 200G;

And here are other questions for you:

  1. Does SN3700 support 5M-cable under 100G? Is there a compatibility list for reference?
  2. Does 100G can be negotiated to the NOFEC Mode through Auto-Negotiation?
  3. How can I get the Switch Model( e.g. SN3700 or SN3700C) by commands on the Switch Side?
  4. Is there any address that I can get the latest switch firmware?

Looking forward to hearing from you!
Regards,
Mathilda.

Hi Mathilda,
Thanks for your reply. It appears we are running Onyx 3.9.1014 on this switch.
On Amphenol website, I see that all NDAAFJ cables are 100G only, although there is no match for the H205 suffix.
https://www.amphenol-cs.com/catalogsearch/result?q=NDAAFJ

In generally (and specifically for the 200G cables), we would recommend using the NVIDIA/Mellanox cables as they have been tested and qualified. You can check a list of compatible cables here:

For any product or compatibility questions, please reach out to your local NVIDIA Account/Sales team.

To answer your questions:

  1. Yes, this should be supported. Please see interconnect link above.
  2. I believe FEC is auto-negotiated. You will need to explicitly disable FEC (although I would suggest against this for 100G links as raw errors can be expected).
  3. You can run “show system type”, or for the part number, run “fae mlxfwmanager”.
  4. You can download the latest software from https://enterprise-support.nvidia.com/s/downloader
    Switches and Gateways → Switch Software
    Please check the Release Notes and upgrade path before upgrading or contact our Support team.

Thanks.
Regards,
Eric

Hi Eric,
For my Q1, I want to know which type of 5m cable does SN3700 support?
For my Q2,we found that the link can be connected only when the FEC mode is set to RSFEC and Auto-Neg is enabled(100G);
If the FEC mode is set to NONE(which I think is NOFEC?) and Auto-Neg is enabled, the link will be disconnected(100G).
By the way, there is no explicitly choice for FEC mode(only RSFEC or NONE).
Is this a special configuration for the SN3700?

Thanks,
Mathilda.

Hi Mathilda,
Q1. For exactly 5m cables at <100G, you could look at:
MCP2M00-A005E26L (25G) MCP2M00-A0xxEyyz 25GbE SFP28 DAC Cable Product Specifications - NVIDIA Docs
MC2210310-005 (40G) MC2210310-xxx 40GbE QSFP+ Active Optical Cable Product Specifications - NVIDIA Docs
You are best to speak to your NVIDIA Account/Sales team about this and other possible options depending on your requirements.

Q2. There is no special configuration needed (unless we are using an incompatible cable and auto-neg isnt working). If there is no configuration on the interface, auto-neg is enabled by default and this should also auto-neg/enable FEC (where both sides are capable and no mismatch FEC config). It is expected 100G can only come up with FEC (hence my previous caution about raw errors) so it should not negotiate to NONE.

Regards,
Eric

Hi Eric,
Thanks to your warm-hearted reply!
Here my last questiion:
In this PDF. we noticed that firmware version 3.8.1888’s link up time takes 36 seconds.
I want to know 3.9.1014 version’s link up time, Does this link time improve for firmware version 3.9.1014?

Regards,
Mathilda.

Hello Mathilda,
Thanks for your reply.
I had a look over the later features, Release Notes and search our archives, but could not see any further changes to this behaviour (ie. For SN3700/SN3700C at 100 GbE, the link up speed now takes up to 36 seconds ).
Regards,
Eric