ConnectX7 no physical link

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hi,

I am trying to install the Mellanox OFED tools to use the Mellanox Technologies MT2910 Family [ConnectX-7] network card.

I am using the tools provided by NVIDIA:

The tools installed normally and I have two network interfaces on Linux but no physical link. Is this normal ? Is there anything else I need to do to get the card working ?

Here is the log containing information about physical links :

ethtool ibP5p1s0f0
Settings for ibP5p1s0f0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 100000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Other
PHYAD: 0
Transceiver: internal
netlink error: Operation not permitted
Link detected: no

Here are the target specifications :

Hardware : AGX Orin

OS : Ubuntu 22.04

Kernel version : Linux ubuntu 5.15.148-tegra #1 SMP PREEMPT Tue Jan 7 17:14:38 PST 2025 aarch64 aarch64 aarch64 GNU/Linux

Device Type: ConnectX7 Part Number: MCX755106AC-HEA_Ax Description: NVIDIA ConnectX-7 HHHL adapter Card; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe 5.0 x16 with x16 PCIe extension option; Crypto Enabled; Secure Boot Enabled;

PSID: MT_0000001045 Versions: Current Available FW 28.43.3608 28.43.3608 PXE 3.7.0500 N/A UEFI 14.37.0050 N/A

2

Hi

Do you want use IB mode or Ethernet Mode ?

Thanks,

Suo

3

Hi,

I want to use Ethernet mode.

Thanks,

Bruce

4

Could you please provide the output of ibstatus

Thanks,

Suo

5

Here is the result of the ibstatus command:

Infiniband device 'mlx5_0' port 1 status:                                                                                           

    default gid:     fe80:0000:0000:0000:ba3f:d2ff:fefe:f4cc                                                                    

    base lid:        0x0                                                                                                        

    sm lid:          0x0                                                                                                        

    state:           1: DOWN                                                                                                    

    phys state:      3: Disabled                                                                                                

    rate:            40 Gb/sec (4X QDR)                                                                                         

    link_layer:      Ethernet

Infiniband device ‘mlx5_1’ port 1 status:

    default gid:     fe80:0000:0000:0000:ba3f:d2ff:fefe:f4cd                                                                    

    base lid:        0x0                                                                                                        

    sm lid:          0x0                                                                                                        

    state:           4: ACTIVE                                                                                                  

    phys state:      5: LinkUp                                                                                                  

    rate:            100 Gb/sec (4X EDR)                                                                                        

    link_layer:      Ethernet

Infiniband device ‘mlx5_2’ port 1 status:

    default gid:     fe80:0000:0000:0000:cc40:f303:008c:6832                                                                    

    base lid:        0xffff                                                                                                     

    sm lid:          0x0                                                                                                        

    state:           1: DOWN                                                                                                    

    phys state:      2: Polling                                                                                                 

    rate:            10 Gb/sec (4X SDR)                                                                                         

    link_layer:      InfiniBand

Infiniband device ‘mlx5_3’ port 1 status:

    default gid:     fe80:0000:0000:0000:cc40:f303:008c:6833                                                                    

    base lid:        0xffff                                                                                                     

    sm lid:          0x0                                                                                                        

    state:           1: DOWN                                                                                                    

    phys state:      2: Polling                                                                                                 

    rate:            10 Gb/sec (4X SDR)                                                                                         

    link_layer:      InfiniBand

On my machine, I have a ConnectX-5 card and a ConnectX-7 card. I only have this problem with the ConnectX7.