Error running the Innova-2 flex open application

Infrastructure & Networking Software And Drivers SoC And SmartNIC
1

The application cannot find the ConnectX device. I am not familiar with setting up Mallanox nic card. This Linux system we using already has a mlx5_core driver loaded upon power-up.

Any suggestion would be very helpful.

Here is more detail

Application Error

master001:~/9.MI2/Innova_2_Flex_Open_18_12/app # ./innova2_flex_app -v

===============================================

Verbosity: 1

BOPE device: None

ConnectX device: None

Cannot find appropriate ConnectX device

lspci Command

master001:~/9.MI2/Innova_2_Flex_Open_18_12/app # lspci -v | grep Mellanox

03:00.0 PCI bridge: Mellanox Technologies Device 1974 (prog-if 00 [Normal decode])

04:08.0 PCI bridge: Mellanox Technologies Device 1974 (prog-if 00 [Normal decode])

04:10.0 PCI bridge: Mellanox Technologies Device 1974 (prog-if 00 [Normal decode])

05:00.0 Class 2000: Mellanox Technologies Device 0264

Subsystem: Mellanox Technologies Device 0264

06:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5, PCIe 3.0]

Subsystem: Mellanox Technologies Device 0046

06:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5, PCIe 3.0]

Subsystem: Mellanox Technologies Device 0046

08:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

Subsystem: Mellanox Technologies Device 016c

More Details

master001:~/9.MI2/Innova_2_Flex_Open_18_12 # lspci -vvv -s 06:00.0

06:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5, PCIe 3.0]

Subsystem: Mellanox Technologies Device 0046

Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Interrupt: pin A routed to IRQ 26

Region 0: Memory at c2000000 (64-bit, prefetchable) [size=32M]

Expansion ROM at c7400000 [disabled] [size=1M]

Capabilities: [60] Express (v2) Endpoint, MSI 00

DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited

ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+

DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-

RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-

MaxPayload 256 bytes, MaxReadReq 512 bytes

DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-

LnkCap: Port #0, Speed unknown, Width x16, ASPM not supported, Exit Latency L0s unlimited, L1 <4us

ClockPM- Surprise- LLActRep- BwNot-

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed unknown, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR-, OBFF Not Supported

DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled

LnkCtl2: Target Link Speed: Unknown, EnterCompliance- SpeedDis-

Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-

Compliance De-emphasis: -6dB

LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+

EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-

Capabilities: [48] Vital Product Data

Product Name: Innova-2 Flex Open for Application Acceleration, dual-port SFP28, 25GbE, KU15P, No Crypto, PCI4.0 x8, HHHL, active heat sink, tall bracket, ROHS R6

Read-only fields:

[PN] Part number: MNV303212A-ADLT

[EC] Engineering changes: AA

[V2] Vendor specific: MNV303212A-ADLT

[SN] Serial number: MT2045X09746

[V3] Vendor specific: b8d4ac17c321eb118000043f72e6e896

[VA] Vendor specific: MLX:MODL=NV303212A:MN=MLNX:CSKU=V2:UUID=V3:PCI=V0

[V0] Vendor specific: PCIeGen4 x8

[RV] Reserved: checksum good, 1 byte(s) reserved

End

Capabilities: [9c] MSI-X: Enable- Count=64 Masked-

Vector table: BAR=0 offset=00002000

PBA: BAR=0 offset=00003000

Capabilities: [c0] Vendor Specific Information: Len=18 <?>

Capabilities: [40] Power Management version 3

Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot-,D3cold+)

Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [100 v1] Advanced Error Reporting

UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-

CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-

CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+

AERCap: First Error Pointer: 04, GenCap+ CGenEn- ChkCap+ ChkEn-

Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)

ARICap: MFVC- ACS-, Next Function: 1

ARICtl: MFVC- ACS-, Function Group: 0

Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)

IOVCap: Migration-, Interrupt Message Number: 000

IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+

IOVSta: Migration-

Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00

VF offset: 2, stride: 1, Device ID: 1018

Supported Page Size: 000007ff, System Page Size: 00000001

Region 0: Memory at 0000000000000000 (64-bit, prefetchable)

VF Migration: offset: 00000000, BIR: 0

Capabilities: [1c0 v1] #19

Capabilities: [230 v1] Access Control Services

ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-

ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-

Capabilities: [320 v1] #27

Capabilities: [370 v1] #26

Capabilities: [420 v1] #25

Kernel modules: mlx5_core

2

Hello Amrish,

Thank you for posting your inquiry on the NVIDIA Networking Community.

Based on the information provided, please open a support ticket by sending an email to networking-support@nvidia.com

We will assist you further through the support ticket.

Thank you and regards,

~NVIDIA Networking Technical Support

3

The ConnectX and BOPE devices are not found. Good news is the driver looks to be installed: Kernel modules: mlx5_core

The ConnectX device (mlx5_fpga_tools) is created by running sudo insmod /usr/lib/modules/uname -r/updates/dkms/mlx5_fpga_tools.ko

The BOPE device is created by running sudo ~/Innova_2_Flex_Open_18_12/driver/make_device

Try the following:

sudo mst start
sudo mst status
sudo mst status -v
sudo flint -d /dev/mst/mt4119_pciconf0 q
cd ~/Innova_2_Flex_Open_18_12/driver/
sudo ./make_device
sudo insmod /usr/lib/modules/`uname -r`/updates/dkms/mlx5_fpga_tools.ko
lsmod | grep mlx
cd ~
sudo ~/Innova_2_Flex_Open_18_12/app/innova2_flex_app -v