I’m trying to get IPoIB working on a Windows desktop using a ConnectX card, for testing against the FreeBSD driver stack.
The oldest Windows drivers that I can see for download - even under the “Archive” heading - are version 4.60. For IPoIB to work, I’ll probably need something older as the User Guide doesn’t mention the original ConnectX gear.
Any hints?
Hi Justin,
Good to see you back! i’ve asked a colleague to look up for an older version of the driver in the attic… she is working on it ;-)
Of course, it is part of the EOL section of things and therefore we might be limited with knowledge around it
Cheers!
Awesome Viki. The firmware is 2.9.1000 already, so that bit’s ok.
I’ll try the newer WinOF now, and see if that fixes things. Thanks.
Hmmm, for WinOF, I’m only seeing version 5.10 on the download page:
http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers
Am I looking in the wrong place, or was a typo?
Just tried the newer WinOF 5.10 drivers. Still no go. The adapter status for both ports says they have no cable plugged in.
Which is weird, as one port definitely does, and one doesn’t.
ibstat output from the Windows box:
>ibstat
CA ‘ibv_device0’
CA type:
Number of ports: 2
Firmware version: 2.9.1000
Hardware version: 0xa0
Node GUID: 0x0002c9030000cc74
System image GUID: 0x0002c9030000cc77
Port 1:
State: Active
Physical state: LinkUp
Rate: 20
Base lid: 2
LMC: 0
SM lid: 1
Capability mask: 0x90580000
Port GUID: 0x0002c9030000cc75
Link layer: IB
Transport: IB
Port 2:
State: Down
Physical state: Polling
Rate: 10
Base lid: 0
LMC: 0
SM lid: 0
Capability mask: 0x90580000
Port GUID: 0x0002c9030000cc76
Link layer: IB
Transport: IB
That shows one port is up and talking to the fabric, and the other port not. ibnetdiscover from the same box:
>ibnetdiscover
#
# Topology file: generated on Mon Apr 25 15:33:13 2016
#
# Initiated from node 0002c9030000cc74 port 0002c9030000cc75
vendid=0x2c9
devid=0x634a
sysimgguid=0x2c9030000cacf
caguid=0x2c9030000cacc
Ca 2 “H-0002c9030000cacc” # “MT25408 ConnectX Mellanox Technologies”
1 “H-0002c9030000cc74”[1] (2c9030000cc75) # lid 1 lmc 0 “WINPC” lid 2 4xDDR
vendid=0x2c9
devid=0x634a
sysimgguid=0x2c9030000cc77
caguid=0x2c9030000cc74
Ca 2 “H-0002c9030000cc74” # “WINPC”
1 “H-0002c9030000cacc”[1] (2c9030000cacd) # lid 2 lmc 0 “MT25408 ConnectX Mellanox Technologies” lid 1 4xDDR
The setup is a super simple testing one. The Windows PC is connected directly to a FreeBSD box, which is running IPoIB and OpenSM.
Anyone have ideas?
Hi Justin,
What is the Windows version that you are using?
Hi Viki,
Using Windows 8.1 x64.
It’s also possible there’s just something wrong with my current configuration of the adapter. (it’s been ages since I used IB cards in any depth)
So far, I’ve tried both the 4.60 and 4.80 version of WinOF. Both versions see my old ConnectX card, and both apparently load the IPoIB or native ethernet driver (depending on mode I set).
The native ethernet driver “just works”. The IPoIB one though isn’t playing ball. It’s just saying there’s no cable connected, even when there definitely is and I can ibping the exact lid from another box (running OpenSM). So, the IB layer underneath is 100% working.
Also, IPoIB from a different box to the same server running OpenSM works, so it’s not “IPoIB is busted in general”.
The request for an older driver is my groping about in the dark. It might not fix the problem.
Hi Justin,
What is the firmware version of the cards?
If it’s not the latest, I recommend to upgrade to firmware version 2.9.1000 and use the latest WinOF 5.2 version that is the latest version for Windows 8.1.
You can find the proper firmware bin file in the following link:
NVIDIA Networking Firmaware Downloads NVIDIA Networking Firmaware Downloads
We did not test the latest driver with EOL ConnectX cards but we did not stop to support it.
BR,
Viki
Didn’t manage to get this working from Windows in the end, regardless of what host/switch/etc I plugged into. (IPoIB was definitely enabled in the switch partition when I tried that)
Gave up on the windows idea, installed a new HDD, put CentOS 6.x on that, and it just worked out of the box.
It looks like there’s almost definitely something wrong with the WinOF drivers available for download from the Mellanox website, when used with ConnectX cards.