I have an existing optical network based on SFP+ and I got an adapter to convert from QSFP to SFP+. It seems like something that should be possible, but having tried it, the DGX Spark does not even detect that there is anything connected to that port. I’ve tried both a DAC cable and standard optical transceivers but couldn’t get either working. Is there some kind of setting that would help, or is this something that can’t be done at all?
In the mean time I’ve gone out and purchased a 10G media converter that will take an optical connection and convert it to 10GBE ethernet, so that will at least get me connected to my 10G network, but it’s annoying not being able to connect straight into my optical network and I don’t understand why this wouldn’t be possible.
I’m using a very basic adapter (one of the only ones I could find, but others using these and similar adapters seem to have success with various devices, for the cable I tried various transceivers and a DAC cable, couldn’t find an option that works with this adapter): https://www.aliexpress.com/item/1005008512462737.html
It feels like this should be possible to allow me to connect the Spark into my existing SFP+ optical network. It would be good to have some sort of option, even if it’s not officially supported, that has been tested to work on the Spark. Not everyone has Connect-X7 networking, so it feels like something that should be doable.
There is another option I’m aware of which is a cable that goes straight from QSFP to SFP+, I can provide the link for that if NVIDIA is willing to get it and try it. From memory it’s around the $30 mark.
sudo tee /etc/systemd/network/10-cx7.link > /dev/null <<‘EOF’
[Match]
Driver=mlx5_core
OriginalName=enP2p1s0f1np1
[Link]
AutoNegotiation=no
BitsPerSecond=10G
EOF
sudo systemctl restart systemd-networkd
Change it to the port you’re plugged in at. Fixed ConnectX-7 link flapping with my switch by disabling auto-negotiation and forcing 10G speed. The CX7 and switch equipment couldn’t agree on link parameters automatically for some reason (at least for my unifi switch) this bypassed that negotiation and locked in stable settings.
This sounds promising, I’m going to try it and see how I go. I know basically nothing about network setup in Linux. So when it didn’t work, I wasn’t really sure where to go from there. Hopefully this gets me going.
No luck for me unfortunately, tried both optical transceivers and a simple DAC cable without success. I suspect if I did get the better converter, it might just work, but not sure if it’s worth taking a gamble on it given my success so far. I appreciate your help though and might revisit this again in future.
As a last ditch effort, are there any commands I can run to check if it’s doing anything at all?
Thanks for all your help, I’m going to give the same FS.com converter a go which you linked to. I did look online to try and see what the original NVIDIA/Mallanox one would cost, but couldn’t even get a price listing for it, never mind finding a place to buy one, so I’ll see how it goes.
You’re awesome!!! I got the same adapter you linked to and it works like a charm with your instructions to disable auto negotiation and lock the link to 10G. I’ve now got it connected straight into my existing 10G SFP+ optical network.