Connection through infiniband network adapter on the DPU

Infrastructure & Networking BlueField
,
1

Hello，
I have some issues whiling using bluefiled2.I will appreciate if you can help on this to me.

I have two ubuntu machines, which I’ll call A and B for convenience.

A is equipped with a bluefield2-DPU, and B is equipped with a connectX-3 infiniband network card.

I connected A to the switch through the network adapter port on the DPU and B to the switch through the connectX-3 network adapter.Then I give the two ports IP addresses in the same subnet.
At this point I think A should be able to ping B,but it didn’t work out.

Here is part of my IP config on A

ens14f0np0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.111.8 netmask 255.255.255.0 broadcast 192.168.111.255
inet6 fe80::ba3f:d2ff:fe54:c116 prefixlen 64 scopeid 0x20
ether b8:3f:d2:54:c1:16 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 19 bytes 1366 (1.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
tmfifo_net0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.100.1 netmask 255.255.255.0 broadcast 192.168.100.255
inet6 fe80::21a:caff:feff:ff02 prefixlen 64 scopeid 0x20
ether 00:1a:ca:ff:ff:02 txqueuelen 1000 (Ethernet)
RX packets 99001 bytes 8496198 (8.4 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 242575 bytes 332808453 (332.8 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Here is part of my IP config on B

ibs3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 2044
inet 192.168.111.9 netmask 255.255.255.0 broadcast 192.168.111.255
inet6 fe80::ee0d:9a03:10:eb41 prefixlen 64 scopeid 0x20
unspec A0-00-02-20-FE-80-00-00-00-00-00-00-00-00-00-00 txqueuelen 256 (UNSPEC )
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 21 bytes 1356 (1.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The switch on the DPU also seems to be working properly

ubuntu@localhost:~/spdk$ sudo ovs-vsctl show
ad38f0a8-75fc-44e1-a1cc-fdc555eb7df3
Bridge ovsbr1
Port pf0hpf
Interface pf0hpf
Port p0
Interface p0
Port ovsbr1
Interface ovsbr1
type: internal
Port en3f0pf0sf0
Interface en3f0pf0sf0
Bridge ovsbr2
Port en3f1pf1sf0
Interface en3f1pf1sf0
Port ovsbr2
Interface ovsbr2
type: internal
Port pf1hpf
Interface pf1hpf
Port p1
Interface p1
ovs_version: “2.17.8-3feee121f”

Could you tell me how to solve it?
Many thanks

2

Hi,

It looks like the DPU side port is configured as ethernet port, and CX3 side is configured as IB port. If your CX3 adapter is VPI, you can try changing it to ethernet using /sbin/connectx_port_config script.

In addition, CX3 adapter is currently end of life and end of support, and it was not tested with Bluefield-2. So proper operation between these adapters cannot be guaranteed.

Best Regards,
Anatoly