BlueField-3 IPSec HW offloading

Infrastructure & Networking BlueField
1

I’d like to test IPSec HW offloading using ip xfrm and strongSwan at BlueField-3 ARM, but it failed to set IP transform state and policy. Below is the base configuration of DPU.

  1. OvS setting
$ sudo ovs-vsctl show
97f6e5b4-02e4-407c-bdef-2051e6dda17e
    Bridge ovsbr1
        Port ovsbr1
            Interface ovsbr1
                type: internal
        Port pf0hpf
            Interface pf0hpf
        Port p0
            Interface p0
        Port en3f0pf0sf0
            Interface en3f0pf0sf0
    Bridge ovsbr2
        Port ovsbr2
            Interface ovsbr2
                type: internal
        Port pf1hpf
            Interface pf1hpf
        Port en3f1pf1sf0
            Interface en3f1pf1sf0
        Port p1
            Interface p1
    ovs_version: "2.9.2-0010-25.02-based-3.3.3"
  1. firmware version
$ sudo mlxfwmanager
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      BlueField3
  Part Number:      900-9D3B6-00CV-A_Ax
  Description:      NVIDIA BlueField-3 B3220 P-Series FHHL DPU; 200GbE (default mode) / NDR200 IB; Dual-port QSFP112; PCIe Gen5.0 x16 with x16 PCIe extension option; 16 Arm cores; 32GB on-board DDR; integrated BMC; Crypto Enabled
  PSID:             MT_0000000884
  PCI Device Name:  /dev/mst/mt41692_pciconf0
  Base MAC:         a088c2461848
  Versions:         Current        Available
     FW             32.43.2566     N/A
     PXE            3.7.0500       N/A
     UEFI           14.37.0013     N/A
     UEFI Virtio blk   22.4.0014      N/A
     UEFI Virtio net   21.4.0013      N/A

  Status:           No matching image found
  1. ip status
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: oob_net0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether a0:88:c2:46:18:6c brd ff:ff:ff:ff:ff:ff
    altname enamlnxbf17i0
    inet6 fe80::a288:c2ff:fe46:186c/64 scope link
       valid_lft forever preferred_lft forever
3: tmfifo_net0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:1a:ca:ff:ff:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.4/30 brd 192.168.100.7 scope global noprefixroute tmfifo_net0
       valid_lft forever preferred_lft forever
    inet6 fe80::21a:caff:feff:ff01/64 scope link
       valid_lft forever preferred_lft forever
4: p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
    link/ether a0:88:c2:46:18:58 brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0np0
5: p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
    link/ether a0:88:c2:46:18:59 brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1np1
6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 72:3f:89:d8:45:ba brd ff:ff:ff:ff:ff:ff
7: ovsbr1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:88:c2:46:18:58 brd ff:ff:ff:ff:ff:ff
8: ovsbr2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a0:88:c2:46:18:59 brd ff:ff:ff:ff:ff:ff
9: pf0hpf: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 96:e5:09:05:18:7b brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0nc1pf0
10: pf1hpf: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 0a:22:d2:76:3d:b7 brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1nc1pf1
    inet6 fe80::822:d2ff:fe76:3db7/64 scope link
       valid_lft forever preferred_lft forever
11: en3f0pf0sf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 82:81:95:fb:5d:2d brd ff:ff:ff:ff:ff:ff
    altname enp3s0f0npf0sf0
12: enp3s0f0s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 02:df:11:2b:a2:5f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::df:11ff:fe2b:a25f/64 scope link
       valid_lft forever preferred_lft forever
13: en3f1pf1sf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 3e:8f:ac:46:cf:21 brd ff:ff:ff:ff:ff:ff
    altname enp3s0f1npf1sf0
14: enp3s0f1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 02:2c:92:69:22:4e brd ff:ff:ff:ff:ff:ff
    inet6 fe80::2c:92ff:fe69:224e/64 scope link
       valid_lft forever preferred_lft forever

And here are questions.

  1. In document, PFs such as p0 and p1 are set by ip xfrm, but I can’t check the packet coming from PFs. I’d like to know the method to receive the external packet at BlueField-3 ARM like sudo ifconfig p0 <ip addr> up

  2. I wonder strongSwan requires setting two ports, p0 for insecure port and p1 for secure port to connect IPSec tunnel.

  3. Does DOCA IPSec target the RDAM as well as TCP/IP?

2

Hi polaris6921,

  1. You can use command ‘ethtool -S p0 | grep ipsec’ to see the packets statistic.
  2. It depends on your configurations.
  3. It depends on your configurations.