ConnectX-6 Dx Crypto and Secure Boot, XDP Hardware offload not possible because of IPSec?

Hey,

I am trying to offload a simple XDP Program to hardware on my ConnectX-6 Dx Crypto and Secure boot device.

e.g. the xdp1 kernel sample (https://github.com/torvalds/linux/tree/master/samples/bpf):

idp@shs01-muc5-fw:~/xdp/linux/samples/bpf$ sudo ./xdp1 -N enp99s0f1

libbpf: elf: skipping unrecognized data section(16) .eh_frame

libbpf: elf: skipping relo section(17) .rel.eh_frame for section(16) .eh_frame

link set xdp fd failed

Or directly with ip:

idp@shs01-muc5-fw:~/xdp/t1$ sudo /opt/mellanox/iproute2/sbin/ip link set dev enp99s0f1 xdp obj simple.c.elf section simple verbose

Prog section ‘simple’ loaded (5)!

  • Type: 6

  • Instructions: 2 (0 over limit)

  • License: WTFPL

Verifier analysis:

0: (b7) r0 = 2

1: (95) exit

processed 2 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

RTNETLINK answers: Invalid argument

According to dmesg this is related to IPSec offloading:

[ 3584.043640] mlx5_core 0000:63:00.1 enp99s0f1: can’t set XDP with IPSec offload

How do I get XDP offloading to work, with or without IPSec?

ethtool -k:

idp@shs01-muc5-fw:~/xdp/linux/samples/bpf$ /opt/mellanox/ethtool/sbin/ethtool -k enp99s0f1

Features for enp99s0f1:

rx-checksumming: on

tx-checksumming: on

tx-checksum-ipv4: off [fixed]

tx-checksum-ip-generic: on

tx-checksum-ipv6: off [fixed]

tx-checksum-fcoe-crc: off [fixed]

tx-checksum-sctp: off [fixed]

scatter-gather: on

tx-scatter-gather: on

tx-scatter-gather-fraglist: off [fixed]

tcp-segmentation-offload: on

tx-tcp-segmentation: on

tx-tcp-ecn-segmentation: off [fixed]

tx-tcp-mangleid-segmentation: off

tx-tcp6-segmentation: on

generic-segmentation-offload: on

generic-receive-offload: on

large-receive-offload: off

rx-vlan-offload: on

tx-vlan-offload: on

ntuple-filters: off

receive-hashing: on

highdma: on [fixed]

rx-vlan-filter: on

vlan-challenged: off [fixed]

tx-lockless: off [fixed]

netns-local: off [fixed]

tx-gso-robust: off [fixed]

tx-fcoe-segmentation: off [fixed]

tx-gre-segmentation: on

tx-gre-csum-segmentation: on

tx-ipxip4-segmentation: on

tx-ipxip6-segmentation: on

tx-udp_tnl-segmentation: on

tx-udp_tnl-csum-segmentation: on

tx-gso-partial: on

tx-sctp-segmentation: off [fixed]

tx-esp-segmentation: on

tx-udp-segmentation: on

fcoe-mtu: off [fixed]

tx-nocache-copy: off

loopback: off [fixed]

rx-fcs: off

rx-all: on

tx-vlan-stag-hw-insert: on

rx-vlan-stag-hw-parse: off [fixed]

rx-vlan-stag-filter: on [fixed]

l2-fwd-offload: off [fixed]

hw-tc-offload: off

esp-hw-offload: on [fixed]

esp-tx-csum-hw-offload: on [fixed]

rx-udp_tunnel-port-offload: on

tls-hw-tx-offload: on

tls-hw-rx-offload: off [fixed]

rx-gro-hw: off [fixed]

tls-hw-record: off [fixed]

Simple XDP:

#include <linux/bpf.h>

#define __section(NAME) attribute((section(NAME), used))

__section(“license”)

char __license = “WTFPL”;

__section(“simple”) int xdp_simple(struct xdp_md *ctx) {

return XDP_PASS;

}

Hi Herberth,

XDP should work with IPsec. There is a bug fix that was posted recently to make XDP available also with IPsec, no need to disable IPsec.

It will be available in kernel v5.12.

Regards,

Chen