Hello,
I have a basic setup with some VMs. Then VFs attached to those VMs using iommu/vfio. That setup was working fine last week, using dpdk-testpmp and Trex inside those VMs.
I could also bring up Linux ip devices, ping or use iperf3 between VMs, via the NICs, again a basic setup:
iperf3 -s in a VM ←> NIC A ←> VLAN aware switch ←> NIC b ←> VM and iperf3 -c
Then I started to change a bit the PF settings… i shouldn’t have done that :-) Since, if I bring up any link inside of such a VM, I’m immediately getting huge packets storms: ip link set up, so packet storm starts.
Either using a dpdk app bringing up ports or bringing up a single Linux ip device produces packets storms, at high rate outside of the box, between my two NICs, over my external switch:
ens1f0np0 RX packets per second: 81841134
TX packets per second: 141
ens3f0np0 RX packets per second: 72014571
TX packets per second: 72023670
NICs are the same:
driver: mlx5_core
version: 6.12.10
firmware-version: 22.37.1014 (MT_0000000437)
expansion-rom-version:
bus-info: 0000:c0:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
I’m using the PF for untaged inband management (the storm kills my management path…).
VFs are each setup with a dedicated MAC plus a VLAN: “vlan 100, spoof checking on, link-state auto, trust off, query_rss off”. In the settings, I have “rx-vlan-offload: on”, “tx-vlan-offload: on”. That should auto tag/untag packets of the VFs.
Is something wrong with my other settings? Are settings written in flash? I rebooted the box, but I couldn’t recover from what I had done last week…
Best regards
Features for ens1f0np0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: on
tx-ipxip6-segmentation: on
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
tx-gso-list: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: on