Dear NVIDIA Forum/Support Team,
I am developing a receiver application on a BlueField-3 DPU using DOCA Flow for hardware offload and DOCA Eth Rx for packet reception. The application runs on a Scalable Function (SF) port, specifically en3f0pf0sf3.
We are encountering a critical issue where RoCEv2 Unreliable Datagram(UD) packets are lost when they are routed through the internal Open vSwitch (OVS) bridge on the DPU, while standard IP traffic (ICMP, TCP, UDP) is successfully received.
We suspect this issue is related to the specific interaction between RoCEv2 UD traffic, OVS forwarding, and the DOCA Flow hardware offload path to the SF.
Environment & Configuration
- DPU Hardware: BlueField-3 (BF3) DPU (P/N: 900-9D3B6-00CV-A_Ax)
- Firmware Version: 32.46.1006
- OS/Kernel: Ubuntu 22.04.5 LTS (Kernel: 5.15.0-1074-bluefield)
- Used DOCA libs: DOCA Flow (Flow setup) + DOCA Eth Rx (Reception)
- RoCEv2 Type: UD Send
Network topology & OVS Configuration🌐 Network Topology & OVS Setup
Traffic is routed internally on the DPU using the br0 OVS bridge:
External Host -> p0(Physical port) -> br0(OVS bridge) -> en3f0pf0sf3(Scalable Function) -> My DOCA Eth Rx App
OVS configratuion
$ ovs-vsctl show
dc0aa193-0097-4026-aa9e-61000ad6fdbe
Bridge br0
Port br0
Interface br0
type: internal
Port p0
Interface p0
Port en3f0pf0sf3
Interface en3f0pf0sf3
Status
- Working case
When the application receives packets directly from p0 (bypassing the bridge/SF), all traffic types (ICMP, TCP, UDP, RoCEv2 UD) are received successfully.
- Fail case
When receiving from en3f0pf0sf3 via the p0 -> br0 -> sf path, ICMP, TCP, and UDP are received, but RoCEv2 UD packets are consistently lost.
Question
Is the loss of RoCEv2 packets when using DOCA Eth Rx on a Scalable Function (SF) port an intended limitation (by design)?
If not, what specific configurations are required?
Thank you for your time and assistance in resolving this issue.