I am trying to write a DPA application that act as bump-in-the wire. The “packet_processing” sample application with flexio SDK, receives a packet then sends it back to the wire. But instead of sending to the wire how can send it back to the host or ARM cores?
To achieve this goal I have modified the packet processing sample application to send the packet to the send queue after receiving the packet in the receive queue without swapping macs. This is giving expected results for UDP packets, forwarding them to host. But this is not working for TCP packets.
So, far I have tried following:
i) First, I tried to write a rule in the TX path so that matched TCP packets in the send queue are forwarded to the host. But, the problem with this approach is that I couldn’t figure out what should be correct vport number for the host representator to write the rule as provided with the sample.
ii) I wrote a OVS rule, to send matching TCP traffic from the physical function(e.g., p1) to host representator(e.g., pf1hpf). This is working without any issue, if I don’t run the DPA application. But if I run the DPA application, no packets are matched.
Any pointer to solve this issue will be greatly appreciated. Please let me know if you need anymore information.
DPDK can receive packets forwared from DPA but socket applications can’t
I am writing this reply to give more context and recent updates.
To use the DPA as a bump-in-the-wire, I have modified the flexio packet processing application to receive packets from the receive queue in the DPA handler, then replace L4 source with a predetermined port number, and finally send the packet to the send queue. L4 source modification is needed; otherwise, I have found that it creates infinite recirculation. As I am changing the L4 header, I have set FLEXIO_ETH_SEG_L3CS and FLEXIO_ETH_SEG_L4CS flags in flexio_dev_swqe_seg_eth_set() to re-calculate the checksum in HW.
My initial assumption that forwarding is working for UDP is somewhat incorrect. If I run a DPDK application on the host, it can receive forwarded packets from DPA without any issues. That’s why I was thinking UDP packet forwarding is working as I was running DPDK on the host. But later, I found that socket applications still can’t see any packets.
From this observation, I thought that the Linux kernel is rejecting the forwarded packets for some reason. If the packets were dropped by the NIC, DPDK wouldn’t be able to see them. So, I checked the UDP drop path in the kernel to check the reason for dropping. But, I can’t see anything there:
sudo perf trace -a -e skb:kfree_skb -e udp:udp_fail_queue_rcv_skb
Finally, I have installed a simple XDP program in the corresponding netdev to see if any packets show up. But I am getting the same behavior as the socket applications. UDP packets show up if the DPA application isn’t running. Otherwise, no packets are received in the XDP hook. That means the kernel is not even seeing those packets?
So, now I am confused. Why can the DPDK application see the packets, but the XDP/socket application can’t?
Thanks for posting your inquiry to the NVIDIA Developer forums!
As this is a technical inquiry related to the DOCA software framework and included reference applications, we highly recommend engaging the DOCA mailing list (at DOCA-Feedback@exchange.nvidia.com) for further assistance.
Thanks, and best regards,
NVIDIA Enterprise Experience