I have been trying to integrate the nv_encap_psp_transport/nv_encap_psp_v4_tunnel as well as the corresponding psp_crypto.encrypt P4 APIs into a program I am working on and have not been having much success. I am wondering if there are any additional environmental setup steps necessary to use PSP on the BlueField3.
Below are my observations:
When the apis are being invoked within their corresponding actions, packets seem to disappear in the pipeline. I have added direct counters to said actions and have validated that without the psp externs, packets properly hit the actions and then continue further packet processing.
I have observed this same behavior in the provided psp reference example where packets hitting the encap table entries appear to be silently dropped. For validation, I added a direct counter to the encap_table and verified that when the psp externs are commented out, the actions are properly hit. When the psp externs are present, the counter no longer registers packets hitting the entries.
I have tried using both the encap and the encrypt table actions in the psp examples and have observed the same silent drop behavior.
The ingress traffic used is basic IPv4 UDP frame:
1.1.1.1 → 2.2.2.3 UDP 110 5060 → 5060 Len=64
Below is some information about the DPU and the SDK version I am using:
sudo mlxfwmanager --query
Querying Mellanox devices firmware …
Device #1:
System Information
——————————-
DOCA version: 3.2.0118-1
DPL Runtime daemon version: 1.3.0-1.rel-1.3.0
Debug Schema Version
Device ID: 1000, doctype: /nvp4c/devtools/base/v14, target_doctype: /nvp4c/devtools/doca/v5
Linux OS version : “Ubuntu 24.04.3 LTS”
OFED Version : OFED-internal-25.10-1.2.8
RDMA Core Version : 2510.0.10-1
Packets could get dropped if we haven’t created/programmed the PSP object and keys from the control plane via DPL runtime controller.
The PSP object (NvPSP) should be created with the DEK and valid spi and sa_index.
The PSP Encryption and DPL Runtime Controller SDK User Guide shows the full flow, including the dpl_sample_psp server/client that perform key exchange and program the PSP keys before the P4 pipeline runs. Hope this helps.
From what I can tell, I have programmed the DEK properly, as I have been able to query the DEK APIs and get a valid hit from the entries that I have supposedly installed. Given that you cannot query everything you have programmed into the DEK (for security reasons), it is still a bit of an assumption that everything has been programmed correctly. That being said, the PSP object only comes into play when actually performing the encrypt operation as far as I know. I have been unsuccessful with using the encapsulation APIs standalone as well which is what I have tried to do in using the example as I am not currently peering with an additional DPU.
Any additional thoughts on this are most welcome. The silent drop is a bit tricky as well, is there anything else that can been looked at to determine root cause of a packet drop?
My prior reply may have been a bit ambiguous. I have so far tried using the encrypt functionality in my own application which is using the encrypt function (as well as just encap). In the PSP sample application I have not tried the encrypt as I don’t have the key exchange setup yet, so was just trying to see if encap worked which I was unable to verify.
I would think the key exchange needs to happen before encap/encrypt.
Maybe try getting “dpl_sample_psp” working and compare the encryption packet path and workflow.
Otherwise you will have to collect counters and logs around the PSP pipeline to see where and why packets are being dropped.
I have verified key installation to the best of my ability. I know where the packet is getting dropped from a P4 action perspective, that much is clear. The why is what is alluding me. Counters are what I am looking for at this point. I have yet to find a useful counter to explain this particular drop. Adding my own counter to the action executing encap/encrypt causes the counter to not increment, but when commenting out said encap/encrypt the counter increments.