DOCA provides a DPI engine that leverages several individual components to provide a framework for examining the full content of data packets. Two sample DPI applications with their source code are included with Bluefield DOCA software:
The files for these applications are found in /opt/mellanox/doca/examples
I tried to run and test the URL filtering application. I am so sorry to say this but the information found at DOCA SDK documentation is kinda useless as it is.
Can someone clarify these important aspects I tried to summarize below?
Itâs a bit long, but I wanted to include everything that others might ask later.
Cause: Application does not support more than 2 ports
Very âmeaningfulâ error.
Furthermore, class=regex:eth is also not clear. DPDK documentation says it is used to access the DPI engine (i.e., RegEx HW accelerator on the Bluefield). It is okay then, but is there any meaning behind regex:? eth is just for ETHERNET, or it tries to identify an interface ID?
sft_en=0 - I have not found any information about this at all.
Besides the EAL options, using the app is also not straightforward. There is no pattern to follow how URLs should be defined. I found out by having a quick look at the source code that is used SuricataâŠso it creates Suricata definitions from the CLI commands.
3.1. How to test the application? There is no information about how the âtestingâ environment should look like.
3.2 What interface we should send out the HTTP queries (for instance, by using SCAPY for brevity) to check the output of the URL filter?
3.3 The URL filters should not be quit, right? The output should be seen on its CLI?
3.4. Also not clear from which part of the system we should send the testing packets? Do they come from the Host? If yes, then there is no information about in which mode the SmartNIC should be? SEPERATED_HOST or EMBEDDED_CPU? I guess the latter, but still, then what application will connect the HOSTâs logical interfaces to the Bluefieldâs physical interfaces, i.e., the interfaces seen as ens5f0 (for instance in my case), connected through pf0hpf logical interface and the p0 physical interface on the Bluefield? Should there be an OvS running? If yes, again, can we run two DPDK applications on the same port? URL filter already consumes all the CPU cores by defaultâŠhow will OvS run then?
I tried to filter on slashdot.org via this filter command:
filter http "asd" "www.slashdot.org"
it was eaten by the app but then did not know how to proceed. My Bluefield ports are, of course, not the internet-facing ports in my experiment, so I tried sending scapy-crafted packets on the physical ports, both from the host and from the Bluefield. In particular, from Hostâs ens5f0, from Bluefieldâs pf0hpf, and p0. Scapy said packet sent, but no message is observed in the URL filter app.
Thanks, and sorry for the avalanche of questions :)
Okay, it seems that after gaining access to all materials, the answers for some of the questions above can be found in the DOCA SDK documentation. Just not necessarily at the particular applicationsâ descriptions (like in the case of URL Filter) but at the programming and DPI/RegEx applicationsâ documentations. It would be nice to have cross-references :)
Create DPI rules for the URL filter app: DPI Compiler :: NVIDIA DOCA SDK Documentation
Regarding the EAL flags, there is a note here: Application Recognition :: NVIDIA DOCA SDK Documentation
About representors: vSwitch and Representors Model :: NVIDIA DOCA SDK Documentation
Thanks
Hi there again :)
After finding the related material, I was playing around with the URL filter application, and I would like to ask whether it is normal that once it alerts on a specific packet it drops all consecutive ones WITHOUT even alerting?
I have the following setup:
2 Host machines (H1,H2) connected back-to-back through Bluefield-2 DPUs.
The Bluefield-2 DPU @ H2 is running the filter application. I have digged deeper a bit in Suricata (still very far from understanding much of it, though :)) and I created a DNS filter instead of the HTTP filter as I was not sure whether the TCP connection to a webserver should be established for the URL filter app to work.
So, I created the following rule description /tmp/signature.txt:
@Bluefield@H2:
alert dns any any -> any 53 (msg:"Test dns.query option"; dns.query; content:"google"; nocase; sid:1;)
Then, after starting the URL filter, I loaded this file as a database to let the URL filter to compile it to CDO.
@Bluefield@H2:
/opt/mellanox/doca/examples/url_filter/bin/doca_url_filter -a 0000:03:00.0,class=regex:eth,representor=[65535],sft_en=0 -- -p
URL FILTER>> commit database /tmp/signature.txt
It is compiled without errors.
Then, I craft a DNS packet on the other Bluefield@H1 (which is connected to the Bluefield@H2) and send it to the corresponding interface.
@Bluefield@H1
>>> dns_packet=Ether(dst="0c:42:a1:a4:8a:08")/IP(dst="8.8.8.8")/UDP()/DNS(rd=1, qd=DNSQR(qname="google.com"))
>>> sendp(dns_packet,iface="p0")
Without having any OvS(-DPDK) running on the Bluefield@H2, I encountered that packets are also sent up to the host itself, i.e., to H2. So, I ran a tcpdump on H2.
@H2: ifconfig ens5f0 up
@H2: tcpdump -ni ens5f0 udp
So, after sending a crafted DNS packet from @Bluefield@H1, I observe the following:
@Bluefield@H2:
INFO: SIG ID: 1, URL MSG: Test dns.query option, SFT_FID: 1
@H2 tcpdump:
04:27:41.284151 IP 192.168.100.2.53 > 8.8.8.8.53: 0+ A? google.com. (28)
However, when I send the same DNS packet over and over again, I do not see any alert anymore, neither I see any packet received @H2 via tcpdump.
Itâs like after the alert it is added to a banlist.
I tried waiting for sometime to make any cached entry expire, but no results.
I can only achieve a new alert and packet received via tcpdump @H2, if I craft another DNS request, e.g., by changing the dst IP, or by restarting URL filter and reloading the database.
As mentioned in the beginning, I am not a suricata expert, so this question might relate to suricata behavior not to the Bluefieldâs directly.
Can someone enlighten me about this?
You can see my terminals below. All of them has the âlocationâ in their title/header.
Thanks
Hi cslev,
Thanks for trying out some of our reference applications.
The âprint on matchâ feature is only intended for the first time a flow is recognized, this means that traffic originating from the same FID is no longer inspected and is now filtered, therefore no tcpdump is seen.
You can try changing the FIDâs values, or more precisely one of the 5-tuple which make this flow unique. If a different port is used, you should see the print again.
I tried to add the same rule as you but with unsuccessful results
alert dns any any -> any 53 (msg:"Test dns.query option"; dns.query; content:"google"; nocase; sid:1;)
sudo /opt/mellanox/doca/examples/url_filter/bin/doca_url_filter -a 0000:03:00.0,class=regex -a auxiliary:mlx5_core.sf.4,sft_en=1 -a auxiliary:mlx5_core.sf.5,sft_en=1 -- -p
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: No available hugepages reported in hugepages-32768kB
EAL: No available hugepages reported in hugepages-64kB
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Device is not NUMA-aware, defaulting socket to 0
EAL: Probe PCI driver: mlx5_pci (15b3:a2d6) device: 0000:03:00.0 (socket 0)
EAL: No legacy callbacks, legacy socket not created
Temporary WARN - Destination table level lower than Source
[15:34:58:806825][DOCA][I][DWRKR]: 7 cores are used as DPI workers
URL FILTER>> commit database /tmp/signature.txt
/tmp/2465221/signatures.rules
rules file is /tmp/2465221/signatures.rules
Info: Setting target hardware version to v5.7...done
Info: Setting virtual prefix mode to 0...done
Info: Setting prefix capacity to 32K...done
Info: Setting compiler objective value to 5...done
Info: Setting number of threads for compilation to 1...done
Info: Reading ruleset...done
Info: Detected 1 rules
Info: Enabling global single-line mode...done
Info: Setting maximum TPE data width to 4...done
Info: Scanning rules...[==============================]...done
Info: Analising possible prefix usage...[==============================]...done
Info: Mapping prefixes, phase 1...[==============================]...done
Info: Mapping prefixes, phase 2...[==============================]...done
Info: Running rules analysis...[==============================]...done
Info: Optimizing memory map...[==============================]...done
Info: Analyzing memory map...[==============================]...done
Info: Calculating thread instructions...[==============================]...done
Info: Beginning to write memory map for ROF2...done
Info: PPE total 1-byte prefix usage: 0/256 (0%)
Info: PPE total 2-byte prefix usage: 0/2048 (0%)
Info: PPE total 3-byte prefix usage: 0/2048 (0%)
Info: PPE total 4-byte prefix usage: 1/32768 (0.00305176%)
Info: TPE instruction RAM TCM partition usage: 2048/2048 (100%)
Info: TPE instruction RAM external memory partition usage: 6183/13M (0.0453582%)
Info: TPE class RAM usage: 2/256 (0.78125%)
Info: Estimated threads/byte: 2.592e-10
Info: Finalizing memory map for ROF2...done
Info: Storing ROF2 data...done
Info: Number of rules compiled = 1/1
Info: Writing ROF2 file to /tmp/2465221/rof/signatures_compiled.rof2
Info: Writing binary ROF2 file to /tmp/2465221/rof/signatures_compiled.rof2.binary...done
mlx5_regex: Rules program failed 22
mlx5_regex: Failed to program rxp rules.
[15:35:02:843753][DOCA][E][UFLTR::Core]: Loading DPI signature failed