Automating CPU Pinning for Offload Worker Queue in HWOL/Conntrack Offload Environment

Infrastructure & Networking Software And Drivers SoC And SmartNIC
1

Hello,

I am currently testing for the introduction of HWOL and conntrack offload.
In this process, I am trying to apply CPU pinning to softirq and offload worker queue.

For CPU pinning of the offload worker queue, I am planning to apply it by entering the CPU bitmask in the /sys/devices/virtual/workqueue/nf_ft_offload_*/cpumask file.
I have verified up to the part where the offload workqueue task is properly assigned with the CPU bitmask specified in this file.

rubyCopy code

root@Qaamdhost02:/opt/script# ll /sys/devices/virtual/workqueue/nf_ft_offload_*/cpumask
drwxr-xr-x 3 root root 0 Jul  4 15:53 /sys/devices/virtual/workqueue/nf_ft_offload_add/cpumask
drwxr-xr-x 3 root root 0 Jul  4 15:53 /sys/devices/virtual/workqueue/nf_ft_offload_del/cpumask
drwxr-xr-x 3 root root 0 Jul  4 15:53 /sys/devices/virtual/workqueue/nf_ft_offload_stats/cpumask

I tried to automate the above setting, but the /sys/devices/virtual/workqueue/nf_ft_offload_* type directory is only created when the conntrack offload task occurs, so it is difficult to automate before or after the VM is deployed.

I am curious if there is another way for CPU pinning of the offload worker queue, or if there is a way to create this directory immediately after the VM is deployed.

Thank you.

2

hi kyoon

As far as I know, current kernel version do not have such API.

Thank you
Meng, shi

3

From our own tests, it appears that the mlx5_core Driver registers a Linux Worker Queue task when a VM is created.

Considering that the Representation Port connected to the VM is activated when the VM is created, we conducted a test using the carrier event of networkd-dispatcher as follows:

cat /etc/networkd-dispatcher/carrier.d/03-offload_pinned
#!/usr/bin/env bash
echo 0000000e,0000000e > /sys/devices/virtual/workqueue/nf_ft_offload_add/cpumask
echo 0000000e,0000000e > /sys/devices/virtual/workqueue/nf_ft_offload_del/cpumask
echo 0000000e,0000000e > /sys/devices/virtual/workqueue/nf_ft_offload_stats/cpumask

We confirmed that it works normally when a VM is created.

Could you please let us know if there are any potential issues with this configuration or if there is a better approach?