I have some Connect x5 NIC. I find a way to isolate my NIC so it only use CPU from where it attached on motherboard to avoid using CPU across NUMA node. Which is the best way to do that? Should I configure it on my software or on my OS? FYI, I wanto to use it with DPDK+Openstack.
Hi Muhammad,
How to find the Numa node connected to the network adapter
cat /sys/class/net//device/numa_node
0
If you have MFT installed, you can also use:
mst start
mst status -v
DEVICE_TYPE MST PCI RDMA NET NUMA
ConnectX6(rev:0) NA 83:00.0 mlx5_0 net-ib0 1
ConnectX6(rev:0) NA 83:00.1 mlx5_1 net-ib1 1
How to find the CPU’s associated with NUMA nodes:
numactl --hardware
node 0 cpus: 0 2 4 6 8 10 12 14
node 1 cpus: 1 3 5 7 9 11 13 15
In general, you can use taskset in order to specify the relevant NUMA and cores, for example:
taskset -c 0 numactl --membind=0
In regards to DPDK - in testpmd you can set the number of cores used by the application using nb-cores flag, and to specify the cores participate in the test with -l flag.
For example:
/testpmd -l 80-83 -n 4 --socket-mem=4096 -w 0000:c1:00.0 -w 0000:c1:00.1 – --burst=64 --txd=512 – rxd=512 --mbcache=512 --rxq=1 --txq=1 --nb-cores=1 --rss-udp --disable-crc-strip --forward-mode=io -a -i
Regards,
Chen