Two Thor devices are directly connected using the mgbe0_0 interface. Bidirectional latency is tested with sockperf. Currently, the maximum latency is around 260 µs. There are many occurrences of latency in the 250–260 µs range, while most of the remaining measurements are below 100 µs.
What is causing the 250–260 µs latency spikes, and how can this issue be resolved?
This is the test record:
sudo taskset -c 12 chrt -f 99 sockperf server
for i in $(seq -w 1 50000); do sudo taskset -c 12 chrt -f 99 sockperf ping-pong -i 192.168.4.3 -t 60 --full-log result/sockperf_${i}.csv; done
======================================================================
[Test: Bidirectional Latency – Statistics After Merging All CSV Files]
======================================================================
Test parameters: --mps=600000 --burst=1 --reply-every=1 --msg-size=14 --time=60
Test mode: ping-pong
Approx. bandwidth: Application layer (one-way) 67.20 Mbps (8.40 MB/s),
Link layer (~84B Ethernet frame) ≈ 806.4 Mbps
----------------------------------------------------------------------
Valid duration: 1,238.5 sec (0.34 hours)
Total SentMessages: 23,592,451, ReceivedMessages: 23,592,451 (no packet loss)
Average per second: 19,049.00 msg/s
Average per millisecond: 19.05 msg/ms
----------------------------------------------------------------------
Average latency: 26.209 us
Maximum latency: 258.977 us
Standard deviation: 1.126 us
10th (10th largest): 257.254 us
100th (100th largest): 52.940 us
1000th (1000th largest): 42.283 us
25%: 25.528 us (ranked 17,752,955th from the end)
50%: 25.930 us (ranked 11,801,736th from the end)
75%: 26.509 us (ranked 5,907,281st from the end)
99%: 30.426 us (ranked 236,020th from the end)
99.9%: 33.310 us (ranked 23,655th from the end)
99.99%: 39.019 us (ranked 2,360th from the end)
99.999%: 49.904 us (ranked 236th from the end)
----------------------------------------------------------------------
>20us ratio: 100.0000% (23,591,401 samples)
>30us ratio: 1.3277% (313,231 samples)
>40us ratio: 0.0073% (1,723 samples)
>50us ratio: 0.0010% (231 samples)
>60us ratio: 0.0002% (47 samples)
>70us ratio: 0.0002% (44 samples)
>80us ratio: 0.0002% (44 samples)
>90us ratio: 0.0002% (44 samples)
>100us ratio: 0.0002% (44 samples)
>110us ratio: 0.0002% (44 samples)
>120us ratio: 0.0002% (44 samples)
>150us ratio: 0.0002% (44 samples)
>200us ratio: 0.0002% (44 samples)
>250us ratio: 0.0002% (41 samples)
----------------------------------------------------------------------
20–30us ratio: 98.6723%
30–40us ratio: 1.3248%
40–50us ratio: 0.0063%
50–60us ratio: 0.0008%
250–260us ratio within >100us: 93.1818%