UDP client CPU utilization has increased on Jetpack 5

I have several UDP senders/receivers running on my AGX Xavier that are now consuming more than double the expected CPU after upgrading to Jetpack 5 from Jetpack 4. Please confirm if this issue is reproducible on your end by running the minimal example below.

Test Commands
Sender:
iperf -s

Receiver:
./topp iperf -c 127.0.0.1 -u -b 30M -t 600

topp is a profiling script that measures CPU utilization of a program(originally posted on stackoverflow)

#!/bin/bash
set -e
topp() (
  if [ -n "$O" ]; then
    $* &
  else
    $* &>/dev/null &
  fi
  pid="$!"
  trap "kill $pid" SIGINT
  o='%cpu,%mem,vsz,rss'
  printf '%s\n' "$o"
  i=0
  while s="$(ps --no-headers -o "$o" -p "$pid")"; do
    printf "$i $s\n"
    i=$(($i + 1))
    sleep "${T:-0.1}"
  done
)

topp $*

CPU Utilization:
Jetpack 4: 3.7%
Jetpack 5: 8.7%

Hi,
Could you run sudo jetson_clocks and compare sudo tegrastats of Jetpack 4 and Jetpack 5? And the latest versions are 4.6.4 and 5.1.3. It would be great if you can compare the latest releases.

tegrastats isn’t very helpful when you have to profile a single program, that is why I used the script I shared above.

However, here’s the output of tegrastats while I had the iperf client running on Jetpack 5.1.1. I am using a carrier board from Connect Tech that currently does not have a 5.1.3-based BSP.

01-30-2024 05:45:28 RAM 3421/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [2%@2263,0%@2263,20%@2263,0%@2263,0%@2262,1%@2263,0%@2262,0%@2261] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@44.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47.25C PMIC@50C GPU 936mW/936mW CPU 1218mW/1218mW SOC 2436mW/2436mW CV 0mW/0mW VDDRQ 280mW/280mW SYS5V 4737mW/4737mW
01-30-2024 05:45:29 RAM 3421/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [4%@2262,0%@2262,0%@2263,17%@2261,1%@2263,0%@2263,0%@2261,0%@2261] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@44.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47C PMIC@50C GPU 937mW/936mW CPU 1218mW/1218mW SOC 2437mW/2436mW CV 0mW/0mW VDDRQ 281mW/280mW SYS5V 4737mW/4737mW
01-30-2024 05:45:30 RAM 3421/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [3%@2263,0%@2259,0%@2262,20%@2262,0%@2262,0%@2262,0%@2262,0%@2262] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@44.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47C PMIC@50C GPU 937mW/936mW CPU 1218mW/1218mW SOC 2437mW/2436mW CV 0mW/0mW VDDRQ 281mW/280mW SYS5V 4737mW/4737mW
01-30-2024 05:45:31 RAM 3421/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [2%@2262,0%@2263,1%@2262,19%@2263,1%@2263,0%@2258,0%@2263,0%@2256] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@44.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47C PMIC@50C GPU 937mW/936mW CPU 1218mW/1218mW SOC 2437mW/2436mW CV 0mW/0mW VDDRQ 281mW/280mW SYS5V 4737mW/4737mW
01-30-2024 05:45:32 RAM 3420/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [4%@2265,1%@2262,0%@2263,19%@2262,0%@2263,0%@2261,0%@2263,0%@2260] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@44.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@46.75C PMIC@50C GPU 937mW/936mW CPU 1218mW/1218mW SOC 2437mW/2436mW CV 0mW/0mW VDDRQ 281mW/280mW SYS5V 4737mW/4737mW
01-30-2024 05:45:33 RAM 3420/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [3%@2263,0%@2261,0%@2265,21%@2263,0%@2263,0%@2262,0%@2263,0%@2262] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@44.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47C PMIC@50C GPU 937mW/936mW CPU 1218mW/1218mW SOC 2437mW/2436mW CV 0mW/0mW VDDRQ 281mW/280mW SYS5V 4776mW/4743mW
01-30-2024 05:45:34 RAM 3430/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [5%@2263,15%@2263,3%@2262,15%@2262,1%@2267,4%@2262,5%@2258,2%@2263] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@45.5C thermal@44.7C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47C PMIC@50C GPU 936mW/936mW CPU 1967mW/1325mW SOC 2436mW/2436mW CV 0mW/0mW VDDRQ 280mW/280mW SYS5V 4776mW/4748mW
01-30-2024 05:45:35 RAM 3446/31012MB (lfb 151x4MB) SWAP 169/36138MB (cached 1MB) CPU [92%@2263,5%@2262,10%@2262,9%@2261,8%@2263,2%@2263,4%@2266,9%@2263] EMC_FREQ 0%@2133 GR3D_FREQ 0%@1377 VIC_FREQ 115 APE 150 AUX@45C CPU@45.5C thermal@44.85C Tboard@44C AO@44C GPU@44.5C iwlwifi@59C Tdiode@47.25C PMIC@50C GPU 936mW/936mW CPU 2715mW/1498mW SOC 2528mW/2448mW CV 0mW/0mW VDDRQ 374mW/292mW SYS5V 4776mW/4751mW

CPU usage has improved considerably after disabling the KPTI mitigation for Meltdown, by adding kpti=0 to the kernel boot parameters. CPU usage for the example posted earlier in the thread is now around 6% of a CPU core.