CPUs do not stay offline

I am attempting to run a program on the TX1 with only cpu0 enabled, but the output of ~/tegrastats indicates that cpu1-3 are not staying offline. How can I force these to stay disabled, if that is possible?

I run the following as root before running the program:

echo 1 > /sys/devices/system/cpu/cpuquiet/tegra_cpuquiet/enable
echo 0 > /sys/devices/system/cpu/cpu1/online
echo 0 > /sys/devices/system/cpu/cpu2/online
echo 0 > /sys/devices/system/cpu/cpu3/online
echo "userspace" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo 102000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq

I also verify that they were actually disabled:

for i in $(seq 3); do
    cat /sys/devices/system/cpu/cpu$i/online
done

Here is a sample of the tegrastats output at the beginning of the program’s execution. The log output for the remainder of the program’s execution is similar.

RAM 876/3995MB (lfb 468x4MB) cpu [0%,0%,0%,0%]@102 EMC 4%@40 AVP 13%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 894/3995MB (lfb 468x4MB) cpu [56%,79%,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 919/3995MB (lfb 468x4MB) cpu [83%,off,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 944/3995MB (lfb 468x4MB) cpu [44%,90%,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 971/3995MB (lfb 468x4MB) cpu [76%,off,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 991/3995MB (lfb 468x4MB) cpu [76%,58%,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 1019/3995MB (lfb 468x4MB) cpu [45%,off,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 1037/3995MB (lfb 468x4MB) cpu [72%,74%,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 0%@307 EDP limit 1734
RAM 1053/3995MB (lfb 468x4MB) cpu [67%,80%,47%,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 4%@307 EDP limit 1734
RAM 1084/3995MB (lfb 468x4MB) cpu [93%,off,off,off]@102 EMC 4%@40 AVP 3%@12 NVDEC 192 MSENC 192 GR3D 9%@307 EDP limit 1734

I have tried disabling the startup script described here http://elinux.org/Jetson/Performance#Making_Settings_Stick and trying again after a fresh boot, but the results are the same. I have also tried it with tegra_cpuquiet both enabled and disabled.

sudo update-rc.d -f ondemand remove

atheatos,
Seems you are using older BSP version. Would you be able to use Jetpack 2.3.1 (or BSP r24.2.1) image and rerun the the test? Thanks.

chijen,

According to the answer to this thread https://devtalk.nvidia.com/default/topic/860092/how-do-i-know-what-version-of-l4t-my-jetson-tk1-is-running-/, I appear to already be running that version.

ubuntu@tegra-ubuntu:~/dev/perf$  head -n 1 /etc/nv_tegra_release
# R24 (release), REVISION: 2.1, GCID: 8028265, BOARD: t210ref, EABI: aarch64, DATE: Thu Nov 10 03:51:59 UTC 2016

Thank you for your time.

Is it possible that they are receiving interrupt signals that force them to execute some subroutines? Utilization on those cpus spike from and back to zero while cpu0 stays well above zero the entire time. If so, then it would appear to be something I have no control over. Performance is dropping as expected and they are clearly drawing less power than when they were all enabled, so toggling “online” to zero at least does something.

I have conducted another experiment, this time with a more intensive program. However, the TX1 appears to completely ignore my settings.

At 102 MHz:

RAM 1966/3995MB (lfb 254x4MB) cpu [54%,100%,59%,100%]@102 EMC 10%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 23%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [52%,100%,63%,100%]@102 EMC 10%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 52%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [60%,100%,51%,100%]@102 EMC 10%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 18%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [98%,60%,43%,100%]@102 EMC 9%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 54%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [100%,59%,37%,100%]@102 EMC 9%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 12%@998 EDP limit 1734

At 921 MHz

RAM 1966/3995MB (lfb 254x4MB) cpu [52%,63%,off,off]@921 EMC 26%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 99%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [54%,59%,off,off]@921 EMC 26%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 86%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [53%,59%,off,off]@921 EMC 26%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 99%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [59%,59%,off,off]@921 EMC 26%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 99%@998 EDP limit 1734
RAM 1966/3995MB (lfb 254x4MB) cpu [53%,61%,off,off]@921 EMC 27%@1600 AVP 0%@12 NVDEC 192 MSENC 192 GR3D 99%@998 EDP limit 1734

I also attempted to toggle “enabled” while the program ran, but nothing happened:

ubuntu@tegra-ubuntu:~$ cat /sys/devices/system/cpu/cpu1/online
1
ubuntu@tegra-ubuntu:~$ sudo su -c "echo 0 > /sys/devices/system/cpu/cpu1/online"
ubuntu@tegra-ubuntu:~$ cat /sys/devices/system/cpu/cpu1/online
1

Hi atheatos,

After flash via JetPack 2.3.1, I just issued below commands and then cpu1&2&3 always keep offline. FYI.

echo 0 > /sys/devices/system/cpu/cpuquiet/tegra_cpuquiet/enable
echo 0 > /sys/devices/system/cpu/cpu1/online
echo 0 > /sys/devices/system/cpu/cpu2/online
echo 0 > /sys/devices/system/cpu/cpu3/online

Thanks for looking into this, vickyy.

Did the cores continue to stay offline while running an intensive program? When I run the same commands, they go offline initially, but spin back up when a heavier process starts executing.


Edit:
Actually, disabling tegra_cpuquiet may have done it. I thought that I had already tried that, but I performed another trial with tegra_cpuquiet disabled and the cpus behaved as specified. I will run a job over the weekend and check the logs on Monday.

This would make sense according to the description of CPUquiet in the glossary of the Tegra Linux Driver Package Developers Guide (version 21.5, 2016/10/20):

For some reason this glossary entry is omitted in the documentation for version 24.2.1, but I assume it is still true.

It looks like tegra_cpuquiet was the culprit, so I’m marking that as the solution. I will update if I discover otherwise.

Thanks again, vickyy.

For example, I expect three instances of “off” in each line:

> grep -no off tegrastats_* | uniq -c | cut -d' ' -f7 | uniq
3