Dvfs

Hi,
I am doing research on Jetson TX2. I want to know whether DVFS strategy can be customized on this platform or not. Does TX2 apply for individual DVFS (allows each core to have a different individual voltage and frequency levels)? Does TX2 have several default DVFS strategies and how to switch?

Plus, does Jetson TX2 or AGX apply for per-core DVFS in hardware? I see only armv8 cpu embedded in these board.

I have referred to this topic:DVFS

By default Jetsons have the program “nvpmodel” installed. This will change the profile of which cores are active and what the minimum and maximum clock speeds are. Once in a profile one can run “jetson_clocks” to set to max clocks. Typically the command “sudo nvpmodel -m 0” is the max available, and so most performance for most cases would imply running “sudo nvpmodel -m 0” followed by “sudo jetson_clocks”.

Note that jetson_clocks is a human readable script, and has some options listed with “jetson_clocks -h”. nvpmodel has options listed with “nvpmodel -h”. You can examine what parameters are used by storing to a file for examination or actual later use (the save files are human readable).

Does TX2 or AGX supply for customized per-core DVFS or I can just only use pre-defined power model?

I see on “nvpmodel”, only 5 pre-defined power models could be used. And I try to store current configuration to file, modify the file and restore it by “jetson_clocks --restore <file>” but it’s not working (nothing changed).

And I firstly run “sudo jetson_clocks” to set min_frequency to maximum, and then run “jetson_clocks --store <file>”, then run “sudo nvpmodel -m 0”, then try to restore the settings, but error messages generated as below:

/usr/bin/jetson_clocks: line 73: echo: write error: Invalid argument
Error: Failed to restore /sys/devices/17000000.gp10b/devfreq/17000000.gp10b/min_freq

This time I run “sudo jetson_clocks --show” it shows the restoration is failed but power model has been changed to the max-performance model:

SOC family:tegra186  Machine:quill
Online CPUs: 0-5
CPU Cluster Switching: Disabled
cpu0: Online=1 Governor=userspace MinFreq=345600 MaxFreq=2035200 CurrentFreq=345600 IdleStates: C1=0 c7=0
cpu1: Online=1 Governor=schedutil MinFreq=345600 MaxFreq=2035200 CurrentFreq=499200 IdleStates: C1=0 c6=0 c7=0
cpu2: Online=1 Governor=schedutil MinFreq=345600 MaxFreq=2035200 CurrentFreq=652800 IdleStates: C1=0 c6=0 c7=0
cpu3: Online=1 Governor=userspace MinFreq=345600 MaxFreq=2035200 CurrentFreq=345600 IdleStates: C1=0 c7=0
cpu4: Online=1 Governor=userspace MinFreq=345600 MaxFreq=2035200 CurrentFreq=345600 IdleStates: C1=0 c7=0
cpu5: Online=1 Governor=userspace MinFreq=345600 MaxFreq=2035200 CurrentFreq=345600 IdleStates: C1=0 c7=0
GPU MinFreq=114750000 MaxFreq=1300500000 CurrentFreq=1134750000
EMC MinFreq=40800000 MaxFreq=1866000000 CurrentFreq=1866000000 FreqOverride=1
Fan: speed=255
NV Power Mode: MAXN

I have not tried to modify it from nvpmodel, but in theory, I think if you edit the saved nvpmodel.conf, and it accepts the changes, then this would probably be a good start. There are some DVFS tables in the kernel, but I think editing those is probably a “bad idea” even if it works. Consider that the “-m 0” model tends to be max performance already, and that any method used to exceed the values in the DVFS tables through kernel edits will probably exceed the hardware’s ability.

For jetson_clocks, you would need to run this after changing the nvpmodel, not before. If you are in a model with a lower minimum frequency, and you store this, and then go to a model where that lower minimum is no longer possible, then it is expected that violating the current model with lower frequencies would fail. nvpmodel is an allowable set of states, and jetson_clocks is setting to specific values in that model state. Removing the model state’s available range implies you will not be able to set the clock to the now “impossible” state.

I could be missing something, but from what you’ve shown it seems that your saved clock configuration is being run from a different model…go back to that model, and then restore, or else save clocks from the new model if you wish to restore it to the new model.

If you have some sort of use case to provide some context of what you want to accomplish it might help.

There are some DVFS tables in the kernel, but I think editing those is probably a “bad idea” even if it works. Consider that the “ -m 0 ” model tends to be max performance already, and that any method used to exceed the values in the DVFS tables through kernel edits will probably exceed the hardware’s ability.

Did you mean that there exists some “power cap” or other resource constraints corresponding to the hardware and the “-m 0” tends to achieve the acceptable boundary of these constraints? Does lower bound also exist for hardware (for example, if I configure a very low voltage to processors by DVFS, will the system still keep work correctly and slowly or crashed?)

I am considering whether I can modify DVFS configuration to control energy consumption by reducing some frequency or voltage at runtime or by cycles. And can I apply this control to each core of processors individually?

There is no “power cap” that I am aware of…power is a side effect of frequencies and voltages, and but there is no “direct” power cap.

The DVFS tables are set up to allow the safe range of operation. The entries which are available may be modified by current conditions, especially temperature. Actual modification of the tables could result in hardware failure. The “dynamic” part, when DVFS tables are not modified, concerns picking some optimum within that table. If configured for energy savings, then this might imply a slower and lower power entry; if work load implies a need to increase power, then a more aggressive table entry might be used.

Using jetson_clocks can disable the dynamic adjustments, but a line within DVFS tables will still be used. The maximum performance table entry used will depend on the current model.

All of the changes made by either nvpmodel or jetson_clocks are just an echo into some file in “/sys”. Those “/sys” files make up different details of the DVFS tables. You can create your own software to fine tune those values. The nvpmodel and jetson_clocks programs are just common recipes a typical developer might be interested in.

Some settings can be changed per core, and some cannot. I’m not sure which can or cannot. Often you will see a different “/sys” subdirectory for individual cores, and the echo of content into that core’s subdirectory would normally affect only that core.

Core 0 is a special case since core 0 handles hardware IRQs and must be running during boot and normal operation (this doesn’t mean it can’t be put into a lower power mode). Other cores can be completely disabled. Whether or not clock frequencies can be individually set I do not know (possibly they share some clocks).

I strongly advise you to look at the jetson_clocks program and the “/sys” files involved. This program is just a human readable bash script. There are similar hints from the config files for nvpmodel, e.g., look at “/etc/nvpmodel.conf” and “/etc/nvpmodel/*” (note that a TX2 is a “t186”).