Fan runs on full on startup

Newly flashed unit, only installed jetpack past initial setup. Fans turn on max on startup, no immediate way to turn off. The unit is cool as a cucumber at startup and cools off even more. The unit did not have this behaviour prior to flashing. How do I fix?
Output of journalctl -e attached.
startup.txt (143.5 KB)

I probably can’t answer, but is this a dev kit? If it is a third party carrier board, then it is very likely it simply needs to be flashed with their board support software (which in turn would be a device tree modified version of what you used). If this is a dev kit, then I don’t know why it is doing that.

I’m afraid it’s a dev kit. Had no problems prior to flashing.

Does anything change if you run “sudo nvpmodel -m 0” (which allows everything to maximum, but does not force fans on; however, note that modes revert at reboot…saving mode is a separate step with the “`-f ” option)?

Almost forgot: What do you see from “systemctl list-units | grep -i nvfancontrol”?

nvfancontrol.service loaded active running nvfancontrol service

sudo nvpmodel -m 0 made no difference even after reboot. Any other ideas?

Does the file “~/.jetsonclocks_fan.txt” exist? If so, can you post a copy?

What do you see from:
jetson_clocks --show

~/.jetsonclocks_fan.txt is not present.

# jetson_clocks --show
SOC family:tegra234  Machine:Jetson AGX Orin
Online CPUs: 0-11
cpu0: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2112000 IdleStates: WFI=1 c7=1
cpu1: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu10: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2112000 IdleStates: WFI=1 c7=1
cpu11: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2188800 IdleStates: WFI=1 c7=1
cpu2: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu3: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu4: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu5: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2112000 IdleStates: WFI=1 c7=1
cpu6: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=729600 IdleStates: WFI=1 c7=1
cpu7: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2188800 IdleStates: WFI=1 c7=1
cpu8: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2188800 IdleStates: WFI=1 c7=1
cpu9: Online=1 Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2188800 IdleStates: WFI=1 c7=1
GPU MinFreq=306000000 MaxFreq=1300500000 CurrentFreq=306000000
EMC MinFreq=204000000 MaxFreq=3199000000 CurrentFreq=2133000000 FreqOverride=0
DLA0_CORE MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000
DLA0_FALCON MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
DLA1_CORE MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000
DLA1_FALCON MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
PVA0_VPS0 MinFreq=0 MaxFreq=1152000000 CurrentFreq=1152000000
PVA0_AXI MinFreq=0 MaxFreq=832000000 CurrentFreq=832000000
FAN Dynamic Speed control=active hwmon1_pwm=255
NV Power Mode: MAXN

The above is why the fan runs as it does. Mine is set like this:
FAN Dynamic Speed control=active hwmon2_pwm=64

I think 255 is likely “always on”. What do you see from:
grep 'FAN_SPEED_OVERRIDE' /usr/bin/jetson_clocks

What do you see from:
systemctl status nvfancontrol.service

Also, what do you see from:
cat /sys/devices/platform/pwm-fan/hwmon/hwmon2/pwm1

$ grep -n 'FAN_SPEED_OVERRIDE' /usr/bin/jetson_clocks
29:FAN_SPEED_OVERRIDE=0
198:			if [ "${FAN_SPEED_OVERRIDE}" -eq "1" ]; then
602:				FAN_SPEED_OVERRIDE=1
$ systemctl status nvfancontrol.service
● nvfancontrol.service - nvfancontrol service
     Loaded: loaded (/etc/systemd/system/nvfancontrol.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-01-09 17:51:00 CET; 14h ago
   Main PID: 528 (nvfancontrol)
      Tasks: 1 (limit: 36321)
     Memory: 468.0K
     CGroup: /system.slice/nvfancontrol.service
             └─528 /usr/sbin/nvfancontrol &

jan 09 17:51:00 hugo systemd[1]: Started nvfancontrol service.
$ cat /sys/devices/platform/pwm-fan/hwmon/hwmon1/pwm1
255

NB: You asked for hwmon2 which does not exist on my unit, so I gave you hwmon1 instead.

Oh! Found the file you asked for. In the root home folder (/root).

$ sudo cat /root/.jetsonclocks_fan.txt
active

Ok, so that is progress. Try removing “/root/.jetsonclocks_fan.txt”, and reboot. If it reappears see what it says. From what I can see in the scripts you quoted hwmon1 (mine is hwmon2, which is why I used that, but since you have hwmon1, then this is what applies) should not be 255, but should instead be something else, e.g., 64.

After you remove that file, and before you reboot, does anything happen with fan speed when you do this?
sudo echo 64 > /sys/devices/platform/pwm-fan/hwmon/hwmon1/pwm1

Fan went quiet. Then, after rebooting, the .jetsonclocks_fan.txt file returned but now in user home folder.

$ cat ~/.jetsonclocks_fan.txt
active
$ cat /sys/devices/platform/pwm-fan/hwmon/hwmon1/pwm1
255

So that narrows it down a lot. I’m guessing it shouldn’t be quite like that. Run this command again:
sudo echo 64 > /sys/devices/platform/pwm-fan/hwmon/hwmon1/pwm1

Remove the “.jetsonclocks_fan.txt” again. Then run command “jetson_clocks --store”. Reboot, and see if the fan is now working as expected.

Did as you suggested. After reboot, I the fan is back on 255.

$ sudo cat /root/.jetsonclocks_fan.txt
active
$ cat ~/.jetsonclocks_fan.txt
cat: ~/.jetsonclocks_fan.txt: No such file or directory
$ cat /sys/devices/platform/pwm-fan/hwmon/hwmon1/pwm1
255

It sounds like the service “nvfancontrol.service” is overwriting the manual settings. Would someone from NVIDIA be able to comment on:

  • Should nvfancontrol.service be disabled?
  • If not, is nvfancontrol.service causing max fan speed at all times?
  • How to set fans to automatic instead of maximum at all times?

Note that nvfancontrol.service is new to the R34.x+ (which is what Orin uses), but did not exist in previous releases (so I don’t know the correct way to configure this; echo of a value to the correct “/sys” file is how control is performed, but I don’t know what software is changing it).

Bump. Any NVIDIA people around?

I’ve seen the same with previous DP version of JP5:

Also do you have file /etc/nvfancontrol.conf ? If yes, check that it has FAN_DEFAULT_PROFILE quiet

I have that file, and it had FAN_DEFAULT_PROFILE cool. I changed it and restarted, but the issue remains. I also tried restarting the service, which did not work either.

hello per.edwardsson,

we don’t see issues on latest Jetpack release.
there’s dynamic fan speed control, we don’t see fan stays-on flat out after system boot-up.

please moving to the latest JetPack release version for confirmation,
thanks

Did sudo apt dist-upgrade, upgraded a few packages, then rebooted. Still fans are on at boot. I believe I have the latest JetPack

$ apt list  | grep jetpack

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libnvidia-container0/bionic,stable,now 0.11.0+jetpack arm64 [installed]
nvidia-jetpack-dev/stable 5.0.2-b231 arm64
nvidia-jetpack-runtime/stable 5.0.2-b231 arm64
nvidia-jetpack/stable 5.0.2-b231 arm64