Jetson nano 4GB starts booting very slowly about 20 minutes

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,

strictly speaking, Nano boots up very fast - about 10 seconds, according to systemd-analyze : Startup finished in 6.479s (kernel) + 4.401s (userspace) = 10.880s

But I connecting to Nano with ssh and I wait for connection about 20 minutes. This occures sometimes, for example, after command “sudo reboot” Nano boots up fast.

Thank you.

3

I would suggest you can monitor the serial console log when such case happens and check the status of ethernet connection.

4

journalctl -u NetworkManager says:

авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.5283] dhcp4 (eth0): dhclient started with pid 5588
авг 09 10:12:37 vova1 dhclient[5588]: DHCPREQUEST of 192.168.137.33 on eth0 to 255.255.255.255 port 67 (xid=0x52495277)
авг 09 10:12:37 vova1 dhclient[5588]: DHCPACK of 192.168.137.33 from 192.168.137.1
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8544] dhcp4 (eth0): address 192.168.137.33
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8545] dhcp4 (eth0): plen 24 (255.255.255.0)
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8546] dhcp4 (eth0): gateway 192.168.137.1
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8546] dhcp4 (eth0): lease time 25200
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8547] dhcp4 (eth0): nameserver ‘192.168.137.1’
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8548] dhcp4 (eth0): state changed unknown → bound
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8587] device (eth0): state change: ip-config → ip-check
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8607] device (eth0): state change: ip-check → secondari
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8614] device (eth0): state change: secondaries → activa
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8618] manager: NetworkManager state is now CONNECTED_LOC
авг 09 10:12:37 vova1 dhclient[5588]: bound to 192.168.137.33 – renewal in 9916 seconds.
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8716] manager: NetworkManager state is now CONNECTED_SIT
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8719] policy: set ‘Wired connection 1’ (eth0) as default
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8731] device (eth0): Activation: successful, device acti
авг 09 10:12:37 vova1 NetworkManager[4012]: [1628493157.8752] manager: startup complete
авг 09 10:12:39 vova1 NetworkManager[4012]: [1628493159.1654] manager: NetworkManager state is now CONNECTED_GLO
авг 09 10:12:40 vova1 NetworkManager[4012]: [1628493160.6652] bluez: use BlueZ version 5
авг 09 10:22:28 vova1 NetworkManager[4012]: [1628493748.5908] connectivity: (eth0) timed out
авг 09 10:22:28 vova1 NetworkManager[4012]: [1628493748.5920] manager: NetworkManager state is now CONNECTED_SIT
авг 09 10:26:59 vova1 NetworkManager[4012]: [1628494019.9349] manager: NetworkManager state is now CONNECTED_GLO

notice time skipping in the end of the passage

5

“you can monitor the serial console log” & “check the status of ethernet connection” - could you supply more details how to do this, please

6

https://forums.developer.nvidia.com/t/debug-tips-when-you-hit-any-kind-of-system-issue-fail-to-boot-keep-reboot-etc/177457/2

7

syslog (686.4 KB)
syslog …

8

Hi,

Can you share the timestamp when the error happens?

9

I suppose at line 6655 in syslog:

Aug 9 10:22:28 vova1 NetworkManager[4012]: [1628493748.5908] connectivity: (eth0) timed out

10

please wait an hour - I will make serial console connector

11

could not connect to serial console, 2 device been tried and 2 cables

[ 2882.714953] usb usb1-port9: Cannot enable. Maybe the USB cable is bad?
[ 2882.840281] usb 1-9: new low-speed USB device number 107 using xhci_hcd
[ 2883.660275] usb 1-9: device descriptor read/64, error -71
[ 2883.896323] usb 1-9: device descriptor read/64, error -71
[ 2884.004340] usb usb1-port9: attempt power cycle
[ 2884.416270] usb 1-9: new low-speed USB device number 108 using xhci_hcd
[ 2884.416749] usb 1-9: Device not responding to setup address.
[ 2884.624695] usb 1-9: Device not responding to setup address.
[ 2884.832268] usb 1-9: device not accepting address 108, error -71
[ 2885.168282] usb 1-9: new low-speed USB device number 109 using xhci_hcd
[ 2885.168691] usb 1-9: Device not responding to setup address.
[ 2885.376270] usb 1-9: Device not responding to setup address.
[ 2885.584258] usb 1-9: device not accepting address 109, error -71
[ 2885.584345] usb usb1-port9: unable to enumerate USB device
[ 2885.856258] usb 1-9: new low-speed USB device number 110 using xhci_hcd
[ 2886.192278] usb 1-9: device descriptor read/64, error -71
[ 2886.636265] usb 1-9: device descriptor read/64, error -71
[ 2886.872250] usb 1-9: new low-speed USB device number 111 using xhci_hcd
[ 2887.004259] usb 1-9: device descriptor read/64, error -71
[ 2887.448257] usb 1-9: device descriptor read/64, error -71
[ 2887.556329] usb usb1-port9: attempt power cycle
[ 2887.972247] usb 1-9: new low-speed USB device number 112 using xhci_hcd
[ 2887.972694] usb 1-9: Device not responding to setup address.
[ 2888.180231] usb 1-9: Device not responding to setup address.
[ 2888.388197] usb 1-9: device not accepting address 112, error -71
[ 2888.724219] usb 1-9: new low-speed USB device number 113 using xhci_hcd
[ 2888.724728] usb 1-9: Device not responding to setup address.
[ 2888.932756] usb 1-9: Device not responding to setup address.
[ 2889.140225] usb 1-9: device not accepting address 113, error -71
[ 2889.140313] usb usb1-port9: unable to enumerate USB device

12

How do you set up the console? The serial console to jetson is a UART interface, you shall not see usb log.

13

yes, on a jetson I connect to UART pins, on the host machine I connect USB slot, to use minicom to emulalete UART via USB, as far as I understand. This log is from host machine, not from Nano device. I use tutorial from Jetson Nano - Serial Console - YouTube

14

Then maybe try different host. Our native test environment is using ubuntu 18.04.

15

my ubuntu has the same version, and unfortunately I could not reach another host right now, I will try change USB driver and so far reinstall ubuntu.

16

Nano is rebooting many times within one boot, see serial console log.

minicom.txt (1.1 MB)

17

This has nothing to do with your original topic.

Please try to reflash the board first.

18

why I should do reflash? this is working unit from robot and it must work years, not two monthes

it has direct relation to my original topic - nano continues rebooting many times and can not stop rebooting - endless rebooting

may be I did not understand you?

19

Hi,

If this is what you see all the time and you have to wait for 20 mins to let the board gets boot ready, then I have to say it has chance your module has hardware defect.

It has kernel panic from the gpu in every boot.

[ 9.930960] Hardware name: NVIDIA Jetson Nano Developer Kit (DT)
[ 9.936954] Call trace:
[ 9.939400] [] dump_backtrace+0x0/0x198
[ 9.944789] [] show_stack+0x24/0x30
[ 9.949831] [] dump_stack+0xa0/0xc8
[ 9.954873] [] panic+0x12c/0x2a8
[ 9.959656] [] nvhost_scale_emc_debug_init.isra.12+0x128/0x1a0
[ 9.967039] [] nvhost_pod_event_handler+0x334/0x400
[ 9.973468] [] devfreq_add_device+0x284/0x408
[ 9.979376] [] devm_devfreq_add_device+0x64/0xc0
[ 9.985767] [] gk20a_scale_init+0xf0/0x190 [nvgpu]
[ 9.992339] [] gk20a_pm_finalize_poweron+0x370/0x400 [nvgpu]
[ 9.999740] [] gk20a_busy+0x1b8/0x4f0 [nvgpu]
[ 10.005651] [] pm_generic_runtime_resume+0x3c/0x58
[ 10.011991] [] __rpm_callback+0x74/0xa0
[ 10.017377] [] rpm_callback+0x34/0x98
[ 10.022590] [] rpm_resume+0x470/0x710
[ 10.027805] [] __pm_runtime_resume+0x4c/0x70
[ 10.033924] [] gk20a_busy+0xd4/0x4f0 [nvgpu]
[ 10.040054] [] gk20a_ctrl_dev_open+0x8c/0x168 [nvgpu]
[ 10.046663] [] chrdev_open+0x94/0x198
[ 10.051882] [] do_dentry_open+0x1d8/0x340
[ 10.057451] [] vfs_open+0x58/0x88
[ 10.062329] [] do_last+0x530/0xfd0
[ 10.067292] [] path_openat+0x90/0x378
[ 10.072517] [] do_filp_open+0x70/0xe8
[ 10.077747] [] do_sys_open+0x174/0x258
[ 10.083068] [] SyS_openat+0x3c/0x50
[ 10.088128] [] el0_svc_naked+0x34/0x38
[ 10.093455] SMP: stopping secondary CPUs
[ 10.097410] Kernel Offset: disabled

Such error should not happen on a fine module.

Your original topic sounds like you know the board is booting up but you just cannot ssh to it. In that case, it is a ethernet problem, but current log shows totally not related to that at all.

20

I sad systemd-analyze reported that booting was 10 seconds long.) Clear for me. Fucki@ china)

Thank you a lot

21

Hello again.
Error dissapears when I remove from crontab launching at the booting time my face recognition program - dlib with cuda. My program works fine when I start it manually.

Let me guess. Cuda is not yet initialized?