failed to boot Jetson TX2 after flashing JetPack 4.2

After flashing JetPack 4.2 on Jetson TX2 with SDKManager and the host of Ubuntu 16.04, I cannot boot Jetson TX2.

TX2 printed this error log and then froze with the blank screen or the setup page for L4T.

vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
vdd-1v8: voltage operation not allowed
sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)

I see this error as well during boot, but it doesn’t stop boot. Are you sure boot actually failed? The GUI is a very bad indicator of boot, and you might try access via network ssh, or via CTRL-ALT-F2. If you can get in that way, then it means debugging the GUI rather than debugging boot. Even ping of the ethernet would be a good sign the unit is actually booted.

Unlike jetpack3.3 /rel-28 based release, you need to use GUI to configure your username/pwd during the first boot. Otherwise it would hang the terminal.

Where do you see such log? Is it from the UART debugger or the monitor (framebuffer)?

I have accessed Jetson via HDMI, not SSH.

I couldn’t use GUI to configure my account during the first boot,
because Jetson froze whenever I booted Jetson.

I saw the log in the monitor while Jetson was booting, and type it.

I see the “could not set regulator OCR” on R32.1 serial console. This appears for me, but does not seem to prevent normal operation:

[    9.753125] vdd-1v8: voltage operation not allowed
[    9.758007] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.789865] vdd-1v8: voltage operation not allowed
[    9.794743] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.826138] vdd-1v8: voltage operation not allowed
[    9.830987] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.834569] vdd-1v8: voltage operation not allowed
[    9.834581] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.979304] vdd-1v8: voltage operation not allowed
[    9.984317] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.992008] vdd-1v8: voltage operation not allowed
[    9.996927] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.005233] vdd-1v8: voltage operation not allowed
[   10.010137] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.768886] vdd-1v8: voltage operation not allowed
[   10.773804] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.800868] vdd-1v8: voltage operation not allowed
[   10.805743] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.834610] vdd-1v8: voltage operation not allowed
[   10.839489] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.848577] vdd-1v8: voltage operation not allowed
[   10.853411] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)

Ubuntu 18.04.2 LTS x2 ttyS0

x2 login: [   10.971683] vdd-1v8: voltage operation not allowed
[   10.976562] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.984243] vdd-1v8: voltage operation not allowed
[   10.989156] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[   10.997637] vdd-1v8: voltage operation not allowed
[   11.002558] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)

I couldn’t use GUI to configure my account during the first boot,
That is why such error is present. Could you elaborate why you cannot use GUI to configure account?

Because Jetson stops on a black screen or the setting page of L4T whenever I booted it,
I can’t configure my account.

If the first setting page of L4T is shown up, either I can’t move a mouse cursor or it freezes as I click the next button.

Is it a hardware issue?

changjun.lim0,

Could you try different HDMI monitor? Are you able to see mouse on monitor?

If you can see mouse but it freezes, I would suggest to flash OS again.

I can see a mouse cursor on a monitor, however, sometimes the mouse freezes, sometimes I can move it but the setting page stops as I click the next button.

I have already flashed OS again, but the same thing happens.

Would you mind falling back to jetpack3.3 and see if devices can work or not?

I flashed JetPack3.3 on Jetson TX2 but still cannot boot Jetson.

Jetson keeps rebooting with different error messages.

In the middle of the logs, there are suspicious messages.

[42.130737] gk20a 17000000.gp10b: gk20a_fifo_reset_engine: failed to HALT gr pipe
[42.136743] gk20a 17000000.gp10b: gr_gk20a_load_falcon_bind_instblk: arbiter complete timeout
[42.136879] gk20a 17000000.gp10b: gr_gk20a_load_falcon_bind_instblk: arbiter complete timeout

And it reboots after the following messages.

[122.158926] Process swapper/4 (pid:0, stack limit = 0xffffffc1ece9c020)
[122.177264] Call trace:
[122.191218] [<                     (null)>]
[122.207394] [<ffffffc000820a94>] cpuidle_enter_state+0x88/0x2dc
[122.207394] [<ffffffc000820d20>] cpuidle_enter+0x18/0x20
[122.207394] [<ffffffc0000e8354>] call_cpuidle+0x28/0x50
[122.207394] [<ffffffc0000e84f8>] cpu_startup_entry+0x17c/0x340
[122.207394] [<ffffffc00008ee44>] secondary_start_kernel+0x12c/0x164
[122.207394] [<0000000080081acc>] 0x80081acc
[122.309230] ---[ end trace 7045f9268f1d7cbb ]---
[122.335319] Kernel panic - not syncing: Attempted to kill the idle task!
[123.721248] SMP: failed to stop secondary CPUs
[123.789374] Rebooting in 5 seconds..
[130.162193] SMP: failed to stop secondary CPUs

I notice your log start after 42 second.Could you share the full log?
May I know if this is on your custom carrier board or nvidia devkit?

Plugging HDMI should be the fast way to trigger the issue right? If you don’t plug in HDMI, will you see any error?

Since I write the logs from the photo of a monitor, I couldn’t get the full log.
But I will share the part of it.

And I am using Jetson TX2 developer kit by connecting HDMI cable.

[42.130705] NV_PGRAPH_PRI_GPCO_TPCO_TPCCS_TPC_EXCEPTION_EN: 0x3
[42.130737] gk20a 17000000.gp10b: gk20a_fifo_reset_engine: failed to HALT gr pipe
[42.136743] gk20a 17000000.gp10b: gr_gk20a_load_falcon_bind_instblk: arbiter complete timeout
[42.136879] gk20a 17000000.gp10b: gr_gk20a_load_falcon_bind_instblk: arbiter complete timeout
[65.984462] INFO: rcu_preempt detected stalls on CPUs/tasks:
[65.996504] 0-...: (1 GPs behind) idle=f3f/140000000000000/0 softirq=6167/6198 fqs=5251
[66.011286] 3-...: (1 GPs behind) idle=833/140000000000000/0 softirq=5520/5521 fqs=5251
[66.026220] 5-...: (1 GPs behind) idle=c5d/140000000000000/0 softirq=6391/6416 fqs=5251
[66.040902] (detected by 4, t=5252, jiffies, g=1895, c=1894, q=291)
[66.053771] Task dump for CPU 0:
[66.063762] irq/73-gk20a_st R  running task        0   118      2 0x00000002
[66.077828] Call trace:
[66.087246] [<ffffffc0000865b8>] __switch_to+0xa4/0xb0
[66.099456] [<ffffffc0004ca198>] __nvgpu_timeout_expired_msg+0x5c/0x84
[66.113290] [<ffffffc000527e28>] pmu_wait_message_cond+0x98/0x17c
[66.126677] [<ffffffc000563614>] gp10b_load_falcon_ucode+0xf0/0x22c
[66.140329] [<ffffffc00054170c>] gr_gm20b_laod_ctxsw_ucode+0x278/0x568
[66.154290] [<ffffffc000504c10>] gr_gk20b_init_ctxsw+0x20/0x2d4
[66.167681] [<ffffffc00050e6c8>] gk20a_gr_reset+0x5c/0x134
[66.180726] [<ffffffc0004e2ba0>] gk20a_fifo_reset_engine+0x9c/0x154
[66.194668] [<ffffffc0004e344c>] gk20a_fifo_handle_mmu_fault+0x5e0/0xee8
[66.209125] [<ffffffc0004e4050>] gk20a_fifo_recover+0x1dc/0x528
[66.222918] [<ffffffc000956234>] fifo_error_isr+0x4ec/0xa04
[66.236251] [<ffffffc0004e66ec>] gk20a_fifo_isr+0x2b4/0x3a0
[66.249741] [<ffffffc00056017c>] mc_gp10b_intr_thread_stall+0x1e8/0x344
[66.264484] [<ffffffc0004d1634>] gk20a_intr_thread_stall+0x18/0x20
[66.278797] [<ffffffc00000f6728>] irq_thread_fnl+0x28/0x4c
[66.292388] [<ffffffc00000f6a40>] irq_thread+0x130/0x1c4
[66.305905] [<ffffffc00000c1f30>] kthread+0xe0/0xf4
[66.319053] [<ffffffc0000084f90>] ret_from_fork+0x10/0x40
[66.332824] Task dump for CPU 3:
[66.344423] nvgpu_channel_p R  running task        0   996       2 0x00000002
[66.360303] Call trace:
[66.371504] [<ffffffc0000865b8>] __switch_to+0xa4/0xb0
[66.385533] [<ffffffc07a821a00>] 0xffffffc07a821a00
[66.399476] Task dump for CPU 5:
[66.411684] kworker/u12:2    R running task         0   117      2 0x00000002
[66.428091] Workqueue: devfreq_wq devfreq_monitor
[66.442287] Call trace:
[66.454096] [<ffffffc0000865b8>] __switch_to+0xa4/0xb0
[66.468926] [<ffffffc07a8223c0>] 0xffffffc07a8223c0
[121.813524] LR is at t18x_a57_enter_state+0x20/0xc4
[121.831928] pc : [<0000000000000000>] lr : [<ffffffc00094eef4>] pstate: 800000c5
[121.853029] sp : ffffffc1ece9fec0
[121.869918] x29 : ffffffc1ece9fec0 x28: ffffffc1ece9c000
[121.888853] x27 : ffffffc000b87f00 x26: 0000001c528db1c0
[121.907668] x25 : ffffffc00140d000 x24: 0000000000000000
[121.926411] x23 : ffffffc001396b48 x22: ffffffc001396b60
[121.945196] x21 : ffffffc00140d6c8 x20: ffffffc0014d8bb8
[121.963956] x19 : 0000000000000000 x18: 0000000000000032
[121.982678] x17 : 0000007f907d61a8 x16: ffffffc000b88a60
[122.001231] x15 : 00000000fa83b2da x14: 0000000000000000
[122.019687] x13 : 00000000fa83b2da x12: 0000000000011d91
[122.038018] x11 : 0000000000011b91 x10: 00000000000008b0
[122.056371] x9 : ffffffc1ece9fec0 x8: ffffffc1ece83b10
[122.074777] x7 : 0000001c54026300 x6: 000000000000e074
[122.093162] x5 : 0000000000000000 x4: 00ffffffffffffff
[122.111349] x3 : 000000003b9aac00 x2: 00000000003b8aa0
[122.128902] x1 : 0000000000000000 x0: 0000000000000000
[122.145933]
[122.158926] Process swapper/4 (pid:0, stack limit = 0xffffffc1ece9c020)
[122.177264] Call trace:
[122.191218] [<          (null)>]           (null)
[122.207394] [<ffffffc000820a94>] cpuidle_enter_state+0x88/0x2dc
[122.207394] [<ffffffc000820d20>] cpuidle_enter+0x18/0x20
[122.207394] [<ffffffc0000e8354>] call_cpuidle+0x28/0x50
[122.207394] [<ffffffc0000e84f8>] cpu_startup_entry+0x17c/0x340
[122.207394] [<ffffffc00008ee44>] secondary_start_kernel+0x12c/0x164
[122.207394] [<0000000080081acc>] 0x80081acc
[122.309230] ---[ end trace 7045f9268f1d7cbb ]---
[122.335319] Kernel panic - not syncing: Attempted to kill the idle task!
[123.721248] SMP: failed to stop secondary CPUs
[123.789374] Rebooting in 5 seconds..
[130.162193] SMP: failed to stop secondary CPUs

It has very high possibility that it is a hardware issue. We cannot tell anything more here since you can only get debug message from your HDMI monitor… Can’t you ssh to device?

Can Jetson connect to the Internet before finishing booting?

If it is a hardware issue, is there any service that NVIDIA provides?

Hi,

We always use UART debugger to trace the bootloader+ kernel logs. If you cannot find a UART log, I would wonder if this issue is only triggered by HDMI since it is related to GPU driver.

If this can be triggered later, could you check if you could first boot into system when device has no hdmi connected?

USB device mode would automatically set up if using rel-28.2.1 release from jetpack and there would be a usb interface(l4tbr0) with ip 192.168.55.1. You could try to ping this ip or even ssh from your host when micro usb port is connected with host.

There is still a problem to boot even though I used UART debugger.

I have attached the log file.

TX2 boot logs.docx (34.7 KB)

Looks like a hardware issue. Please start the RMA process.