Jetson nano failed to start up

Hi, we caught a problem that two jetson nano module could not start up after several month.

Our customer found that two our product could not start up after several month usage, then they sent them back to us to check.

Below is serial console output:

[0001.167] *** Booting KFS0.
[0001.170] Read GPT from (0:3)
[0001.173] Using GPT Primary to query partitions
[0001.178] Loading Tboot-CPU binary
[0001.185] Verifying TBC in OdmNonSecureSBK mode
[0001.195] Bootloader load address is 0xa0000000, entry address is 0xa0000258
[0001.202] Bootloader downloaded successfully.
[0001.206] Downloaded Tboot-CPU binary to 0xa0000258
[0001.211] MAX77620_GPIO5 configured
[0001.215] CPU power rail is up
[0001.217] CPU clock enabled
[0001.221] Performing RAM repair
[0001.224] Updating A64 Warmreset Address to 0xa00002e9
[0001.229] Loading NvTbootBootloaderDTB
[0001.246] Verifying NvTbootBootloaderDTB in OdmNonSecureSBK mode
[0001.312] Bootloader DTB Load Address: 0x83000000
[0001.317] Loading NvTbootKernelDTB
[0001.333] Verifying NvTbootKernelDTB in OdmNonSecureSBK mode
[0001.399] Kernel DTB Load Address: 0x83100000
[0001.404] Loading cboot binary
[0001.415] Verifying EBT in OdmNonSecureSBK mode
[0001.457] Bootloader load address is 0x92c00000, entry address is 0x92c00258
[0001.464] Bootloader downloaded successfully.
[0001.469] PT: Partition NCT NOT found !
[0001.472] Warning: Find Partition via PT Failed
[0001.477] Next binary entry address: 0x92c00258
[0001.481] BoardId: 3448
[0001.486] Overriding pmu board id with proc board id
[0001.490] Display board id is not available
[0001.502] Verifying SC7EntryFw in OdmNonSecureSBK mode
[0001.554] /bpmp deleted
[0001.557] SC7EntryFw header found loaded at 0xff700000
[0001.746] OVR2 PMIC
[0001.748] Bpmp FW successfully loaded
[0001.753] WB0 init successfully at 0xff780000
[0001.757] Set NvDecSticky Bits
[0001.761] GSC2 address ff53fffc value c0edbbcc
[0001.767] GSC MC Settings done
[0001.771] TOS Image length 53680
[0001.774] Monitor size 53680
[0001.777] OS size 0
[0001.782] Secure Os AES-CMAC Verification Success!
[0001.786] TOS image cipher info: plaintext
[0001.790] Loading and Validation of Secure OS Successful
[0001.806] SC7 Entry Firmware - 0xff700000, 0x4000
[0001.811] NvTbootPackSdramParams: start.
[0001.816] NvTbootPackSdramParams: done.
[0001.820] Tegraboot started after 150276 us
[0001.824] Basic modules init took 1381081 us
[0001.828] Sec Bootdevice Read Time = 544 ms, Read Size = 14459 KB
[0001.834] Sec Bootdevice Write Time = 0 ms, Write Size = 0 KB
[0001.839] Next stage binary read took 7388 us
[0001.843] Carveout took -19289 us
[0001.846] CPU initialization took 344305 us
[0001.850] Total time taken by TegraBoot 1713485 us

[0001.855] Starting CPU & Halting co-processor

64NOTICE: BL31: v1.3(release):41d46a9cf
NOTICE: BL31: Built : 21:18:12, Jun 25 2020
ERROR: Error initializing runtime service trusty_fast
[0001.977] RamCode = 0
[0001.982] LPDDR4 Training: Read DT: Number of tables = 2
[0001.987] EMC Training (SRC-freq: 204000; DST-freq: 1600000)
[0002.000] EMC Training Successful
[0002.003] 408000 not found in DVFS table
[0002.009] RamCode = 0
[0002.013] DT Write: emc-table@204000 succeeded
[0002.018] DT Write: emc-table@1600000 succeeded
[0002.023] LPDDR4 Training: Write DT: Number of tables = 2
[0002.080]
[0002.081] Debug Init done
[0002.084] Marked DTB cacheable
[0002.086] Bootloader DTB loaded at 0x83000000
[0002.091] Marked DTB cacheable
[0002.094] Kernel DTB loaded at 0x83100000
[0002.098] DeviceTree Init done
[0002.111] Pinmux applied successfully
[0002.115] gicd_base: 0x50041000
[0002.119] gicc_base: 0x50042000
[0002.122] Interrupts Init done
[0002.126] Using base:0x60005090 & irq:208 for tick-timer
[0002.131] Using base:0x60005098 for delay-timer
[0002.136] platform_init_timer: DONE
[0002.139] Timer(tick) Init done
[0002.143] osc freq = 38400 khz
[0002.147]
[0002.148] Welcome to L4T Cboot
[0002.151]
[0002.152] Cboot Version: 00.00.2018.01-t210-3a7ad565
[0002.157] calling constructors
[0002.160] initializing heap
[0002.163] initializing threads
[0002.166] initializing timers
[0002.169] creating bootstrap completion thread
[0002.173] top of bootstrap2()
[0002.176] CPU: ARM Cortex A57
[0002.179] CPU: MIDR: 0x411FD071, MPIDR: 0x80000000
[0002.184] initializing platform
[0002.237] Config for emmc ddr50 mode completed
[0002.242] sdmmc bdev is already initialized
[0002.246] Enable APE clock
[0002.248] Un-powergate APE partition
[0002.252] of_register: registering tegra_udc to of_hal
[0002.257] of_register: registering inv20628-driver to of_hal
[0002.263] of_register: registering ads1015-driver to of_hal
[0002.268] of_register: registering lp8557-bl-driver to of_hal
[0002.274] of_register: registering bq2419x_charger to of_hal
[0002.279] of_register: registering bq27441_fuel_gauge to of_hal
[0002.291] gpio framework initialized
[0002.294] of_register: registering tca9539_gpio to of_hal
[0002.300] of_register: registering tca9539_gpio to of_hal
[0002.305] of_register: registering i2c_bus_driver to of_hal
[0002.311] of_register: registering i2c_bus_driver to of_hal
[0002.316] of_register: registering i2c_bus_driver to of_hal
[0002.322] pmic framework initialized
[0002.325] of_register: registering max77620_pmic to of_hal
[0002.331] regulator framework initialized
[0002.335] of_register: registering tps65132_bl_driver to of_hal
[0002.341] initializing target
[0002.346] gpio_driver_register: register ‘tegra_gpio_driver’ driver
[0002.357] fixed regulator driver initialized
[0002.375] initializing OF layer
[0002.378] NCK carveout not present
[0002.381] Skipping dts_overrides
[0002.385] of_children_init: Ops found for compatible string nvidia,tegra210-i2c
[0002.402] I2C Bus Init done
[0002.405] of_children_init: Ops found for compatible string nvidia,tegra210-i2c
[0002.415] I2C Bus Init done
[0002.418] of_children_init: Ops found for compatible string nvidia,tegra210-i2c
[0002.428] I2C Bus Init done
[0002.431] of_children_init: Ops found for compatible string nvidia,tegra210-i2c
[0002.441] I2C Bus Init done
[0002.444] of_children_init: Ops found for compatible string nvidia,tegra210-i2c
[0002.454] I2C Bus Init done
[0002.457] of_children_init: Ops found for compatible string maxim,max77620
[0002.467] max77620_init using irq 118
[0002.472] register ‘maxim,max77620’ pmic
[0002.476] gpio_driver_register: register ‘max77620-gpio’ driver
[0002.482] of_children_init: Ops found for compatible string nvidia,tegra210-i2c
[0002.493] I2C Bus Init done
[0002.496] NCK carveout not present
[0002.506] Find /i2c@7000c000’s alias i2c0
[0002.510] get eeprom at 1-a0, size 256, type 0
[0002.519] Find /i2c@7000c500’s alias i2c2
[0002.523] get eeprom at 3-a0, size 256, type 0
[0002.527] get eeprom at 3-ae, size 256, type 0
[0002.532] pm_ids_update: Updating 1,a0, size 256, type 0
[0002.537] I2C slave not started
[0002.540] I2C write failed
[0002.543] Writing offset failed
[0002.546] eeprom_init: EEPROM read failed
[0002.550] pm_ids_update: eeprom init failed
[0002.554] pm_ids_update: Updating 3,a0, size 256, type 0
[0002.584] pm_ids_update: The pm board id is 3448-0002-401
[0002.591] Adding plugin-manager/ids/3448-0002-401=/i2c@7000c500:module@0x50
[0002.598] pm_ids_update: pm id update successful
[0002.603] pm_ids_update: Updating 3,ae, size 256, type 0
[0002.608] I2C slave not started
[0002.611] I2C write failed
[0002.614] Writing offset failed
[0002.617] eeprom_init: EEPROM read failed
[0002.621] pm_ids_update: eeprom init failed
[0002.651] eeprom_get_mac: EEPROM invalid MAC address (all 0xff)
[0002.657] shim_eeprom_update_mac:267: Failed to update 0 MAC address in DTB
[0002.665] eeprom_get_mac: EEPROM invalid MAC address (all 0xff)
[0002.671] shim_eeprom_update_mac:267: Failed to update 1 MAC address in DTB
[0002.680] updating /chosen/nvidia,ethernet-mac node 3c:6d:66:00:5f:2a
[0002.686] Plugin Manager: Parse ODM data 0x00094000
[0002.696] shim_cmdline_install: /chosen/bootargs: earlycon=uart8250,mmio32,0x70006000
[0002.711] Find /i2c@7000c000’s alias i2c0
[0002.715] get eeprom at 1-a0, size 256, type 0
[0002.724] Find /i2c@7000c500’s alias i2c2
[0002.728] get eeprom at 3-a0, size 256, type 0
[0002.732] get eeprom at 3-ae, size 256, type 0
[0002.736] pm_ids_update: Updating 1,a0, size 256, type 0
[0002.742] I2C slave not started
[0002.745] I2C write failed
[0002.747] Writing offset failed
[0002.750] eeprom_init: EEPROM read failed
[0002.754] pm_ids_update: eeprom init failed
[0002.758] pm_ids_update: Updating 3,a0, size 256, type 0
[0002.789] pm_ids_update: The pm board id is 3448-0002-401
[0002.795] Adding plugin-manager/ids/3448-0002-401=/i2c@7000c500:module@0x50
[0002.802] pm_ids_update: pm id update successful
[0002.807] pm_ids_update: Updating 3,ae, size 256, type 0
[0002.812] I2C slave not started
[0002.815] I2C write failed
[0002.818] Writing offset failed
[0002.821] eeprom_init: EEPROM read failed
[0002.825] pm_ids_update: eeprom init failed
[0002.855] Add serial number:1422024328158 as DT property
[0002.863] Applying platform configs
[0002.869] platform-init is not present. Skipping
[0002.874] calling apps_init()
[0002.890] Found 17 GPT partitions in “sdmmc3_user”
[0002.894] Proceeding to Cold Boot
[0002.897] starting app android_boot_app
[0002.901] Device state: unlocked
[0002.904] display console init
[0002.913] could not find regulator
[0002.916] hdmi cable connected
[0002.919] I2C slave not started
[0002.922] I2C write failed
[0002.925] Writing offset failed
[0002.928] could not read edid
[0002.933] tmds-config node not found
[0002.937] pmc_set_io_pad_voltage: Error -2 retrieving platform-io-pad-voltagepropsetting ‘avdd-io-hdmi-dp’ regulator td
v[0002.955] olts
[0002.959] setting ‘vdd-1v8’ regulator to 1800000 micro volts
[0002.966] could not find regulator
[0002.969] could not find regulator
[0002.973] could not find regulator
[0003.001] using default cmu settings
[0003.005] dc_hdmi_enable, starting HDMI initialisation
[0003.011] dc_hdmi_enable, HDMI initialisation complete
[0003.016] list and configure display window
[0003.021] display console init completed
[0003.028] subnode volume_up is not found !
[0003.032] subnode back is not found !
[0003.036] subnode volume_down is not found !
[0003.040] subnode menu is not found !
[0003.044] Gpio keyboard init success
[0003.285] decompressor handler not found
[0003.288] load_bmp_blob: panelresolution=480 type=0
[0003.338] decompressor handler not found
[0003.342] load_firmware_blob: Firmware blob loaded, entries=2
[0003.348] -------> se_aes_verify_sbk_clear: 747
[0003.352] se_aes_verify_sbk_clear: Error
[0003.356] bl_battery_charging: connected to external power supply
[0003.364] xusb is supported
[0003.370] error while finding nvidia,portmap
[0003.874] xusb blob version 0 size 124416
[0003.878] firmware size 124416
[0003.882] Firmware timestamp: 0x5da88fc3, Version: 50.25 release
[0003.899] xhci0: 64 bytes context size, 32-bit DMA
[0003.939] usbus0: 5.0Gbps Super Speed USB v3.0
[0003.959] uhub0: <Nvidia XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
[0004.609] uhub0: 9 ports with 9 removable, self powered
[0005.609] failed to get HID devices
[0005.612] failed to init xhci or no usb device attached
[0005.626] device_query_partition_size: failed to open partition sdmmc3_user:MSC !
[0005.633] MSC Partition not found
[0005.643] device_query_partition_size: failed to open partition sdmmc3_user:USP !
[0005.650] USP partition read failed!
[0005.654] blob_init: blob-partition USP header read failed
[0005.659] android_boot Unable to update recovery partition
[0005.665] kfs_getpartname: name = LNX
[0005.668] Loading kernel from LNX
[0005.732] load kernel from storage
[0005.742] decompressor handler not found
[0005.761] Successfully loaded kernel and ramdisk images
[0005.768] load_bmp_blob: panelresolution=480 type=0
[0005.841] display bmp image done
[0005.844] NCK carveout not present
[0005.848] Skipping dts_overrides
[0005.851] NCK carveout not present
[0005.861] Find /i2c@7000c000’s alias i2c0
[0005.865] get eeprom at 1-a0, size 256, type 0
[0005.873] Find /i2c@7000c500’s alias i2c2
[0005.877] get eeprom at 3-a0, size 256, type 0
[0005.882] get eeprom at 3-ae, size 256, type 0
[0005.886] pm_ids_update: Updating 1,a0, size 256, type 0
[0005.891] I2C slave not started
[0005.894] I2C write failed
[0005.897] Writing offset failed
[0005.900] eeprom_init: EEPROM read failed
[0005.904] pm_ids_update: eeprom init failed
[0005.908] pm_ids_update: Updating 3,a0, size 256, type 0
[0005.938] pm_ids_update: The pm board id is 3448-0002-401
[0005.945] Adding plugin-manager/ids/3448-0002-401=/i2c@7000c500:module@0x50
[0005.953] pm_ids_update: pm id update successful
[0005.957] pm_ids_update: Updating 3,ae, size 256, type 0
[0005.962] I2C slave not started
[0005.965] I2C write failed
[0005.968] Writing offset failed
[0005.971] eeprom_init: EEPROM read failed
[0005.975] pm_ids_update: eeprom init failed
[0006.005] eeprom_get_mac: EEPROM invalid MAC address (all 0xff)
[0006.011] shim_eeprom_update_mac:267: Failed to update 0 MAC address in DTB
[0006.019] eeprom_get_mac: EEPROM invalid MAC address (all 0xff)
[0006.025] shim_eeprom_update_mac:267: Failed to update 1 MAC address in DTB
[0006.034] updating /chosen/nvidia,ethernet-mac node 3c:6d:66:00:5f:2a
[0006.040] Plugin Manager: Parse ODM data 0x00094000
[0006.050] shim_cmdline_install: /chosen/bootargs: earlycon=uart8250,mmio32,0x70006000
[0006.058] Add serial number:1422024328158 as DT property
[0006.067] “bpmp” doesn’t exist, creating
[0006.073] Updated bpmp info to DTB
[0006.078] Updated initrd info to DTB
[0006.081] “proc-board” doesn’t exist, creating
[0006.087] Updated board info to DTB
[0006.090] “pmu-board” doesn’t exist, creating
[0006.096] Updated board info to DTB
[0006.099] “display-board” doesn’t exist, creating
[0006.105] Updated board info to DTB
[0006.108] “reset” doesn’t exist, creating
[0006.113] Updated reset info to DTB
[0006.116] Cmdline: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane
[0006.155] DTB cmdline: earlycon=uart8250,mmio32,0x70006000
[0006.160] boot image cmdline: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbco
[0006.176] Updated bootarg info to DTB
[0006.180] Adding uuid 00000001643df3c52000000014020440 to DT
[0006.186] Adding eks info 0 to DT
[0006.192] WARNING: Failed to pass NS DRAM ranges to TOS, err: -7
[0006.197] Updated memory info to DTB
[0006.206] set vdd_core voltage to 1075 mv
[0006.210] setting ‘vdd-core’ regulator to 1075000 micro volts
[0006.229] Found secure-pmc; disable BPMP

U-Boot 2016.07-ge6da093be3 (Jun 25 2020 - 21:18:08 -0700)

TEGRA210
Model: NVIDIA P3450-Porg
Board: NVIDIA P3450-PORG
DRAM: 4 GiB
MMC: Tegra SD/MMC: 0, Tegra SD/MMC: 1
SF: Failed to get idcodes
*** Warning - spi_flash_probe() failed, using default environment

In: serial
Out: serial
Err: serial
Net: No ethernet found.
Card did not respond to voltage select!
** Bad device mmc 1 **
Hit any key to stop autoboot: 0
Card did not respond to voltage select!
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1…
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
989 bytes read in 156 ms (5.9 KiB/s)
1: primary kernel
Retrieving file: /boot/initrd
5488163 bytes read in 175 ms (29.9 MiB/s)
Retrieving file: /boot/Image
34334728 bytes read in 810 ms (40.4 MiB/s)
append: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=

Flattened Device Tree blob at 83100000

Booting using the fdt blob at 0x83100000
reserving fdt memory region: addr=80000000 size=20000
Using Device Tree in place at 0000000083100000, end 000000008317794a

Starting kernel …

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.9.140-tegra (jenkins@name) (gcc version 7.5.0 (Linaro GCC 7.5-2019.12) ) #0 SMP PREEMPT 3
[ 0.000000] Boot CPU: AArch64 Processor [411fd071]
[ 0.000000] OF: fdt:memory scan node memory@80000000, reg size 48,
[ 0.000000] OF: fdt: - 80000000 , 7ee00000
[ 0.000000] OF: fdt: - 100000000 , 7f200000
[ 0.000000] Found tegra_fbmem: 00140000@92cb0000
[ 0.000000] earlycon: uart8250 at MMIO32 0x0000000070006000 (options ‘’)
[ 0.000000] bootconsole [uart8250] enabled
[ 1.091929] adxl34x 0-0053: Failed to probe ADXL34x accelerometer
[ 1.254783] imx219 7-0010: imx219_board_setup: error during i2c read probe (-121)
[ 1.262315] imx219 7-0010: board setup failed
[ 1.290461] imx219 8-0010: imx219_board_setup: error during i2c read probe (-121)
[ 1.297985] imx219 8-0010: board setup failed
[ 1.309639] pwm_fan_driver pwm-fan: fan tach gpio request failed
[ 1.752784] JBD2: Invalid checksum recovering block 218 in log
[ 1.759890] EXT4-fs (mmcblk0p1): error loading journal

This is the first time we have seen this kind of output msg.

What is the reason to cause the problem?

Can we avoid it or can it be fixed except for reflashing the entire img?

Thanks.

Hi, is there any suggestions?

Please refer to Debug tips when you hit any issue (fail to boot/ keep reboot …etc) - Jetson & Embedded Systems / Jetson Nano - NVIDIA Developer Forums to see if can help.

Thanks

Above error told your rootfs seems corrupted.

Thanks for your reply.
We use jetson nano emmc module with custom carrier board and jetpack 4.4 version.
We have not been able to reproduce the issue so far because it is a problem with a very low probability of occurrence.

Thanks for your reply.

According to the serial console output message, we have learned some knowledge of jbd2.

Theoretically the ext4 filesystem can recover by jbd2.

Now it failed to recover, is it possible due to partition of log for jbd2 corrupted?