New failure booting from USB flash drive on Xavier NX

I’ve set up our Xavier NX to boot from a USB flash drive. It has worked fine like this for several months. However, this morning at 1 am during a regularly scheduled reboot, the Jetson failed to boot from the USB drive and instead booted from the emmc. I ssh’ed into the Jetson and I see that the drive is at least recognized by the Jetson:

$ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0          7:0    0    16M  1 loop 
sda            8:0    1 119.5G  0 disk 
└─sda1         8:1    1  55.9G  0 part 
mtdblock0     31:0    0    32M  0 disk 
mmcblk0      179:0    0  14.7G  0 disk 
├─mmcblk0p1  179:1    0    14G  0 part /
├─mmcblk0p2  179:2    0    64M  0 part 
....

I rebooted the Jetson and again it rebooted from the emmc instead of the USB drive.
We have been running 5 Jetsons to boot from the USB drive since February and this is the 3rd time we have encountered this failure. In the past, to work around this failure, we’ve installed a new, formatted, configured USB drive into the same USB port of the Jetson and the Jetson boots from the new USB drive.

I’m wondering if others have experienced this type of failure and also how to diagnose why the failure occurs.

Which jetpack version in use here?

Any uart log to share?

We are using r32.6.1, which I believe corresponds to JetPack 4.6. I will try to get a uart log to share a little later.

I am working on this problem as well - but I have been unable to find in the documentation WHICH uart I should be connecting to to get early boot-time logs; the one exposed on the development host when connected to the carrier board’s USB (which the Jetson Xavier sees as “ttyGS0”) does not show anything until the Xavier is fully booted. Is there another UART we should be connecting to which will get us bootloader debugging? That would be helpful for this and other problems…

never mind; I discovered the correct UART through trial and error. Here is the console log:

[0000.025] W> RATCHET: MB1 binary ratchet value 4 is too large than ratchet level 2 from HW fuses.
[0000.034] I> MB1 (prd-version: 1.5.1.7-t194-41334769-98030a79)
[0000.039] I> Boot-mode: Coldboot
[0000.042] I> Chip revision : A02P
[0000.045] I> Bootrom patch version : 15 (correctly patched)
[0000.050] I> ATE fuse revision : 0x200
[0000.054] I> Ram repair fuse : 0x0
[0000.057] I> Ram Code : 0x0
[0000.059] I> rst_source : 0x0
[0000.062] I> rst_level : 0x0
[0000.066] I> Boot-device: QSPI
[0000.068] I> Qspi flash params source = brbct
[0000.072] I> Qspi using bpmp-dma
[0000.075] I> Qspi clock source : pllp
[0000.079] I> QSPI Flash Size = 32 MB
[0000.082] I> Qspi initialized successfully
[0000.086] W> No valid slot number is found in scratch register
[0000.092] W> Return default slot: _a
[0000.095] I> Active Boot chain : 0
[0000.098] I> Boot-device: QSPI
[0000.101] I> Qspi flash params source = brbct
[0000.107] W> MB1_PLATFORM_CONFIG: device prod data is empty in MB1 BCT.
[0000.114] I> Temperature = 31500
[0000.117] W> Skipping boost for clk: BPMP_CPU_NIC
[0000.122] W> Skipping boost for clk: BPMP_APB
[0000.126] W> Skipping boost for clk: AXI_CBB
[0000.130] W> Skipping boost for clk: AON_CPU_NIC
[0000.134] W> Skipping boost for clk: CAN1
[0000.138] W> Skipping boost for clk: CAN2
[0000.142] I> Boot-device: QSPI
[0000.145] I> Boot-device: QSPI
[0000.148] I> Qspi flash params source = mb1bct
[0000.152] I> Qspi using bpmp-dma
[0000.155] I> Qspi clock source : pllc_out0
[0000.158] I> Qspi reinitialized
[0000.161] I> Qspi flash params source = mb1bct
[0000.167] I> ECC region[0]: Start:0x0, End:0x0
[0000.171] I> ECC region[1]: Start:0x0, End:0x0
[0000.175] I> ECC region[2]: Start:0x0, End:0x0
[0000.179] I> ECC region[3]: Start:0x0, End:0x0
[0000.183] I> ECC region[4]: Start:0x0, End:0x0
[0000.187] I> Non-ECC region[0]: Start:0x80000000, End:0x100000000
[0000.193] I> Non-ECC region[1]: Start:0x0, End:0x0
[0000.198] I> Non-ECC region[2]: Start:0x0, End:0x0
[0000.202] I> Non-ECC region[3]: Start:0x0, End:0x0
[0000.207] I> Non-ECC region[4]: Start:0x0, End:0x0
[0000.212] E> FAILED: Thermal config
[0000.219] E> FAILED: MEMIO rail config
[0000.229] I> Boot-device: QSPI
[0000.232] I> Qspi flash params source = mb1bct
[0000.241] I> Qspi flash params source = mb1bct
[0000.253] I> Qspi flash params source = mb1bct
[0000.319] I> Qspi flash params source = mb1bct
[0000.328] I> Qspi flash params source = mb1bct
[0000.357] I> Qspi flash params source = mb1bct
[0000.369] I> MB1 done

����main enter
SPE VERSION #: R01.00.14 Created: Sep 19 2018 @ 11:03:21
HW Function test
Start Scheduler.
in late init
��
  [0000.377] I> Welcome to MB2(TBoot-BPMP) (version: 00.00.2018.32-mobile-938a8cd8)
[0000.378] I> DMA Heap @ [0x526fa000 - 0x52ffa000]
[0000.378] I> Default Heap @ [0xd486400 - 0xd48a400]
[0000.379] E> DEVICE_PROD: Invalid value data = 70020000, size = 0.
[0000.385] W> device prod register failed
[0000.389] I> Boot-device: QSPI
[0000.391] I> Boot_device: QSPI_FLASH instance: 0
[0000.397] I> QSPI Flash Size = 32 MB
[0000.402] I> Qspi initialized successfully
[0000.403] I> qspi flash-0 params source = boot args
[0000.788] I> sdmmc DDR50 mode
[0000.790] I> sdmmc-3 params source = safe params
[0000.795] I> Found 47 partitions in QSPI_FLASH (instance 0)
[0000.797] W> Cannot find any partition table for 00000003
[0000.801] I> Found 11 partitions in SDMMC_USER (instance 3)
[0000.802] W> No valid slot number is found in scratch register
[0000.803] W> Return default slot: _a
[0000.803] I> Active Boot chain : 0
[0000.804] I> parsing oem signed section of bpmp-fw header done
[0000.805] I> bpmp-fw binary init read from storage
[0000.809] I> oem authentication of bpmp-fw header done
[0000.823] I> bpmp-fw binary done read from storage
[0000.824] I> bpmp-fw: Authentication init Done
[0000.825] I> parsing oem signed section of cpubl header done
[0000.828] I> cpubl binary init read from storage
[0000.833] I> bpmp-fw: Authentication Finalize Done
[0000.838] I> oem authentication of cpubl header done
[0000.845] I> cpubl binary done read from storage
[0000.847] I> cpubl: Authentication init Done
[0000.851] I> parsing oem signed section of rce header done
[0000.856] I> rce binary init read from storage
[0000.861] I> Relocating BR-BCT
[0000.863] I> cpubl: Authentication Finalize Done
[0000.868] I> oem authentication of rce header done
[0000.873] I> rce binary done read from storage
[0000.877] I> rce: Authentication init Done
[0000.881] I> parsing oem signed section of ape header done
[0000.886] I> ape binary init read from storage
[0000.890] I> rce: Authentication Finalize Done
[0000.895] I> oem authentication of ape header done
[0000.899] I> ape binary done read from storage
[0000.903] I> ape: Authentication init Done
[0000.908] I> parsing oem signed section of tos header done
[0000.913] I> tos binary init read from storage
[0000.917] I> ape: Authentication Finalize Done
[0000.922] I> oem authentication of tos header done
[0000.928] I> tos binary done read from storage
[0000.930] I> tos: Authentication init Done
[0000.934] I> parsing oem signed section of bpmp-fw-dtb header done
[0000.940] I> bpmp-fw-dtb binary init read from storage
[0000.945] I> tos: Authentication Finalize Done
[0000.951] I> oem authentication of bpmp-fw-dtb header done
[0000.955] I> bpmp-fw-dtb binary done read from storage
[0000.960] I> bpmp-fw-dtb: Authentication init Done
[0000.965] I> parsing oem signed section of cpubl-dtb header done
[0000.970] I> cpubl-dtb binary init read from storage
[0000.975] I> bpmp-fw-dtb: Authentication Finalize Done
[0001.027] I> oem authentication of cpubl-dtb header done
[0001.027] I> cpubl-dtb binary done read from storage
[0001.028] I> cpubl-dtb: Authentication init Done
[0001.029] I> parsing oem signed section of eks header done
[0001.030] I> eks binary init read from storage
[0001.030] I> cpubl-dtb: Authentication Finalize Done
[0001.031] I> oem authentication of eks header done
[0001.035] I> eks binary done read from storage
[0001.040] I> eks: Authentication init Done
[0001.044] I> eks: Authentication Finalize Done
[0001.048] I> EKB detected (length: 0x410) @ VA:0x5270a400
��NOTICE:  BL31: v1.3(release):b5eeb33f7
NOTICE:  BL31: Built : 12:15:32, Jul 26 2021
ipc-unittest-main: 1519: Welcome to IPC unittest!!!
ipc-unittest-main: 1531: waiting forever
ipc-unittest-srv: 329: Init unittest services!!!
hwkey-agent: 41: hwkey-agent is running!!
hwkey-agent: 315: key_mgnt_processing .......
hwkey-agent: 223: Setting EKB key 0 to slot 14
hwkey-agent: 178: Init hweky-agent services!!
luks-srv: 40: luks-srv is running!!
luks-srv: 157: Init luks-srv IPC services!!
platform_bootstrap_epilog: trusty bootstrap complete
��

welcome to lk
calling constructors
initializing heap
creating bootstrap completion thread
top of bootstrap2()
initializing platform
bpmp: platform_init
tag is e73a758761f0c6d24a1e69a2ac6b5035
tag_show initialized
dt initialized
mail initialized
chipid initialized
fuse initialized
sku initialized
speedo initialized
ec_get_ec_list: found 45 ecs
ec initialized
ec_mrq initialized
vmon_populate_monitors: found 3 monitors
vmon initialized
adc initialized
fmon_populate_monitors: found 73 monitors
fmon initialized
fmon_mrq initialized
reset initialized
nvhs initialized
391 clocks registered
clk_mrq_init: mrq handler registered
clk initialized
nvlink initialized
io_dpd initialized
io_dpd initialized
thermal initialized
i2c5 controller initialized
initialized i2c mrq handling
i2c initialized
regulator initialized
avfs_clk_platform_init: bad clk id in clock@cluster1_avfs
avfs_clk_platform initialized
soctherm initialized
aotag initialized
powergate initialized
dvs initialized
pm initialized
pg_late initialized
strap initialized
tag initialized
emc initialized
clk_dt initialized
avfs_ccplex_platform initialized
tj_max: dt node not found
tj_init initialized
uphy_mrq_init: mrq handler registered
uphy_dt initialized
uphy initialized
safereg_init: period 80 ms
ec_late initialized
mrq initialized
��
  [0001.405] I> Welcome to Cboot
��WARNING: no registered clock for FMON_NAFLL_CLUSTER1 (id 281)
fmon_post initialized
��[0001.406] I> Cboot Version: t194-cfa2bc91
[0001.408] I> CPU-BL Params @ 0xf2820000
[0001.411] I>  0) Base:0x00000000 Size:0x00000000
[0001.416] I>  1) Base:0xf1100000 Size:0x00100000
[0001.420] I>  2) Base:0xf2000000 Size:0x00200000
��clk_set_parent failed for clk i2c2, parent pll_aon (-22)
clk_set_parent failed for clk i2c8, parent pll_aon (-22)
clk_dt_late initialized
machine_check initialized
pm_post initialized
dbells initialized
avfs_clk_platform_post initialized
dmce initialized
cvc initialized
ccplex_avfs_hw_init: nafll_cluster0: not monitored
ccplex_avfs_hw_init: nafll_cluster2: not monitored
ccplex_avfs_hw_init: nafll_cluster3: not monitored
avfs_clk_mach_post initialized
regulator_post initialized
rm initialized
sc7_diag initialized
thermal_test initialized
serial_late initialized
clk_post initialized
clk_dt_post initialized
mc_reg initialized
pg_post initialized
dyn_modules initialized
sku_debugfs initialized
speedo_debugfs initialized
adc_debugfs initialized
Failed to register PTO counter for id 281
Failed to register PTO counter for id 281
Failed to register PTO counter for id 281
Failed to register PTO counter for id 281
clk_debugfs initialized
emc_debugfs initialized
dvs_debugfs initialized
fmon_debugfs_init_one: no clock debugfs node to attach FMON_NAFLL_CLUSTER1
fmon_debugfs initialized
vmon_debugfs initialized
pg_debugfs initialized
profile_fs initialized
debugfs_cons initialized
mail_fs initialized
profile initialized
cvc_debugfs initialized
dmce_debugfs initialized
ec_debugfs initialized
rm_rail_debugfs_init: /rm/vdd_cpu: failed
rm_rail_debugfs_init: /rm/vdd_cpu: failed
rm_debugfs initialized
soctherm_debug initialized
gr_reader initialized
mods initialized
dt_fs initialized
debugfs_mrq initialized
debug_mrq initialized
debug_safereg initialized
initializing target
calling apps_init()
starting app shell
entering main console loop
] ��[0001.425] I>  3) Base:0xf1200000 Size:0x00200000
[0001.579] I>  4) Base:0xf1000000 Size:0x00100000
[0001.583] I>  5) Base:0xf0f00000 Size:0x00100000
[0001.588] I>  6) Base:0xf3800000 Size:0x00400000
[0001.592] I>  7) Base:0xf1c00000 Size:0x00400000
[0001.597] I>  8) Base:0xf0e00000 Size:0x00100000
[0001.601] I>  9) Base:0xf0d00000 Size:0x00100000
[0001.606] I> 10) Base:0xf3000000 Size:0x00800000
[0001.610] I> 11) Base:0x40000000 Size:0x00040000
[0001.615] I> 12) Base:0xf0c00000 Size:0x00100000
[0001.619] I> 13) Base:0x40046000 Size:0x00002000
[0001.623] I> 14) Base:0x40048000 Size:0x00002000
[0001.628] I> 15) Base:0xac000000 Size:0x00004000
[0001.632] I> 16) Base:0x4004a000 Size:0x00002000
[0001.637] I> 17) Base:0xf0b00000 Size:0x00100000
[0001.641] I> 18) Base:0x4004c000 Size:0x00002000
[0001.646] I> 19) Base:0xf2200000 Size:0x00600000
[0001.650] I> 20) Base:0x4004e000 Size:0x00002000
[0001.655] I> 21) Base:0xf0ad0000 Size:0x0000c000
[0001.659] I> 22) Base:0x00000000 Size:0x00000000
[0001.664] I> 23) Base:0xf0ae0000 Size:0x00020000
[0001.668] I> 24) Base:0xf6000000 Size:0x02000000
[0001.673] I> 25) Base:0x40050000 Size:0x00002000
[0001.677] I> 26) Base:0x40040000 Size:0x00006000
[0001.681] I> 27) Base:0xf1800000 Size:0x00400000
[0001.686] I> 28) Base:0xf4c00000 Size:0x01400000
[0001.690] I> 29) Base:0xf1400000 Size:0x00400000
[0001.695] I> 30) Base:0x00000000 Size:0x00000000
[0001.699] I> 31) Base:0x00000000 Size:0x00000000
[0001.704] I> 32) Base:0xf8000000 Size:0x08000000
[0001.708] I> 33) Base:0x00000000 Size:0x00000000
[0001.713] I> 34) Base:0xf3c00000 Size:0x01000000
[0001.717] I> 35) Base:0xab000000 Size:0x01000000
[0001.722] I> 36) Base:0xa0000000 Size:0x0b000000
[0001.726] I> 37) Base:0xf2800000 Size:0x00800000
[0001.731] I> 38) Base:0x80000000 Size:0x20000000
[0001.735] I> 39) Base:0xb0000000 Size:0x08000000
[0001.739] I> 40) Base:0x00000000 Size:0x00000000
[0001.744] I> 41) Base:0x00000000 Size:0x00000000
[0001.748] I> 42) Base:0x00000000 Size:0x00000000
[0001.753] I> 43) Base:0x00000000 Size:0x00000000
[0001.757] I> 44) Base:0x00000000 Size:0x00000000
[0001.762] I> 45) Base:0x00000000 Size:0x00000000
[0001.766] GIC-SPI Target CPU: 0
[0001.769] Interrupts Init done
[0001.772] calling constructors
[0001.775] initializing heap
[0001.778] I> Heap: [0xa0699b80 ... 0xab000000]
[0001.782] initializing threads
[0001.785] initializing timers
[0001.788] creating bootstrap completion thread
[0001.792] top of bootstrap2()
[0001.795] CPU: MIDR: 0x4E0F0040, MPIDR: 0x80000000
[0001.799] initializing platform
[0001.802] E> DEVICE_PROD: Invalid value data = 0, size = 0.
[0001.808] W> device prod register failed
[0001.812] I> Bl_dtb @0xaaf00000
[0001.817] W> "plugin-manager" doesn't exist, creating
[0001.819] W> "ids" doesn't exist, creating
[0001.824] W> "connection" doesn't exist, creating
[0001.828] W> "configs" doesn't exist, creating
[0001.839] E> failed to read label property for node 158736: 13
[0001.841] E> failed to read reg property for node 158800: 13
[0001.845] E> failed to read reg property for node 158852: 13
[0001.850] E> failed to read reg property for node 158936: 13
[0001.856] I> Find /i2c@3160000's alias i2c0
[0001.858] I> Reading eeprom i2c=0 address=0x50
[0001.888] I> Device at /i2c@3160000:0x50
[0001.888] I> Reading eeprom i2c=0 address=0x57
[0001.889] E> I2C: slave not found in slaves.
[0001.889] E> I2C: Could not write 0 bytes to slave: 0x00ae with repeat start true.
[0001.890] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.891] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xae at 0x00000000 via instance 0.
[0001.898] E> eeprom: Failed to read I2C slave device
[0001.903] I> Eeprom read failed 0x3526070d
[0001.908] I> Find /i2c@c240000's alias i2c1
[0001.911] I> Reading eeprom i2c=1 address=0x50
[0001.916] E> I2C: slave not found in slaves.
[0001.919] E> I2C: Could not write 0 bytes to slave: 0x00a0 with repeat start true.
[0001.927] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.933] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa0 at 0x00000000 via instance 1.
[0001.942] E> eeprom: Retry to read I2C slave device.
[0001.947] E> I2C: slave not found in slaves.
[0001.951] E> I2C: Could not write 0 bytes to slave: 0x00a0 with repeat start true.
[0001.959] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.964] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa0 at 0x00000000 via instance 1.
[0001.974] E> eeprom: Failed to read I2C slave device
[0001.979] I> Eeprom read failed 0x3526070d
[0001.983] I> create_pm_ids: id: 3668-0001-300-B, len: 15
[0001.988] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0001.999] I> Adding plugin-manager/ids/3668-0001-300=/i2c@3160000:module@0x50
[0002.007] W> "i2c@3160000" doesn't exist, creating
[0002.011] W> "module@0x50" doesn't exist, creating
[0002.016] I> Adding plugin-manager/cvm
[0002.019] W> "chip-id" doesn't exist, creating
[0002.023] I> Adding plugin-manager/chip-id/A02P
[0002.028] I> Plugin-manager override starting
[0002.032] I> node /plugin-manager/fragment-pcie-c5-rp matches
[0002.040] I> node /plugin-manager/fragement-tegra-wdt-en matches
[0002.045] I> node /plugin-manager/fragement-tegra-sdhci-sd-dis matches
[0002.053] I> Disable plugin-manager status in FDT
[0002.054] I> Plugin-manager override finished successfully
[0002.059] I> gpio framework initialized
[0002.064] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio' driver
[0002.071] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio-aon' driver
[0002.078] I> tegrabl_tca9539_init: i2c bus: 1, slave addr: 0x46
[0002.085] W> fetch_driver_phandle_from_dt: failed to get node with compatible ti,tca9539
[0002.093] W> fetch_driver_phandle_from_dt: failed to get node with compatible nxp,tca9539
[0002.100] W> tegrabl_tca9539_init: failed to fetch phandle from dt
[0002.106] I> tegrabl_tca9539_init: i2c bus: 1, slave addr: 0x44
[0002.113] W> fetch_driver_phandle_from_dt: failed to get node with compatible ti,tca9539
[0002.121] W> fetch_driver_phandle_from_dt: failed to get node with compatible nxp,tca9539
[0002.128] W> tegrabl_tca9539_init: failed to fetch phandle from dt
[0002.135] I> fixed regulator driver initialized
[0002.142] I> register 'maxim' power off handle
[0002.143] I> virtual i2c enabled
[0002.146] I> registered 'maxim,max20024' pmic
[0002.150] I> tegrabl_gpio_driver_register: register 'max20024-gpio' driver
[0002.156] I> Boot-device: QSPI
[0002.159] I> Boot_device: QSPI_FLASH instance: 0
[0002.164] I> QSPI source rate = 204000 Khz
[0002.168] I> Requested rate for QSPI clock = 34000 Khz
[0002.173] I> BPMP-set rate for QSPI clk = 34000 Khz
[0002.178] I> QSPI Flash Size = 32 MB
[0002.185] I> Qspi initialized successfully
[0002.186] I> qspi flash-0 params source = boot args
[0002.190] I> create_pm_ids: id: 3668-0001-300-B, len: 15
[0002.195] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0002.210] I> sdmmc-3 params source = safe params
[0002.212] I> Found 47 partitions in QSPI_FLASH (instance 0)
[0002.232] W> Cannot find any partition table for 00000003
[0002.244] I> Found 11 partitions in SDMMC_USER (instance 3)
[0002.252] I> regulator 'vdd-hdmi-5v0' already enabled
[0002.255] I> regulator 'vdd-hdmi-5v0' already enabled
[0002.256] E> tegrabl_display_init_regulator: hdmi cable is not connected
[0002.256] E> tegrabl_display_get_pdata, failed to parse dtb settings
[0002.260] E> invalid display type
[0002.261] E> cannot find any other nvdisp nodes
[0002.261] E> no valid display unit config found in dtb
[0002.263] W> display init failed
[0002.265] I> Load in CBoot Boot Options partition and parse it
[0002.271] E> Error -9 when finding node with path /boot-configuration
[0002.277] E> tegrabl_cbo_parse_info: "boot-configuration" not found in CBO file.
[0002.284] I> Using default boot order
[0002.288] I> boot-dev-order :-
[0002.290] I> 1.sd
[0002.292] I> 2.usb
[0002.294] I> 3.nvme
[0002.296] I> 4.emmc
[0002.298] I> 5.net
[0002.300] I> Hit any key to stop autoboot:	4	3	2	1
[0004.307] initializing target
[0004.307] calling apps_init()
[0004.308] starting app kernel_boot_app
[0004.318] I> found decompressor handler: lz4-legacy
[0004.319] I> decompressing BMP blob ...
[0004.330] I> Kernel type = Normal
[0004.331] I> ########## SD (0) boot ##########
[0004.331] W> Error: failed to get sd-card params
[0004.332] I> -0 params source = 
[0004.332] W> Failed to initialize device 6-0
[0004.332] E> SD boot failed, err: 252641293
[0004.333] I> ########## USB (0) boot ##########
[0004.338] W> No valid slot number is found in scratch register
[0004.338] W> Return default slot: _a
[0004.354] I> Validate XUSB-FW ...
[0004.354] I> T19x: Authenticate XUSB-FW (bin_type: 11), max size 0x30000
[0004.356] I> Encryption fuse is not ON
[0004.356] I> USB Firmware Version: 60.09 release
[0004.414] I> regulator of usb2-0 already enabled
[0004.418] I> regulator of usb2-1 already enabled
[0004.422] I> regulator of usb2-2 already enabled
[0005.488] I> USB 2.0 port 2 new high-speed USB device detected
[0005.490] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3a00, cmd_ring.dma = 0xa42d3ac0
[0005.591] I> Start to enumerate device
[0005.592] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3a00, cmd_ring.dma = 0xa42d3ac0
[0005.599] I> 
[0005.599] I> Enumerated USB Device 0bda:5411
[0005.599] I> 
[0005.600] I> Hub:
[0005.601] I> 	Downstream ports        : 4
[0005.601] I> 	Power switching         : Ganged
[0005.601] I> 	Compound device         : No
[0005.601] I> 	Over-current protection : Global
[0005.604] I> 	Hub avg delay           : 4 ns
[0005.608] I> Set interface = 1 (alt setting)
[0005.613] I> Enable hub ports
[0005.623] I> Device detected on port 4
[0006.638] I> Reset port 4
[0006.693] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3a00, cmd_ring.dma = 0xa42d3ac0
[0006.794] I> Start to enumerate device
[0006.795] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3a00, cmd_ring.dma = 0xa42d3ac0
[0006.800] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3b00, cmd_ring.dma = 0xa42d3ac0
[0006.802] I> 
[0006.802] I> Enumerated USB Device 090c:1000
[0006.802] I> 
[0006.925] I> Max LUN = 0
[0009.934] W> xusbh_wait_irq: Timed out! status = 0x00
[0010.060] E> Bad CSW SIG (0)!
[0010.161] E> USBMSD: Invalid value CSW SIG 53425355.
[0010.161] E> INQUIRY returned error 0x7c7c1102 ...
[0013.171] W> xusbh_wait_irq: Timed out! status = 0x00
[0013.171] E> Bad CSW SIG (0)!
[0013.272] E> USBMSD: Invalid value CSW SIG 0.
[0016.304] W> xusbh_wait_irq: Timed out! status = 0x00
[0016.304] E> Bad CSW SIG (0)!
[0016.405] E> USBMSD: Invalid value CSW SIG 0.
[0019.415] W> xusbh_wait_irq: Timed out! status = 0x00
[0019.415] E> Bad CSW SIG (0)!
[0019.516] E> USBMSD: Invalid value CSW SIG 0.
[0022.547] W> xusbh_wait_irq: Timed out! status = 0x00
[0022.548] E> Bad CSW SIG (0)!
[0022.649] E> USBMSD: Invalid value CSW SIG 0.
[0022.649] W> TEST UNIT READY returned error 0x7c7c1102, retries @ 5 ...
[0022.650] I> -0 params source = 
[0022.650] E> Blockdev read block: exit error = e0e0104
[0022.651] E> Blockdev read block: exit error = e0e0104
[0022.651] W> Cannot find any partition table for 00050000
[0022.653] I> Detect filesystem
[0022.656] E> Either offset or requested data size from that offset is beyond device size
[0022.664] E> Blockdev read: exit error = e0e0004
[0022.668] I> fs_detect:160: Failed to read superblock
[0022.673] I> Loading kernel-bootctrl from partition
[0022.678] I> Loading partition kernel-bootctrl at 0xa4360000 from device(0x1)
[0022.692] W> tegrabl_get_kernel_bootctrl: magic number(0x00000000) is invalid
[0022.693] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0022.699] E> USB boot failed, err: 724238349
[0022.703] I> ########## NVME (0) boot ##########
[0022.707] I> Initializing nvme device instance 0
[0022.712] I> Initializing nvme controller
[0022.716] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14180000
[0022.722] I> vpcie3v3-supply not found
[0022.726] I> vpcie12v-supply not found
[0022.729] W> Failed to get nvidia,plat-gpios
[0022.733] I> tegrabl_pcie_soc_preinit: (0):
[0022.738] I> Unpowergate
[0022.740] I> tegrabl_car_clk_disable(0) ...
[0022.744] I> tegrabl_car_rst_set(CORE, 0) ...
[0022.748] I> tegrabl_car_rst_set(APB, 0) ...
[0022.752] I> tegrabl_car_clk_enable(0) ...
[0022.756] I> tegrabl_car_rst_clear(APB, 0) ...
[0022.760] I> tegrabl_set_ctrl_state(0)
[0022.764] I> CLR PCIE_APB:6
[0022.767] I> tegrabl_pcie_soc_init: (0):
[0022.770] I> APPL initialization ...
[0022.774] I> poweron phys
[0022.776] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14180000
[0022.783] I> tegrabl_power_on_phy: controller 0 not available
[0022.788] E> Failed to power on phy on controller-0
[0022.793] W> Failed tegrabl_pcie_soc_init(), error=0x1
[0022.798] I> Failed to initialize SoC Host PCIe controller
[0022.803] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(0); error=0x1
[0022.810] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800601
[0022.816] W> Failed to open NVME-0, err = 80800601
[0022.821] W> Failed to initialize device 10-0
[0022.825] E> NVME (0) boot failed, err: 0x80800601
[0022.830] I> ########## NVME (1) boot ##########
[0022.834] I> Initializing nvme device instance 1
[0022.839] I> Initializing nvme controller
[0022.843] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14100000
[0022.849] I> vpcie3v3-supply not found
[0022.852] I> vpcie12v-supply not found
[0022.856] W> Failed to get nvidia,plat-gpios
[0022.860] I> tegrabl_pcie_soc_preinit: (1):
[0022.864] I> Unpowergate
[0022.867] I> tegrabl_car_clk_disable(1) ...
[0022.871] I> tegrabl_car_rst_set(CORE, 1) ...
[0022.875] I> tegrabl_car_rst_set(APB, 1) ...
[0022.879] I> tegrabl_car_clk_enable(1) ...
[0022.883] I> tegrabl_car_rst_clear(APB, 1) ...
[0022.887] I> tegrabl_set_ctrl_state(1)
[0022.891] I> CLR PCIE_APB:6
[0022.893] I> tegrabl_pcie_soc_init: (1):
[0022.897] I> APPL initialization ...
[0022.900] I> poweron phys
[0022.903] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14100000
[0022.909] I> tegrabl_power_on_phy: controller 1 not available
[0022.915] E> Failed to power on phy on controller-1
[0022.920] W> Failed tegrabl_pcie_soc_init(), error=0x1
[0022.925] I> Failed to initialize SoC Host PCIe controller
[0022.930] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(1); error=0x1
[0022.936] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800601
[0022.943] W> Failed to open NVME-1, err = 80800601
[0022.947] W> Failed to initialize device 10-1
[0022.952] E> NVME (1) boot failed, err: 0x80800601
[0022.956] I> ########## NVME (2) boot ##########
[0022.961] I> Initializing nvme device instance 2
[0022.965] I> Initializing nvme controller
[0022.969] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14120000
[0022.975] I> vpcie3v3-supply not found
[0022.979] I> vpcie12v-supply not found
[0022.983] W> Failed to get nvidia,plat-gpios
[0022.987] I> tegrabl_pcie_soc_preinit: (2):
[0022.991] I> Unpowergate
[0022.993] I> tegrabl_car_clk_disable(2) ...
[0022.997] I> tegrabl_car_rst_set(CORE, 2) ...
[0023.001] I> tegrabl_car_rst_set(APB, 2) ...
[0023.005] I> tegrabl_car_clk_enable(2) ...
[0023.009] I> tegrabl_car_rst_clear(APB, 2) ...
[0023.014] I> tegrabl_set_ctrl_state(2)
[0023.017] I> CLR PCIE_APB:6
[0023.020] I> tegrabl_pcie_soc_init: (2):
[0023.024] I> APPL initialization ...
[0023.027] I> poweron phys
[0023.030] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14120000
[0023.036] I> tegrabl_power_on_phy: controller 2 not available
[0023.041] E> Failed to power on phy on controller-2
[0023.047] W> Failed tegrabl_pcie_soc_init(), error=0x1
[0023.051] I> Failed to initialize SoC Host PCIe controller
[0023.057] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(2); error=0x1
[0023.063] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800601
[0023.069] W> Failed to open NVME-2, err = 80800601
[0023.074] W> Failed to initialize device 10-2
[0023.078] E> NVME (2) boot failed, err: 0x80800601
[0023.083] I> ########## NVME (3) boot ##########
[0023.087] I> Initializing nvme device instance 3
[0023.092] I> Initializing nvme controller
[0023.096] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14140000
[0023.102] I> vpcie3v3-supply not found
[0023.106] I> vpcie12v-supply not found
[0023.109] W> Failed to get nvidia,plat-gpios
[0023.113] I> tegrabl_pcie_soc_preinit: (3):
[0023.117] I> Unpowergate
[0023.120] I> tegrabl_car_clk_disable(3) ...
[0023.124] I> tegrabl_car_rst_set(CORE, 3) ...
[0023.128] I> tegrabl_car_rst_set(APB, 3) ...
[0023.132] I> tegrabl_car_clk_enable(3) ...
[0023.136] I> tegrabl_car_rst_clear(APB, 3) ...
[0023.140] I> tegrabl_set_ctrl_state(3)
[0023.144] I> CLR PCIE_APB:6
[0023.147] I> tegrabl_pcie_soc_init: (3):
[0023.150] I> APPL initialization ...
[0023.154] I> poweron phys
[0023.156] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14140000
[0023.162] I> tegrabl_power_on_phy: controller 3 not available
[0023.168] E> Failed to power on phy on controller-3
[0023.173] W> Failed tegrabl_pcie_soc_init(), error=0x1
[0023.178] I> Failed to initialize SoC Host PCIe controller
[0023.183] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(3); error=0x1
[0023.190] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800601
[0023.196] W> Failed to open NVME-3, err = 80800601
[0023.201] W> Failed to initialize device 10-3
[0023.205] E> NVME (3) boot failed, err: 0x80800601
[0023.209] I> ########## NVME (4) boot ##########
[0023.214] I> Initializing nvme device instance 4
[0023.218] I> Initializing nvme controller
[0023.223] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14160000
[0023.229] I> vpcie3v3-supply not found
[0023.232] I> vpcie12v-supply not found
[0023.236] W> Failed to get nvidia,plat-gpios
[0023.240] I> tegrabl_pcie_soc_preinit: (4):
[0023.244] I> Unpowergate
[0023.247] I> tegrabl_car_clk_disable(4) ...
[0023.250] I> tegrabl_car_rst_set(CORE, 4) ...
[0023.255] I> tegrabl_car_rst_set(APB, 4) ...
[0023.259] I> tegrabl_car_clk_enable(4) ...
[0023.263] I> tegrabl_car_rst_clear(APB, 4) ...
[0023.267] I> tegrabl_set_ctrl_state(4)
[0023.271] I> CLR PCIE_APB:6
[0023.273] I> tegrabl_pcie_soc_init: (4):
[0023.277] I> APPL initialization ...
[0023.280] I> poweron phys
[0023.283] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x14160000
[0023.289] I> tegrabl_power_on_phy: power on phy @0x3f40000
[0024.395] C> Failed to link up controller-4
[0024.395] W> Failed tegrabl_pcie_soc_init(), error=0x12
[0024.396] I> Failed to initialize SoC Host PCIe controller
[0024.396] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(4); error=0x12
[0024.397] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800612
[0024.397] W> Failed to open NVME-4, err = 80800612
[0024.402] W> Failed to initialize device 10-4
[0024.406] E> NVME (4) boot failed, err: 0x80800612
[0024.410] I> ########## NVME (5) boot ##########
[0024.415] I> Initializing nvme device instance 5
[0024.419] I> Initializing nvme controller
[0024.424] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x141a0000
[0024.430] I> vpcie3v3-supply not found
[0024.433] I> vpcie12v-supply not found
[0024.437] W> Failed to get nvidia,plat-gpios
[0024.441] I> tegrabl_pcie_soc_preinit: (5):
[0024.445] I> Unpowergate
[0024.447] I> tegrabl_car_clk_disable(5) ...
[0024.451] I> tegrabl_car_rst_set(CORE, 5) ...
[0024.456] I> tegrabl_car_rst_set(APB, 5) ...
[0024.460] I> tegrabl_car_clk_enable(5) ...
[0024.464] I> tegrabl_car_rst_clear(APB, 5) ...
[0024.468] I> tegrabl_set_ctrl_state(5)
[0024.471] I> CLR PCIE_APB:6
[0024.474] I> tegrabl_pcie_soc_init: (5):
[0024.478] I> APPL initialization ...
[0024.481] I> poweron phys
[0024.484] I> tegrabl_locate_pcie_ctrl_in_dt: found match at 0x141a0000
[0024.490] I> tegrabl_power_on_phy: power on phy @0x3eb0000
[0024.495] I> tegrabl_power_on_phy: power on phy @0x3ec0000
[0024.501] I> tegrabl_power_on_phy: power on phy @0x3ed0000
[0024.506] I> tegrabl_power_on_phy: power on phy @0x3ee0000
[0024.512] I> tegrabl_power_on_phy: power on phy @0x3ef0000
[0024.517] I> tegrabl_power_on_phy: power on phy @0x3f00000
[0024.522] I> tegrabl_power_on_phy: power on phy @0x3f10000
[0024.528] I> tegrabl_power_on_phy: power on phy @0x3f20000
[0025.633] C> Failed to link up controller-5
[0025.633] W> Failed tegrabl_pcie_soc_init(), error=0x12
[0025.634] I> Failed to initialize SoC Host PCIe controller
[0025.634] E> tegrabl_nvme_init: Failed tegrabl_pcie_init(5); error=0x12
[0025.635] W> tegrabl_nvme_bdev_open: Failed NVME INIT; error=0x80800612
[0025.635] W> Failed to open NVME-5, err = 80800612
[0025.640] W> Failed to initialize device 10-5
[0025.644] E> NVME (5) boot failed, err: 0x80800612
[0025.649] I> ########## Fixed storage boot ##########
[0025.654] I> Loading kernel-bootctrl from partition
[0025.658] I> Loading partition kernel-bootctrl at 0xa4360000 from device(0x1)
[0025.673] W> tegrabl_get_kernel_bootctrl: magic number(0x00000000) is invalid
[0025.673] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0025.679] I> Already published: 00010003
[0025.683] I> Look for boot partition
[0025.686] I> Fallback: assuming 0th partition is boot partition
[0025.692] I> Detect filesystem
[0025.719] I> Loading extlinux.conf ...
[0025.720] I> Loading extlinux.conf binary from rootfs ...
[0025.720] I> rootfs path: /sdmmc_user/boot/extlinux/extlinux.conf
[0025.769] I> Loading extlinux.conf sig file from rootfs ...
[0025.769] I> rootfs path: /sdmmc_user/boot/extlinux/extlinux.conf.sig
[0031.980] I> lookup_linear_dir:447: Invalid file block num
[0031.981] I> ext2_walk:142: 'extlinux.conf.sig' lookup failed
[0031.981] I> ext4_open_file:666: '/boot/extlinux/extlinux.conf.sig' lookup failed
[0031.982] E> file /sdmmc_user/boot/extlinux/extlinux.conf.sig open failed!!
[0031.982] W> Failed to load extlinux.conf sig file (err=202113041)
[0031.986] I> Validate extlinux.conf ...
[0031.990] I> T19x: Authenticate extlinux.conf (bin_type: 54), max size 0x2000
[0031.997] E> Stage2Signature validation failed with SHA2!!
[0032.002] C> OEM authentication of extlinux.conf header failed!
[0032.008] W> Failed to validate extlinux.conf binary from rootfs (err=1077936152, fail=1)
[0032.016] W> Security fuse not burned, ignore validation failure
[0032.022] I> restore load_size to 1177
[0032.026] I> L4T boot options
[0032.028] I> [1]: "primary kernel"
[0032.032] I> [2]: "Custom Header Config: <HDR40 HX711 IIO and SPI for Olyns>"
[0032.039] I> Enter choice: 
[0035.042] I> Continuing with default option: 2
[0035.042] I> Loading kernel ...
[0035.042] I> Loading kernel binary from rootfs ...
[0035.043] I> rootfs path: /sdmmc_user/boot/Image
[0035.449] I> Loading kernel sig file from rootfs ...
[0035.449] I> rootfs path: /sdmmc_user/boot/Image.sig
[0035.480] I> overload load_size to 34484232 (from 34484240)
[0035.480] I> Validate kernel ...
[0035.481] I> T19x: Authenticate kernel (bin_type: 37), max size 0x5000000
[0035.795] I> Encryption fuse is not ON
[0035.811] I> Loading kernel-dtb ...
[0035.811] I> Loading kernel-dtb binary from rootfs ...
[0035.812] I> rootfs path: /sdmmc_user/boot/kernel_tegra194-p3668-all-p3509-0000-user-custom.dtb
[0035.850] I> Loading kernel-dtb sig file from rootfs ...
[0035.851] I> rootfs path: /sdmmc_user/boot/kernel_tegra194-p3668-all-p3509-0000-user-custom.dtb.sig
[0035.875] I> ext2_walk:142: 'kernel_tegra194-p3668-all-p3509-0000-user-custom.dtb.sig' lookup failed
[0035.876] I> ext4_open_file:666: '/boot/kernel_tegra194-p3668-all-p3509-0000-user-custom.dtb.sig' lookup failed
[0035.877] E> file /sdmmc_user/boot/kernel_tegra194-p3668-all-p3509-0000-user-custom.dtb.sig open failed!!
[0035.879] W> Failed to load kernel-dtb sig file (err=202113041)
[0035.885] I> Validate kernel-dtb ...
[0035.888] I> T19x: Authenticate kernel-dtb (bin_type: 38), max size 0x400000
[0035.895] E> Stage2Signature validation failed with SHA2!!
[0035.900] C> OEM authentication of kernel-dtb header failed!
[0035.906] W> Failed to validate kernel-dtb binary from rootfs (err=1077936152, fail=1)
[0035.914] W> Security fuse not burned, ignore validation failure
[0035.919] I> restore load_size to 205319
[0035.923] I> Loading ramdisk from rootfs ...
[0035.927] I> Loading ramdisk binary from rootfs ...
[0035.932] I> rootfs path: /sdmmc_user/boot/initrd
[0036.045] I> Loading ramdisk sig file from rootfs ...
[0036.046] I> rootfs path: /sdmmc_user/boot/initrd.sig
[0036.076] I> overload load_size to 7238344 (from 7238352)
[0036.077] I> Validate ramdisk ...
[0036.077] I> T19x: Authenticate ramdisk (bin_type: 54), max size 0x4000000
[0036.143] I> Encryption fuse is not ON
[0036.157] I> Kernel hdr @0xa4360000
[0036.157] I> Kernel dtb @0x90000000
[0036.157] I> decompressor handler not found
[0036.158] I> Copying kernel image (34484232 bytes) from 0xa4360000 to 0x80080000 ... [0036.168] I> Done
[0036.169] I> Updated bpmp info to DTB
[0036.170] I> Ramdisk: Base: 0x91000000; Size: 0x6e72c8
[0036.170] I> Updated initrd info to DTB
[0036.171] W> WARN: Fail to override "console=none" in commandline
[0036.171] I> Active rootfs suffix: 
[0036.173] E> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0036.181] E> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0036.188] E> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0036.195] W> No valid slot number is found in scratch register
[0036.201] W> Return default slot: _a
[0036.204] I> Active slot suffix: 
[0036.207] I> add_boot_slot_suffix: slot_suffix = 
[0036.212] I> Linux Cmdline: console=ttyTCU0,115200 video=tegrafb earlycon=tegra_comb_uart,mmio32,0x0c168000 gpt rootfs.slot_suffix= usbcore.old_scheme_first=1 tegraid=19.1.2.0.0 maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.4.2 vpr_resize sdhci_tegra.en_boot_part_access=1 
[0036.236] I> Updated bootarg info to DTB
[0036.240] W> MAC addr invalid!
[0036.243] E> Failed to get WIFI MAC address
[0036.247] W> MAC addr invalid!
[0036.250] E> Failed to get Bluetooth MAC address
[0036.254] I> eeprom_get_mac_addr: MAC (type: 2): 48:b0:2d:51:0b:07
[0036.261] W> "plugin-manager" doesn't exist, creating
[0036.265] I> Adding /chosen/plugin-manager/cvm
[0036.270] W> "chip-id" doesn't exist, creating
[0036.274] I> Adding /chosen/plugin-manager/chip-id
[0036.279] W> "configs" doesn't exist, creating
[0036.283] I> Adding /chosen/plugin-manager/configs
[0036.287] W> "ids" doesn't exist, creating
[0036.291] I> Adding /chosen/plugin-manager/ids
[0036.296] W> "odm-data" doesn't exist, creating
[0036.300] I> Adding /chosen/plugin-manager/odm-data
[0036.307] W> "memory" doesn't exist, creating
[0036.309] I> [0] START: 0x80000000, END: 0xac000000
[0036.314] I> [1] START: 0xac004000, END: 0xf0ad0000
[0036.318] I> [2] START: 0xf0adc000, END: 0xf0ae0000
[0036.323] I> dram_block larger than 80000000
[0036.327] I> [3] START: 0x100000000, END: 0x280000000
[0036.332] I> added [base:0x80000000, size:0x2c000000] to /memory
[0036.338] I> added [base:0xac200000, size:0x44800000] to /memory
[0036.344] I> added [base:0x100000000, size:0x180000000] to /memory
[0036.350] I> Updated memory info to DTB
[0036.354] E> add_disp_param: failed to get display params for du=0
[0036.360] W> "reset" doesn't exist, creating
[0036.364] I> Adding ecid(00000001640560822000000003fd0140) to DT
[0036.370] I> NVG: Logical CPU: 0; MPIDR: 0x80000000
[0036.374] I> NVG: Logical CPU: 1; MPIDR: 0x80000001
[0036.379] I> NVG: Logical CPU: 2; MPIDR: 0x80000100
[0036.384] I> NVG: Logical CPU: 3; MPIDR: 0x80000101
[0036.388] I> NVG: Logical CPU: 4; MPIDR: 0x80000200
[0036.393] I> NVG: Logical CPU: 5; MPIDR: 0x80000201
[0036.398] I> Deleted cpu@6 node in DT
[0036.401] I> Deleted cpu@7 node in DT
[0036.405] I> Deleted cluster3 node in FDT
[0036.409] I> Update arm-pmu in FDT
[0036.412] I> - update property: interrupts
[0036.416] I> - update property: interrupt-affinity
[0036.421] W> "misc-data" doesn't exist, creating
[0036.425] I> Boot-device: QSPI
[0036.428] W> Unknown boot device
[0036.431] I> Add storage-qspi to plugin-manager/misc-data
[0036.436] W> Unknown storage device
[0036.440] I> Add serial number:1421821016474 as DT property
[0036.446] I> Plugin-manager override starting
[0036.450] I> node /plugin-manager/fragment-pcie-c5-rp matches
[0036.456] I> node /plugin-manager/fragement-tegra-wdt-en matches
[0036.462] I> node /plugin-manager/fragement-tegra-sdhci-sd-dis matches
[0036.470] I> Disable plugin-manager status in FDT
[0036.472] I> Plugin-manager override finished successfully
[0036.477] I> tegrabl_load_kernel_and_dtb: Done
[0036.481] E> tegrabl_display_clear: display is not initialized
[0036.487] W> Boot logo display failed...
[0036.491] I> Kernel EP: 0x80080000, DTB: 0x90000000
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.253-tegra (buildbrain@mobile-u64-5497-d3000) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #1 SMP PREEMPT Mon Jul 26 12:19:28 PDT 2021
[    0.000000] Boot CPU: AArch64 Processor [4e0f0040]
[    0.000000] OF: fdt:memory scan node memory, reg size 48,
[    0.000000] OF: fdt: - 80000000 ,  2c000000
[    0.000000] OF: fdt: - ac200000 ,  44800000
[    0.000000] OF: fdt: - 100000000 ,  180000000
[    0.000000] earlycon: tegra_comb_uart0 at MMIO32 0x000000000c168000 (options '')
[    0.000000] bootconsole [tegra_comb_uart0] enabled
[    1.934679] imx219 9-0010: imx219_board_setup: error during i2c read probe (-121)
[    1.934973] imx219 9-0010: board setup failed
[    1.958832] imx219 10-0010: imx219_board_setup: error during i2c read probe (-121)
[    1.959070] imx219 10-0010: board setup failed
��WARNING: at platform/drivers/pg/pg-gpu-t194.c:185
WARNING: at platform/drivers/pg/pg-gpu-t194.c:185
��[    3.774458] cgroup: cgroup2: unknown option "nsdelegate"
��WARNING: at platform/drivers/pg/pg-gpu-t194.c:185
WARNING: at platform/drivers/pg/pg-gpu-t194.c:185
��[    5.017341] scsi 0:0:0:0: Direct-Access     Samsung  Flash Drive FIT  1100 PQ: 0 ANSI: 6
[    5.018662] sd 0:0:0:0: [sda] 250626566 512-byte logical blocks: (128 GB/120 GiB)
[    5.020406] sd 0:0:0:0: [sda] Write Protect is off
[    5.020807] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    5.047456] sd 0:0:0:0: [sda] Attached SCSI removable disk
[    5.283400] random: crng init done
[    5.283517] random: 7 urandom warning(s) missed due to ratelimiting
[    5.428774] using random self ethernet address
[    5.428906] using random host ethernet address
[    6.158295] using random self ethernet address
[    6.158425] using random host ethernet address
[    6.362420] CPU4: shutdown
[    6.407692] CPU5: shutdown
[    7.180298] Bridge firewalling registered

Ubuntu 18.04.5 LTS jetson-xavier ttyTCU0

jetson-xavier login: 

Looks like the usb boot failed. Did you remember what you’ve done before this drive fails to boot?

Are you sure your cboot was booting from usb drive before? I just feel you are not using cboot log to tell whether the device is booting from usb.

[0005.599] I> Enumerated USB Device 0bda:5411
[0005.599] I>
[0005.600] I> Hub:
[0005.601] I> Downstream ports : 4
[0005.601] I> Power switching : Ganged
[0005.601] I> Compound device : No
[0005.601] I> Over-current protection : Global
[0005.604] I> Hub avg delay : 4 ns
[0005.608] I> Set interface = 1 (alt setting)
[0005.613] I> Enable hub ports
[0005.623] I> Device detected on port 4
[0006.638] I> Reset port 4
[0006.693] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3a00, cmd_ring.dma = 0xa42d3ac0
[0006.794] I> Start to enumerate device
[0006.795] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3a00, cmd_ring.dma = 0xa42d3ac0
[0006.800] W> WARNING: event and command not matching, cmd_trb_ptr = 0xa42d3b00, cmd_ring.dma = 0xa42d3ac0
[0006.802] I>
[0006.802] I> Enumerated USB Device 090c:1000
[0006.802] I>
[0006.925] I> Max LUN = 0
[0009.934] W> xusbh_wait_irq: Timed out! status = 0x00
[0010.060] E> Bad CSW SIG (0)!
[0010.161] E> USBMSD: Invalid value CSW SIG 53425355.
[0010.161] E> INQUIRY returned error 0x7c7c1102 …
[0013.171] W> xusbh_wait_irq: Timed out! status = 0x00
[0013.171] E> Bad CSW SIG (0)!
[0013.272] E> USBMSD: Invalid value CSW SIG 0.
[0016.304] W> xusbh_wait_irq: Timed out! status = 0x00
[0016.304] E> Bad CSW SIG (0)!
[0016.405] E> USBMSD: Invalid value CSW SIG 0.
[0019.415] W> xusbh_wait_irq: Timed out! status = 0x00
[0019.415] E> Bad CSW SIG (0)!
[0019.516] E> USBMSD: Invalid value CSW SIG 0.
[0022.547] W> xusbh_wait_irq: Timed out! status = 0x00
[0022.548] E> Bad CSW SIG (0)!
[0022.649] E> USBMSD: Invalid value CSW SIG 0.
[0022.649] W> TEST UNIT READY returned error 0x7c7c1102, retries @ 5 …

Yes, the USB boot failed. The issue here is that we have many identical USB flash drives installed and booting in systems. After some period of time, they stop being able to boot (see the log) even though the partitions are OK and fully readable from the Xavier NX or any other Linux system.

Out of about 10-15 systems, we have had at least three fail in this way.

Answering your questions:

  • before the drive began failing to boot. it was in normal operation for several months. After a scheduled reboot, it stopped working.

  • we are 100% confident the device was booting from USB.

What we’re hoping to get is a specific indicator of WHAT is failing, so that we can prevent it in the future. If we’re wearing out a particular sector of the drive, we want to identify it so that we don’t have units mysteriously failing in the field as they’re doing now.

What can you tell us about this? What (specifically) should we be looking at?

I don’t see any issue reported regarding this recently.

Is it possible to

  1. Try to use name-brand devices (Sandisk, Samsung, Transend, HP, etc.). USB drive.

  2. Move to new jetpack release (32.7.1) and devkit carrier board to try.

  3. Follow the steps from developer guide to boot from usb.