Jetson TX2NX fails to boot over time (Hard lockup on CPU)

Hello! I have a weird issue with a couple of Jetson TX2NX’s. I have flashed about 10 Jetson TX2NX’s on my custom board and they are working flawlessly. Over a couple of weeks, 2 of them have simply locked up and stopped working ever since. They keep bootlooping with this boot message:

[0000.226] I> Welcome to MB2(TBoot-BPMP)(version: 01.00.160913-t186-M-00.00-mobile-4f229333)
[0000.235] I> Boot-device: eMMC
[0000.242] I> sdmmc bdev is already initialized
[0000.247] I> pmic: reset reason (nverc)        : 0x40
[0000.280] I> Found 19 partitions in SDMMC_BOOT (instance 3)
[0000.300] I> Found 34 partitions in SDMMC_USER (instance 3)
[0000.305] W> No valid slot number is found in scratch register
[0000.311] W> Return default slot: _a
[0000.315] I> A/B: bin_type (16) slot 0
[0000.318] I> Loading partition bpmp-fw at 0xd7800000
[0000.323] I> Reading two headers - addr:0xd7800000 blocks:1
[0000.329] I> Addr: 0xd7800000, start-block: 29417480, num_blocks: 1
[0000.355] I> Binary(16) of size 534416 is loaded @ 0xd7800000
[0000.361] W> No valid slot number is found in scratch register
[0000.367] W> Return default slot: _a
[0000.370] I> A/B: bin_type (17) slot 0
[0000.374] I> Loading partition bpmp-fw-dtb at 0xd79f0000
[0000.379] I> Reading two headers - addr:0xd79f0000 blocks:1
[0000.384] I> Addr: 0xd79f0000, start-block: 29419896, num_blocks: 1
[0000.405] I> Binary(17) of size 113248 is loaded @ 0xd79e4400
[0000.433] I> Loading SCE-FW ...
[0000.436] W> No valid slot number is found in scratch register
[0000.442] W> Return default slot: _a
[0000.445] I> A/B: bin_type (12) slot 0
[0000.449] I> Loading partition sce-fw at 0xd7300000
[0000.453] I> Reading two headers - addr:0xd7300000 blocks:1
[0000.459] I> Addr: 0xd7300000, start-block: 29423992, num_blocks: 1
[0000.479] I> Binary(12) of size 125632 is loaded @ 0xd7300000
[0000.485] I> Init SCE
[0000.487] I> Loading APE-FW ...
[0000.490] W> No valid slot number is found in scratch register
[0000.496] W> Return default slot: _a
[0000.499] I> A/B: bin_type (11) slot 0
[0000.503] I> Loading partition adsp-fw at 0xd7400000
[0000.508] I> Reading two headers - addr:0xd7400000 blocks:1
[0000.513] I> Addr: 0xd7400000, start-block: 29401096, num_blocks: 1
[0000.533] I> Binary(11) of size 77216 is loaded @ 0xd7400000
[0000.539] I> Copy BTCM section
[0000.542] W> No valid slot number is found in scratch register
[0000.547] W> Return default slot: _a
[0000.551] I> A/B: bin_type (13) slot 0
[0000.554] I> Loading partition cpu-bootloader at 0x96000000
[0000.560] I> Reading two headers - addr:0x96000000 blocks:1
[0000.565] I> Addr: 0x96000000, start-block: 29380616, num_blocks: 1
[0000.588] I> Binary(13) of size 309648 is loaded @ 0x96000000
[0000.594] W> No valid slot number is found in scratch register
[0000.599] W> Return default slot: _a
[0000.603] I> A/B: bin_type (20) slot 0
[0000.606] I> Loading partition bootloader-dtb at 0x85205400
[0000.612] I> Reading two headers - addr:0x85205400 blocks:1
[0000.617] I> Addr: 0x85205400, start-block: 29382664, num_blocks: 1
[0000.638] I> Binary(20) of size 193792 is loaded @ 0x85205400
[0000.644] W> No valid slot number is found in scratch register
[0000.649] W> Return default slot: _a
[0000.653] I> A/B: bin_type (14) slot 0
[0000.656] I> Loading partition secure-os at 0x85305600
[0000.661] I> Reading two headers - addr:0x85305600 blocks:1
[0000.667] I> Addr: 0x85305600, start-block: 29384712, num_blocks: 1
[0000.691] I> Binary(14) of size 406960 is loaded @ 0x85305600
[0000.698] I> TOS boot-params @ 0x85000000
[0000.702] I> TOS params prepared
[0000.705] I> Loading EKS ...
[0000.708] I> A/B: bin_type (15) slot 0
[0000.712] I> Loading partition eks at 0x85905800
[0000.716] I> Reading two headers - addr:0x85905800 blocks:1
[0000.722] I> Addr: 0x85905800, start-block: 29397000, num_blocks: 1
[0000.741] I> Binary(15) of size 1040 is loaded @ 0x85905800
[0000.747] I> EKB detected (length: 0x400) @ 0x85905800
[0000.752] I> Copied encrypted keys
[0000.756] I> boot profiler @ 0x175844000
[0000.759] I> boot profiler for TOS @ 0x175844000
[0000.764] I> Unhalting SCE
[0000.767] I> Primary Memory Start:80000000 Size:70000000
[0000.772] I> Extended Memory Start:f0110000 Size:856f0000
[0000.779] I> MB2(TBoot-BPMP) done

NOTICE:  BL31: v1.3(release):c67386022
NOTICE:  BL31: Built : 16:06:30, Jun 11 2024
ipc-unittest-main: 1519: Welcome to IPC unittest!!!
ipc-unittest-main: 1531: waiting forever
ipc-unittest-srv: 329: Init unittest services!!!
hwkey-agent: 41: hwkey-agent is running!!
hwkey-agent: 347: key_mgnt_processing .......
hwkey-agent: 255: Setting EKB key 0 to slot 14
hwkey-agent: 178: Init hweky-agent services!!
luks-srv: 40: luks-srv is running!!
luks-srv: 157: Init luks-srv IPC services!!
platform_bootstrap_epilog: trusty bootstrap complete
[0001.001] I> Welcome to Cboot
[0001.004] I> Cboot Version: t186-8828c893
[0001.008] I> CPU-BL Params @ 0x175800000
[0001.011] I>  0) Base:0x00000000 Size:0x00000000
[0001.016] I>  1) Base:0x177f00000 Size:0x00100000
[0001.020] I>  2) Base:0x177e00000 Size:0x00100000
[0001.025] I>  3) Base:0x177d00000 Size:0x00100000
[0001.029] I>  4) Base:0x177c00000 Size:0x00100000
[0001.034] I>  5) Base:0x177b00000 Size:0x00100000
[0001.038] I>  6) Base:0x177800000 Size:0x00200000
[0001.043] I>  7) Base:0x177400000 Size:0x00400000
[0001.048] I>  8) Base:0x177a00000 Size:0x00100000
[0001.052] I>  9) Base:0x177300000 Size:0x00100000
[0001.057] I> 10) Base:0x176800000 Size:0x00800000
[0001.061] I> 11) Base:0x30000000 Size:0x00040000
[0001.066] I> 12) Base:0xf0000000 Size:0x00100000
[0001.070] I> 13) Base:0x30040000 Size:0x00001000
[0001.074] I> 14) Base:0x30048000 Size:0x00001000
[0001.079] I> 15) Base:0x30049000 Size:0x00001000
[0001.083] I> 16) Base:0x3004a000 Size:0x00001000
[0001.088] I> 17) Base:0x3004b000 Size:0x00001000
[0001.092] I> 18) Base:0x3004c000 Size:0x00001000
[0001.097] I> 19) Base:0x3004d000 Size:0x00001000
[0001.101] I> 20) Base:0x3004e000 Size:0x00001000
[0001.105] I> 21) Base:0x3004f000 Size:0x00001000
[0001.110] I> 22) Base:0x00000000 Size:0x00000000
[0001.114] I> 23) Base:0xf0100000 Size:0x00010000
[0001.119] I> 24) Base:0x00000000 Size:0x00000000
[0001.123] I> 25) Base:0x00000000 Size:0x00000000
[0001.128] I> 26) Base:0x00000000 Size:0x00000000
[0001.132] I> 27) Base:0x00000000 Size:0x00000000
[0001.136] I> 28) Base:0x84400000 Size:0x00400000
[0001.141] I> 29) Base:0x30000000 Size:0x00010000
[0001.145] I> 30) Base:0x178000000 Size:0x08000000
[0001.150] I> 31) Base:0x00000000 Size:0x00000000
[0001.154] I> 32) Base:0x176000000 Size:0x00600000
[0001.159] I> 33) Base:0x80000000 Size:0x70000000
[0001.163] I> 34) Base:0xf0110000 Size:0x856f0000
[0001.168] I> 35) Base:0x00000000 Size:0x00000000
[0001.172] I> 36) Base:0x00000000 Size:0x00000000
[0001.176] I> 37) Base:0x1772e0000 Size:0x00020000
[0001.181] I> 38) Base:0x84000000 Size:0x00400000
[0001.185] I> 39) Base:0x96000000 Size:0x02000000
[0001.190] I> 40) Base:0x85000000 Size:0x01200000
[0001.194] I> 41) Base:0x175800000 Size:0x00500000
[0001.199] I> 42) Base:0x00000000 Size:0x00000000
[0001.203] I> 43) Base:0x00000000 Size:0x00000000
[0001.208] GIC-SPI Target CPU: 4
[0001.211] Interrupts Init done
[0001.214] calling constructors
[0001.217] initializing heap
[0001.220] initializing threads
[0001.223] initializing timers
[0001.226] creating bootstrap completion thread
[0001.231] top of bootstrap2()
[0001.234] CPU: ARM Cortex A57
[0001.238] CPU: MIDR: 0x411FD073, MPIDR: 0x80000100
[0001.243] initializing platform
[0001.246] I> Bl_dtb @0x85205400
[0001.249] I> gpio framework initialized
[0001.254] I> tegrabl_gpio_driver_register: register 'nvidia,tegra186-gpio' driver
[0001.263] I> tegrabl_gpio_driver_register: register 'nvidia,tegra186-gpio-aon' driver
[0001.271] I> GPIO framework and drivers are initialized.
[0001.276] I> Boot-device: eMMC
[0001.283] I> sdmmc bdev is already initialized
[0001.314] I> Found 19 partitions in SDMMC_BOOT (instance 3)
[0001.332] I> Found 34 partitions in SDMMC_USER (instance 3)
[0001.337] W> opt-in fuse is not set, skip fuse_burning
[0001.342] I> Reserved memory at 0xfbe00000 for U-Boot relocation
[0001.348] W> No valid slot number is found in scratch register
[0001.353] W> Return default slot: _a
[0001.363] I> A/B: bin_type (21) slot 0
[0001.366] I> Loading kernel-dtb from partition
[0001.371] I> Loading partition kernel-dtb at 0x80000000 from device(0x1)
[0001.391] I> Kernel_dtb @0x80000000
[0001.394] I> tegrabl_tca9539_init: i2c bus: 0, slave addr: 0xee
[0001.402] W> fetch_driver_phandle_from_dt: failed to get node with compatible ti,tca9539
[0001.412] W> fetch_driver_phandle_from_dt: failed to get node with compatible nxp,tca9539
[0001.420] W> tegrabl_tca9539_init: failed to fetch phandle from dt
[0001.426] I> tegrabl_tca9539_init: i2c bus: 0, slave addr: 0xe8
[0001.434] W> fetch_driver_phandle_from_dt: failed to get node with compatible ti,tca9539
[0001.444] W> fetch_driver_phandle_from_dt: failed to get node with compatible nxp,tca9539
[0001.452] W> tegrabl_tca9539_init: failed to fetch phandle from dt
[0001.460] I> fixed regulator driver initialized
[0001.482] I> register 'maxim' power off handle
[0001.487] I> virtual i2c enabled
[0001.490] I> registered 'maxim,max77620' pmic
[0001.495] I> tegrabl_gpio_driver_register: register 'max77620-gpio' driver
[0001.505] E> failed to read label property for node 161356: 13
[0001.512] E> failed to read reg property for node 161452: 13
[0001.518] E> failed to read reg property for node 161504: 13
[0001.525] E> failed to read label property for node 161588: 13
[0001.532] E> failed to read reg property for node 161656: 13
[0001.539] E> failed to read reg property for node 161728: 13
[0001.547] I> Find /i2c@c250000's alias i2c7
[0001.551] I> Reading eeprom i2c=7 address=0x50
[0001.580] I> Device at /i2c@c250000:0x50
[0001.584] I> Reading eeprom i2c=7 address=0x57
[0001.612] I> Device at /i2c@c250000:0x57
[0001.617] I> Find /i2c@3160000's alias i2c0
[0001.621] I> Reading eeprom i2c=0 address=0x50
[0001.626] E> I2C: slave not found in slaves.
[0001.630] E> I2C: Could not write 0 bytes to slave: 0x00a0 with repeat start true.
[0001.637] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.643] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa0 at 0x00000000 via instance 0.
[0001.653] E> eeprom: Failed to read I2C slave device
[0001.657] I> Eeprom read failed 0x3526070d
[0001.662] I> Find /i2c@3180000's alias i2c2
[0001.666] I> Reading eeprom i2c=2 address=0x54
[0001.671] I> Enabling gpio chip_id = 2, gpio pin = 9
[0001.675] C> GPIO driver for chip_id 0x2 could not be found
[0001.681] E> cam_eeprom_read: Can't get gpio driver
[0001.685] I> Eeprom read failed 0x4d4d000d
[0001.689] I> Reading eeprom i2c=2 address=0x57
[0001.694] I> Enabling gpio chip_id = 2, gpio pin = 9
[0001.698] C> GPIO driver for chip_id 0x2 could not be found
[0001.704] E> cam_eeprom_read: Can't get gpio driver
[0001.708] I> Eeprom read failed 0x4d4d000d
[0001.712] I> create_pm_ids: id: 3636-0001-301-D, len: 15
[0001.717] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0001.728] I> create_pm_ids: id: 3449-0000-400-D, len: 15
[0001.734] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0001.752] E> dsi not supported yet
[0001.755] E> tegrabl_display_get_pdata, failed to parse dtb settings
[0001.794] I> regulator 'vdd-hdmi-5v0' already enabled
[0001.806] I> regulator 'vdd-hdmi-5v0' already enabled
[0001.814] I> DP is not connected
[0001.817] E> tegrabl_display_get_pdata, failed to parse dtb settings
[0001.824] E> cannot find any other nvdisp nodes
[0001.828] E> no valid display unit config found in dtb
[0001.834] W> display init failed
[0001.837] initializing target
[0001.840] calling apps_init()
[0001.843] starting app kernel_boot_app
[0001.865] I> found decompressor handler: lz4-legacy
[0001.870] I> decompressing BMP blob ...
[0001.942] I> Kernel type = Normal
[0001.945] I> ########## Fixed storage boot ##########
[0001.950] I> Loading kernel-bootctrl from partition
[0001.955] I> Loading partition kernel-bootctrl at 0xa8000000 from device(0x1)
[0001.972] W> tegrabl_get_kernel_bootctrl: magic number(0x00000000) is invalid
[0001.979] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0001.986] W> No valid slot number is found in scratch register
[0001.992] W> Return default slot: _a
[0001.995] I> A/B: bin_type (24) slot 0
[0002.011] I> Boot image load address: 0x80400000
[0002.015] I> Loading kernel from partition
[0002.019] I> Loading partition kernel at 0x80400000 from device(0x1)
[0003.108] I> Validate kernel ...
[0003.111] I> T18x: Authenticate kernel (bin_type 24), max size 0x4000000
[0003.117] I> cboot: Info: Handle RSA_SBK as RSA.[0003.123] I> Decrypt the buffer ... [0003.128] I> done
[0003.129] I> Checking boot.img header magic ... [0003.134] I> [OK]
[0003.135] I> kernel-dtb is already loaded
[0003.139] I> Validate kernel-dtb ...
[0003.143] I> T18x: Authenticate kernel-dtb (bin_type 21), max size 0x100000
[0003.150] I> Decrypt the buffer ... [0003.154] I> done
[0003.156] I> Kernel hdr @0x80400000
[0003.159] I> Kernel dtb @0x80000000
[0003.162] I> decompressor handler not found
[0003.166] I> Copying kernel image (626733 bytes) from 0x80400800 to 0x85600000 ... [0003.174] I> Done
[0003.175] I> Move ramdisk (len: 0) from 0x8049a000 to 0x99730000
[0003.182] I> Updated bpmp info to DTB
[0003.187] I> Ramdisk: Base: 0x99730000; Size: 0x0
[0003.192] I> Updated initrd info to DTB
[0003.195] W> WARN: Fail to override "console=none" in commandline
[0003.201] I> Active rootfs suffix:
[0003.205] W> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0003.212] W> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0003.219] W> tegrabl_linuxboot_add_disp_param, du 0 failed to get display params
[0003.226] I> disabled_core_mask: 0xffffff0c
[0003.230] W> No valid slot number is found in scratch register
[0003.236] W> Return default slot: _a
[0003.239] I> Active slot suffix:
[0003.242] I> add_boot_slot_suffix: slot_suffix =
[0003.247] I> Linux Cmdline: console=ttyS0,115200 root=UUID=c96c0f1b-6a96-453a-831f-b3ac7a1a0d3d rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 isolcpus=1-2  video=tegrafb earlycon=uart8250,mmio32,0x3100000 nvdumper_reserved=0x1772e0000 gpt rootfs.slot_suffix= usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 no_console_suspend boot.slot_suffix= boot.ratchetvalues=0.2031647.1 vpr_resize bl_prof_dataptr=0x10000@0x175840000 sdhci_tegra.en_boot_part_access=1
[0003.291] I> Updated bootarg info to DTB
[0003.295] W> MAC addr invalid!
[0003.298] E> Failed to get WIFI MAC address
[0003.302] W> MAC addr invalid!
[0003.305] E> Failed to get Bluetooth MAC address
[0003.309] I> eeprom_get_mac_addr: MAC (type: 2): 48:b0:2d:68:2e:28
[0003.315] E> Found no plugin manager ids in source DT
[0003.320] W> Add plugin manager ids from board info
[0003.325] W> "plugin-manager" doesn't exist, creating
[0003.330] W> "ids" doesn't exist, creating
[0003.334] W> "connection" doesn't exist, creating
[0003.338] W> "configs" doesn't exist, creating
[0003.343] I> create_pm_ids: id: 3636-0001-301-D, len: 15
[0003.348] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0003.359] I> create_pm_ids: id: 3449-0000-400-D, len: 15
[0003.364] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0003.375] I> Adding plugin-manager/ids/3636-0001-301=/i2c@c250000:module@0x50
[0003.382] W> "i2c@c250000" doesn't exist, creating
[0003.387] W> "module@0x50" doesn't exist, creating
[0003.391] I> Adding plugin-manager/ids/3449-0000-400=/i2c@c250000:module@0x57
[0003.399] W> "module@0x57" doesn't exist, creating
[0003.404] I> Adding plugin-manager/ids/3636-0001-301-D
[0003.410] I> Adding plugin-manager/configs/3636-mem-type 00
[0003.416] I> Adding plugin-manager/configs/3636-power-config 00
[0003.422] I> Adding plugin-manager/configs/3636-misc-config 00
[0003.428] I> Adding plugin-manager/configs/3636-modem-config 00
[0003.433] I> Adding plugin-manager/configs/3636-touch-config 00
[0003.439] I> Adding plugin-manager/configs/3636-display-config 00
[0003.445] I> Adding plugin-manager/cvm
[0003.449] W> "chip-id" doesn't exist, creating
[0003.453] I> Adding plugin-manager/chip-id/A02P
[0003.458] W> "odm-data" doesn't exist, creating
[0003.462] I> Adding /chosen/plugin-manager/odm-data
[0003.470] I> added [base:0x80000000, size:0x70000000] to /memory
[0003.476] I> added [base:0xf0200000, size:0x85600000] to /memory
[0003.482] I> added [base:0x175e00000, size:0x200000] to /memory
[0003.488] I> added [base:0x176600000, size:0x200000] to /memory
[0003.493] I> added [base:0x177000000, size:0x200000] to /memory
[0003.499] I> Updated memory info to DTB
[0003.503] W> add_disp_param: failed to get display params for du=0
[0003.510] W> "reset" doesn't exist, creating
[0003.514] W> "pmc-reset-reason" doesn't exist, creating
[0003.520] W> "pmic-reset-reason" doesn't exist, creating
[0003.526] I> Adding ecid(000000016429e045040000000e050180) to DT
[0003.532] I> disabled_core_mask: 0xffffff0c
[0003.540] I> Add serial number:1422022054284 as DT property
[0003.547] I> Plugin-manager override starting
[0003.552] I> node /plugin-manager/fragement@0 matches
[0003.560] I> node /plugin-manager/fragement@3 matches
[0003.571] I> Disable plugin-manager status in FDT
[0003.576] I> Plugin-manager override finished successfully
[0003.581] I> tegrabl_load_kernel_and_dtb: Done
[0003.585] W> tegrabl_display_clear: display is not initialized
[0003.591] W> Boot logo display failed...
[0003.595] I> Kernel EP: 0x85600000, DTB: 0x80000000


U-Boot 2020.04-g4335beb (Apr 17 2022 - 02:21:01 -0700)

SoC: tegra186
Model: NVIDIA P3636-0001
Board: NVIDIA P3636-0001
DRAM:  3.8 GiB
MMC:   sdhci@3400000: 1, sdhci@3460000: 0
Loading Environment from MMC... *** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:
Warning: ethernet@2490000 using MAC address from ROM
eth0: ethernet@2490000
Hit any key to stop autoboot:  0
Card did not respond to voltage select!
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
1377 bytes read in 41 ms (32.2 KiB/s)
1:      primary kernel
Retrieving file: /boot/initrd
11354849 bytes read in 308 ms (35.2 MiB/s)
Retrieving file: /boot/Image
34560008 bytes read in 857 ms (38.5 MiB/s)
append: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 fbcon=map:0 net.ifnames=0 isolcpus=1-2  video=tegrafb earlycon=uart8250,mmio32,0x3100000 nvdumper_reserved=0x1772e0000 gpt rootfs.slot_suffix= usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 no_console_suspend boot.slot_suffix= boot.ratchetvalues=0.2031647.1 vpr_resize bl_prof_dataptr=0x10000@0x175840000 sdhci_tegra.en_boot_part_access=1 root=UUID=c96c0f1b-6a96-453a-831f-b3ac7a1a0d3d rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 isolcpus=1-2
## Flattened Device Tree blob at 80000000
   Booting using the fdt blob at 0x80000000
ERROR: reserving fdt memory region failed (addr=0 size=0)
ERROR: reserving fdt memory region failed (addr=0 size=0)
ERROR: reserving fdt memory region failed (addr=0 size=0)
   Using Device Tree in place at 0000000080000000, end 000000008003449f
copying carveout for /host1x@13e00000/display-hub@15200000/display@15200000...
copying carveout for /host1x@13e00000/display-hub@15200000/display@15210000...
copying carveout for /host1x@13e00000/display-hub@15200000/display@15220000...

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x100
[    0.000000] Linux version 4.9.253-tegra (flasher@ubuntu) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #7 SMP PREEMPT Tue Dec 10 01:54:40 PST 2024
[    0.000000] Boot CPU: AArch64 Processor [411fd073]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 80,
[    0.000000] OF: fdt: - 80000000 ,  70000000
[    0.000000] OF: fdt: - f0200000 ,  85600000
[    0.000000] OF: fdt: - 175e00000 ,  200000
[    0.000000] OF: fdt: - 176600000 ,  200000
[    0.000000] OF: fdt: - 177000000 ,  200000
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000003100000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fc000000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 24 pages/cpu s57624 r8192 d32488 u98304
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 990776
[    0.000000] Kernel command line: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 fbcon=map:0 net.ifnames=0 isolcpus=1-2  video=tegrafb earlycon=uart8250,mmio32,0x3100000 nvdumper_reserved=0x1772e0000 gpt rootfs.slot_suffix= usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 no_console_suspend boot.slot_suffix= boot.ratchetvalues=0.2031647.1 vpr_resize bl_prof_dataptr=0x10000@0x175840000 sdhci_tegra.en_boot_part_access=1 root=UUID=c96c0f1b-6a96-453a-831f-b3ac7a1a0d3d rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 isolcpus=1-2
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 163840 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 262144 bytes
[    0.000000] early log buf free: 29980(91%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Memory: 3152024K/4026368K available (15358K kernel code, 2966K rwdata, 6748K rodata, 8640K init, 610K bss, 120680K reserved, 753664K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f80000   ( 15360 KB)
[    0.000000]     .rodata : 0xffffff8008f80000 - 0xffffff8009620000   (  6784 KB)
[    0.000000]       .init : 0xffffff8009620000 - 0xffffff8009e90000   (  8640 KB)
[    0.000000]       .data : 0xffffff8009e90000 - 0xffffff800a175808   (  2967 KB)
[    0.000000]        .bss : 0xffffff800a175808 - 0xffffff800a20e0f4   (   611 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf03dc8000   (    61 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0f7200000   (  3954 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 64.
[    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=6.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=6
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000003] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.009243] Console: colour dummy device 80x25
[    0.013917] console [tty0] enabled
[    0.017483] bootconsole [uart8250] disabled
[    0.000000] Booting Linux on physical CPU 0x100
[    0.000000] Linux version 4.9.253-tegra (flasher@ubuntu) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #7 SMP PREEMPT Tue Dec 10 01:54:40 PST 2024
[    0.000000] Boot CPU: AArch64 Processor [411fd073]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 80,
[    0.000000] OF: fdt: - 80000000 ,  70000000
[    0.000000] OF: fdt: - f0200000 ,  85600000
[    0.000000] OF: fdt: - 175e00000 ,  200000
[    0.000000] OF: fdt: - 176600000 ,  200000
[    0.000000] OF: fdt: - 177000000 ,  200000
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000003100000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fc000000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 24 pages/cpu s57624 r8192 d32488 u98304
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 990776
[    0.000000] Kernel command line: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 fbcon=map:0 net.ifnames=0 isolcpus=1-2  video=tegrafb earlycon=uart8250,mmio32,0x3100000 nvdumper_reserved=0x1772e0000 gpt rootfs.slot_suffix= usbcore.old_scheme_first=1 tegraid=18.1.2.0.0 maxcpus=6 no_console_suspend boot.slot_suffix= boot.ratchetvalues=0.2031647.1 vpr_resize bl_prof_dataptr=0x10000@0x175840000 sdhci_tegra.en_boot_part_access=1 root=UUID=c96c0f1b-6a96-453a-831f-b3ac7a1a0d3d rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 isolcpus=1-2
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 163840 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 262144 bytes
[    0.000000] early log buf free: 29980(91%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Memory: 3152024K/4026368K available (15358K kernel code, 2966K rwdata, 6748K rodata, 8640K init, 610K bss, 120680K reserved, 753664K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f80000   ( 15360 KB)
[    0.000000]     .rodata : 0xffffff8008f80000 - 0xffffff8009620000   (  6784 KB)
[    0.000000]       .init : 0xffffff8009620000 - 0xffffff8009e90000   (  8640 KB)
[    0.000000]       .data : 0xffffff8009e90000 - 0xffffff800a175808   (  2967 KB)
[    0.000000]        .bss : 0xffffff800a175808 - 0xffffff800a20e0f4   (   611 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf03dc8000   (    61 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0f7200000   (  3954 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 64.
[    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=6.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=6
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000003] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.009243] Console: colour dummy device 80x25
[    0.013917] console [tty0] enabled
[    0.017483] bootconsole [uart8250] disabled
[    0.021870] kmemleak: Kernel memory leak detector disabled
[    0.021885] Calibrating delay loop (skipped), value calculated using timer frequency.. 62.50 BogoMIPS (lpj=125000)
[    0.021900] pid_max: default: 32768 minimum: 301
[    0.022209] Security Framework initialized
[    0.022369] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.022378] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.022956] ftrace: allocating 47208 entries in 185 pages
[    0.147077] sched-energy: Sched-energy-costs installed from DT
[    0.147096] ASID allocator initialised with 65536 entries
[    0.181233] tegra-id: chipid=21817.
[    0.181247] tegra-id: opt_subrevision=1.
[    0.181270] Tegra Revision: A02p SKU: 0xdc CPU Process: 0 SoC Process: 0
[    0.181282] DTS File Name: /home/flasher/kernel-development/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t18x/lanai/kernel-dts/tegra186-p3636-0001-p3509-0000-a01.dts
[    0.181297] DTB Build time: <unknown>
[    0.230973] CPU1: Booted secondary processor [4e0f0030]
[    0.262359] CPU2: Booted secondary processor [4e0f0030]
[    0.293382] CPU3: Booted secondary processor [411fd073]
[    0.325349] CPU4: Booted secondary processor [411fd073]
[    0.357342] CPU5: Booted secondary processor [411fd073]
[    0.357399] Brought up 6 CPUs
[    0.357430] SMP: Total of 6 processors activated.
[    0.357441] CPU features: detected feature: 32-bit EL0 Support
[    0.359530] CPU: All CPU(s) started at EL2
[    0.359707] alternatives: patching kernel code
[    0.362212] devtmpfs: initialized
[    0.384525] Initilizing CustomIPI irq domain
[    0.384681] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.384704] futex hash table entries: 2048 (order: 5, 131072 bytes)
[    0.392580] pinctrl core: initialized pinctrl subsystem
[    0.392837] OS set in device tree is not L4T.
[    0.393005] regulator-dummy: no parameters
[    0.393147] Initializing plugin-manager
[    0.393214] Plugin module not found
[    0.393255] Plugin-manager status disabled
[    0.394342] NET: Registered protocol family 16
[    0.395255] pstore: using zlib compression
[    0.395522] console [pstore-1] enabled
[    0.395533] pstore: Registered ramoops as persistent store backend
[    0.395545] ramoops: attached 0x200000@0x175880000, ecc: 0/0
[    0.409395] cpuidle: using governor menu
[    0.410378] bpmp: waiting for handshake
[    0.410393] bpmp: handshake completed
[    0.410420] bpmp: synchronizing channels
[    0.410432] bpmp: channels synchronized
[    0.410443] bpmp: mail init ok
[    0.411947] arm-smmu 12000000.iommu: probing hardware configuration...
[    0.411965] arm-smmu 12000000.iommu: SMMUv2 with:
[    0.411978] arm-smmu 12000000.iommu:         stage 1 translation
[    0.411990] arm-smmu 12000000.iommu:         stage 2 translation
[    0.412003] arm-smmu 12000000.iommu:         nested translation
[    0.412018] arm-smmu 12000000.iommu:         stream matching with 128 register groups, mask 0x7f80
[    0.412036] arm-smmu 12000000.iommu: SMMU address space size (0x800000) differs from mapped region size (0x1000000)!
[    0.412054] arm-smmu 12000000.iommu:         64 context banks (0 stage-2 only)
[    0.412068] arm-smmu 12000000.iommu:         Stage-1: 39-bit VA -> 48-bit IPA
[    0.412080] arm-smmu 12000000.iommu:         Stage-2: 39-bit IPA -> 48-bit PA
[    0.413806] vdso: 2 pages (1 code @ ffffff8008f87000, 1 data @ ffffff8009e94000)
[    0.413835] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.417029] atomic_pool_init():622: DMA: preallocated 1024 KiB pool for atomic allocations
[    0.418021] Registering BPMP clocks...
[    0.418100] tegra_bpmp_clk_init: clock init ok (638 clks)
[    0.418560] Serial: AMBA PL011 UART driver
[    0.419449] tegra_powergate_init: DONE
[    0.419466] DTS File Name: /home/flasher/kernel-development/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t18x/lanai/kernel-dts/tegra186-p3636-0001-p3509-0000-a01.dts
[    0.419492] DTB Build time: <unknown>
[    0.420795] Tegra reboot handler registered.
[    0.421944] tegra-pmc c360000.pmc: scratch reg offset dts data not present
[    0.422017] tegra-pmc: get_secure_pmc_setting: done secure_pmc=0
[    0.422039] tegra-pmc: ### PMC reset source: TEGRA_POWER_ON_RESET
[    0.422051] tegra-pmc: ### PMC reset level: TEGRA_RESET_LEVEL_L0
[    0.422062] tegra-pmc: ### PMC reset status reg: 0x0
[    0.422222] padctrl padctrl.0: Pad control driver tegra-pmc-padctrl registered
[    0.422248] tegra-pmc c360000.pmc: IO padctrl driver initialized
[    0.422525] iommu: Adding device 3460000.sdhci to group 0
[    0.423744] iommu: Adding device 3160000.i2c to group 1
[    0.424148] iommu: Adding device c240000.i2c to group 2
[    0.424449] iommu: Adding device 3180000.i2c to group 3
[    0.424739] iommu: Adding device 3190000.i2c to group 4
[    0.425124] iommu: Adding device 31b0000.i2c to group 5
[    0.425480] iommu: Adding device 31c0000.i2c to group 6
[    0.425774] iommu: Adding device c250000.i2c to group 7
[    0.426070] iommu: Adding device 31e0000.i2c to group 8
[    0.427512] iommu: Adding device 3210000.spi to group 9
[    0.427818] iommu: Adding device c260000.spi to group 10
[    0.428465] iommu: Adding device 3100000.serial to group 11
[    0.428768] iommu: Adding device 3110000.serial to group 12
[    0.429070] iommu: Adding device c280000.serial to group 13
[    0.429386] iommu: Adding device 3130000.serial to group 14
[    0.430555] iommu: Adding device 2490000.ether_qos to group 15
[    0.430578] arm-smmu: forcing sodev map for 2490000.ether_qos
[    0.431408] iommu: Adding device b000000.rtcpu to group 16
[    0.433137] iommu: Adding device 13e10000.host1x to group 17
[    0.433463] iommu: Adding device 13e10000.host1x:ctx0 to group 18
[    0.433750] iommu: Adding device 13e10000.host1x:ctx1 to group 19
[    0.434045] iommu: Adding device 13e10000.host1x:ctx2 to group 20
[    0.434340] iommu: Adding device 13e10000.host1x:ctx3 to group 21
[    0.434629] iommu: Adding device 13e10000.host1x:ctx4 to group 22
[    0.434917] iommu: Adding device 13e10000.host1x:ctx5 to group 23
[    0.435218] iommu: Adding device 13e10000.host1x:ctx6 to group 24
[    0.435514] iommu: Adding device 13e10000.host1x:ctx7 to group 25
[    0.435938] iommu: Adding device 150c0000.nvcsi to group 26
[    0.436494] iommu: Adding device 15700000.vi to group 27
[    0.436817] iommu: Adding device 15600000.isp to group 28
[    0.437026] iommu: Adding device 15200000.dc_common to group 29
[    0.437555] iommu: Adding device 15200000.nvdisplay to group 30
[    0.437882] iommu: Adding device 15210000.nvdisplay to group 31
[    0.438337] iommu: Adding device 15340000.vic to group 32
[    0.438538] iommu: Adding device 154c0000.nvenc to group 33
[    0.438740] iommu: Adding device 15480000.nvdec to group 34
[    0.439061] iommu: Adding device 15380000.nvjpg to group 35
[    0.439263] iommu: Adding device 15500000.tsec to group 36
[    0.439467] iommu: Adding device 15100000.tsecb to group 37
[    0.439956] iommu: Adding device 15810000.se to group 38
[    0.440153] iommu: Adding device 15820000.se to group 39
[    0.440354] iommu: Adding device 15830000.se to group 40
[    0.440550] iommu: Adding device 15840000.se to group 41
[    0.441313] iommu: Adding device smmu_test to group 42
[    0.441894] mc: mapped MMIO address: 0xffffff800a7b0000 -> 0x2c10000
[    0.441917] mc: mapped MMIO address: 0xffffff800a7d0000 -> 0x2c20000
[    0.441937] mc: mapped MMIO address: 0xffffff800b110000 -> 0x2c30000
[    0.441960] mc: mapped MMIO address: 0xffffff800b130000 -> 0x2c40000
[    0.441979] mc: mapped MMIO address: 0xffffff800b150000 -> 0x2c50000
[    0.441997] nv-tegra-mc 2c10000.mc: No mssnvlink node
[    0.442016] mc-err: mcerr ops are set to t18x
[    0.442165] dram-ecc: DRAM ECC disabled-MC_ECC_CONTROL:0x0000000c
[    0.442928] iommu: Adding device 3530000.xhci to group 43
[    0.442955] arm-smmu: forcing sodev map for 3530000.xhci
[    0.443354] iommu: Adding device 3550000.xudc to group 44
[    0.443377] arm-smmu: forcing sodev map for 3550000.xudc
[    0.443876] tegra186-aowake c370000.pmc: WAKE_AOWAKE_CNTRL_83 = 0x10a
[    0.443894] tegra186-aowake c370000.pmc: WAKE_AOWAKE_CTRL_0 = 3
[    0.443908] tegra186-aowake c370000.pmc: WAKE_AOWAKE_CNTRL_24(PMU_INT) = 256
[    0.444492] iommu: Adding device 17000000.gp10b to group 45
[    0.445438] tegra-reset 5000000.clock: registered 193 resets.
[    0.447319] iommu: Adding device d000000.bpmp to group 46
[    0.447534] bpmp: ping status is 0
[    0.447619] bpmp d000000.bpmp: firmware tag is 91572a54614f84d0fd0c270beec2c56f
[    0.449238] bpmp d000000.bpmp: probe ok
[    0.451878] iommu: Adding device 2600000.dma to group 47
[    0.452713] GPIO line 408 (w-disable1) hogged as output/high
[    0.452763] GPIO line 410 (w-disable2) hogged as output/high
[    0.452934] gpiochip_setup_dev: registered GPIOs 320 to 511 on device: gpiochip0 (tegra-gpio)
[    0.456768] gpiochip_setup_dev: registered GPIOs 256 to 319 on device: gpiochip1 (tegra-gpio-aon)
[    0.458385] iommu: Adding device 10003000.pcie-controller to group 48
[    0.458412] arm-smmu: forcing sodev map for 10003000.pcie-controller
[    0.458729] iommu: Adding device sound to group 49
[    0.459044] iommu: Adding device 3510000.hda to group 50
[    0.465483] iommu: Adding device c1a0000.aon to group 51
[    0.466011] vdd-1v8-ap: 1800 mV
[    0.466230] vdd-ac-bat: 5000 mV
[    0.466422] vdd-3v3-disp-cvb: 3300 mV
[    0.466698] vdd-3v3-cvb: 3300 mV
[    0.466887] vdd-1v8-cvb: 1800 mV
[    0.467074] vdd-fan: 5000 mV
[    0.467275] vdd-hdmi-5v0: 5000 mV
[    0.467466] vdd_sys_en: 1200 mV
[    0.497788] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.500359] eventlib_kernel: keventlib is initialized, test id: 0
[    0.500589] SCSI subsystem initialized
[    0.500885] usbcore: registered new interface driver usbfs
[    0.500966] usbcore: registered new interface driver hub
[    0.501006] usbcore: registered new device driver usb
[    0.502964] random: fast init done
[    0.510562] max77620 4-003c: PMIC Version OTP:0x45 and ES:0x8
[    0.513068] vddio-ddr: at 1125 mV
[    0.516926] avdd_dsi_csi: 1200 mV
[    0.520924] vdd-1v8: 1800 mV
[    0.524923] vdd-3v3-sys: 3300 mV
[    0.525790] spmic-ldo0: at 1800 mV
[    0.526438] spmic-ldo1: at 800 mV
[    0.528926] vddio-3v3: 3300 mV
[    0.529599] vddio-sdmmc1: 1800 <--> 3300 mV at 3300 mV
[    0.532926] vdd-rtc: at 800 mV
[    0.533620] avdd-ts-hv: 1800 <--> 3300 mV at 1800 mV
[    0.536925] spmic-ldo6: at 800 mV
[    0.540924] vdd-pex-1v00: 1000 mV
[    0.541523] dvdd-pex: 1000 mV
[    0.543545] GPIO line 253 (spmic_gpio_input) hogged as input
[    0.543691] GPIO line 254 (spmic_gpio_input) hogged as input
[    0.543858] gpiochip_setup_dev: registered GPIOs 248 to 255 on device: gpiochip2 (max77620-gpio)
[    0.544051] max77620 4-003c: max77620 probe successful
[    0.544417] media: Linux media interface: v0.10
[    0.544455] Linux video capture interface: v2.00
[    0.547626] pps_core: LinuxPPS API ver. 1 registered
[    0.547640] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.547668] PTP clock support registered
[    0.547866] tegra_wdt_t18x 30c0000.watchdog: Expiry count is deprecated
[    0.548061] tegra_wdt_t18x 30c0000.watchdog: Tegra WDT init timeout = 120 sec
[    0.548095] tegra_wdt_t18x 30c0000.watchdog: Registered successfully
[    0.549295] trusty trusty: trusty version: Built: 16:08:32 Jun 11 2024
[    0.549323] trusty trusty: selected api version: 3 (requested 3)
[    0.550109] Advanced Linux Sound Architecture Driver Initialized.
[    0.550446] Bluetooth: Core ver 2.22
[    0.550484] NET: Registered protocol family 31
[    0.550496] Bluetooth: HCI device and connection manager initialized
[    0.550513] Bluetooth: HCI socket layer initialized
[    0.550527] Bluetooth: L2CAP socket layer initialized
[    0.550549] Bluetooth: SCO socket layer initialized
[    0.551104] vdd-sys-bl: 3300 mV
[    0.552736] camchar: rtcpu character device driver loaded
[    0.553194] gpio tegra-gpio wake61 for gpio=92(L:4)
[    0.553277] extcon-gpio-states external-connection:extcon@1: Cable state:1, cable id:1
[    0.554235] clocksource: Switched to clocksource arch_sys_counter
[    0.576436] VFS: Disk quotas dquot_6.6.0
[    0.576527] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.576784] nvmap_heap_init: nvmap_heap_init: created heap block cache
[    0.577229] dma_declare_coherent_resizable_cma_memory:324: resizable heap=vpr, base=0x00000000c6000000, size=0x2a000000
[    0.577353] cma: enabled page replacement for spfn=c6000, epfn=f0000
[    0.577368] dma_declare_coherent_resizable_cma_memory:373: resizable cma heap=vpr create successful
[    0.577385] nvmap: nvmap_select_cache_ops() nvmap cache ops set to roc
[    0.577402] nvmap_page_pool_init: Total RAM pages: 976422
[    0.577414] nvmap_page_pool_init: nvmap page pool size: 122052 pages (476 MB)
[    0.577469] nvmap_background_zero_thread: PP zeroing thread starting.
[    0.577936] misc nvmap: created heap vpr base 0x00000000c6000000 size (688128KiB)
[    0.580021] tegra-gpcdma 2600000.dma: GPC DMA driver register 32 channels
[    0.585677] thermal thermal_zone0: Registering thermal zone thermal_zone0 for type BCPU-therm
[    0.585879] thermal thermal_zone1: Registering thermal zone thermal_zone1 for type MCPU-therm
[    0.586236] thermal thermal_zone2: Registering thermal zone thermal_zone2 for type GPU-therm
[    0.586324] thermal thermal_zone3: Registering thermal zone thermal_zone3 for type PLL-therm
[    0.586475] thermal thermal_zone4: Registering thermal zone thermal_zone4 for type PMIC-Die
[    0.586707] la/ptsa driver initialized.
[    0.586726] pre_t19x_iso_plat_init(): iso emc max clk=1600000KHz
[    0.586737] pre_t19x_iso_plat_init(): max_iso_bw=23040000KB
[    0.587033] NET: Registered protocol family 2
[    0.587687] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    0.587858] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    0.588164] TCP: Hash tables configured (established 32768 bind 32768)
[    0.588221] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    0.588290] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    0.588564] NET: Registered protocol family 1
[    0.588868] RPC: Registered named UNIX socket transport module.
[    0.588883] RPC: Registered udp transport module.
[    0.588893] RPC: Registered tcp transport module.
[    0.588904] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.589024] Trying to unpack rootfs image as initramfs...
[    0.918993] Freeing initrd memory: 11088K
[    0.933662] host1x 13e10000.host1x: initialized
[    0.934941] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[    0.937089] audit: initializing netlink subsys (disabled)
[    0.937129] audit: type=2000 audit(0.816:1): initialized
[    0.937526] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.943969] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.944936] ntfs: driver 2.1.32 [Flags: R/W].
[    0.945457] 9p: Installing v9fs 9p2000 file system support
[    0.947393] Key type asymmetric registered
[    0.947408] Asymmetric key parser 'x509' registered
[    0.947593] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 240)
[    0.947664] io scheduler noop registered
[    0.947871] io scheduler cfq registered (default)
[    0.950699] gic 2a41000.agic-controller: GIC IRQ controller registered
[    1.011436] iommu: Adding device aconnect@2a41000:adsp_audio to group 52
[    1.031637] iommu: Adding device 2993000.adsp to group 53
[    1.032185] tegra-aconnect aconnect@2a41000: Tegra ACONNECT bus registered
[    1.036875] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[    1.037769] tegra-pwm 3290000.pwm: PWM clk can sleep in ops
[    1.038602] tegra-pwm 32a0000.pwm: PWM clk can sleep in ops
[    1.039399] tegra-pwm c340000.pwm: PWM clk can sleep in ops
[    1.041297] tegra-pcie 10003000.pcie-controller: 2x1, 1x1, 1x1 configuration
[    1.042023] tegra-pcie 10003000.pcie-controller: PCIE: Enable power rails
[    1.042338] tegra-pcie 10003000.pcie-controller: probing port 0, using 2 lanes
[    1.042523] tegra_camera_platform tegra-camera-platform: tegra_camera_probe:camera_platform_driver probe
[    1.042647] misc tegra_camera_ctrl: tegra_camera_isomgr_register tpg_max_iso = 3916800KBs
[    1.042669] misc tegra_camera_ctrl: tegra_camera_isomgr_register isp_iso_bw=1000000, vi_iso_bw=1600000, max_bw=3916800
[    1.044533] tegra-pcie 10003000.pcie-controller: probing port 2, using 1 lanes
[    1.044675] tsec 15500000.tsec: initialized
[    1.045589] tsec 15100000.tsecb: initialized
[    1.047728] nvdec 15480000.nvdec: initialized
[    1.051383] falcon 15340000.vic: initialized
[    1.052493] falcon 154c0000.nvenc: initialized
[    1.053533] falcon 15380000.nvjpg: initialized
[    1.055656] iommu_context_dev 13e10000.host1x:ctx0: initialized (streamid=56)
[    1.057177] iommu_context_dev 13e10000.host1x:ctx1: initialized (streamid=57)
[    1.058741] iommu_context_dev 13e10000.host1x:ctx2: initialized (streamid=58)
[    1.060267] iommu_context_dev 13e10000.host1x:ctx3: initialized (streamid=59)
[    1.061771] iommu_context_dev 13e10000.host1x:ctx4: initialized (streamid=60)
[    1.063355] iommu_context_dev 13e10000.host1x:ctx5: initialized (streamid=61)
[    1.064891] iommu_context_dev 13e10000.host1x:ctx6: initialized (streamid=62)
[    1.066416] iommu_context_dev 13e10000.host1x:ctx7: initialized (streamid=63)
[    1.069087] tegradccommon 15200000.dc_common: host1x channel mapped
[    1.069109] tegradccommon 15200000.dc_common: dc_common syncpt # 1 allocated
[    1.069147] tegradccommon 15200000.dc_common: dma mapping done
[    1.069627] tegradc 15200000.nvdisplay: disp0 connected to head0->/host1x/dsi
[    1.069774] tegradc 15200000.nvdisplay: No hpd-gpio in DT
[    1.069826] tegradc 15200000.nvdisplay: DT parsed successfully
[    1.069855] tegradc 15200000.nvdisplay: Display dc.ffffff800b3e0000 registered with id=0
[    1.075472] tegra_nvdisp_bandwidth_register_max_config: max config iso bw = 16727000 KB/s
[    1.075493] tegra_nvdisp_bandwidth_register_max_config: max config EMC floor = 665600000 Hz
[    1.075512] tegra_nvdisp_bandwidth_register_max_config: max config hubclk = 357620000 Hz
[    1.075687] tegradc 15200000.nvdisplay: vblank syncpt # 8 for dc 0
[    1.075703] tegradc 15200000.nvdisplay: vpulse3 syncpt # 9 for dc 0
[    1.080119] tegradc 15200000.nvdisplay: probed
[    1.080526] Console: switching to colour frame buffer device 60x50
[    1.080560] tegradc 15200000.nvdisplay: fb registered
[    1.081665] tegra-adma 2930000.adma: Tegra210 ADMA driver registered 32 channels
[    1.082206] tegra_nvdisp_handle_pd_enable: Unpowergated Head0 pd
[    1.082279] tegra-fuse-burn 3820000.efuse:efuse-burn: shutdown limit check disabled
[    1.082285] tegra_nvdisp_handle_pd_enable: Unpowergated Head1 pd
[    1.082316] tegra-fuse-burn 3820000.efuse:efuse-burn: Fuse burn driver initialized
[    1.083099] MIPI Display: enable
[    1.083116] kfuse 3830000.kfuse: initialized
[    1.084590] tegra-pmc-iopower pmc-iopower: NO_IOPOWER setting 0x0
[    1.086396] tegradc 15200000.nvdisplay: DSI: HS clock rate is 152500 305 38097000
[    1.087981] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.092235] console [ttyS0] disabled
[    1.092286] 3100000.serial: ttyS0 at MMIO 0x3100000 (irq = 33, base_baud = 25500000) is a Tegra
[    1.466112] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    1.868119] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    2.270110] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    2.272124] tegra-pcie 10003000.pcie-controller: link 0 down, ignoring
[    2.672119] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    3.074109] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    3.284230] console [ttyS0] enabled
[    3.288897] 3110000.serial: ttyTHS1 at MMIO 0x3110000 (irq = 34, base_baud = 0) is a TEGRA_UART
[    3.298310] c280000.serial: ttyTHS2 at MMIO 0xc280000 (irq = 35, base_baud = 0) is a TEGRA_UART
[    3.307172] serial-tegra 3130000.serial: RX in PIO mode
[    3.312980] 3130000.serial: ttyTHS3 at MMIO 0x3130000 (irq = 36, base_baud = 0) is a TEGRA_UART
[    3.322607] [drm] Initialized
[    3.496120] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    3.504666] tegra-pcie 10003000.pcie-controller: link 2 down, ignoring
[    3.715382] tegra-pcie 10003000.pcie-controller: PCIE: no end points detected
[   19.338125] random: crng init done
[   22.234104] INFO: rcu_preempt detected stalls on CPUs/tasks:
[   22.239777]  3-...: (1 GPs behind) idle=813/140000000000000/0 softirq=194/199 fqs=25
[   22.247598]  (detected by 4, t=5281 jiffies, g=-219, c=-220, q=434)
[   22.253872] Task dump for CPU 3:
[   22.257096] kworker/u12:0   R  running task        0     6      2 0x00000002
[   22.264177] Workqueue: events_unbound async_run_entry_fn
[   22.269493] Call trace:
[   22.271943] [<ffffff800808604c>] __switch_to+0x9c/0xc0
[   22.277081] [<ffffff8009846000>] bp_hardening_data+0x0/0x10
[   24.262331] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 3
[   24.269778] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.9.253-tegra #7
[   24.276412] Hardware name: lanai-3636 (DT)
[   24.280585] Call trace:
[   24.283122] [<ffffff800808ba40>] dump_backtrace+0x0/0x198
[   24.288712] [<ffffff800808c004>] show_stack+0x24/0x30
[   24.293943] [<ffffff8008f6a0d4>] dump_stack+0xa0/0xc4
[   24.299184] [<ffffff8008f67178>] panic+0x12c/0x2a8
[   24.304160] [<ffffff8008181994>] watchdog_check_hardlockup_other_cpu+0x11c/0x120
[   24.311736] [<ffffff8008180b08>] watchdog_timer_fn+0x98/0x2c0
[   24.317611] [<ffffff8008138f10>] __hrtimer_run_queues+0xd8/0x360
[   24.323795] [<ffffff8008139860>] hrtimer_interrupt+0xa8/0x1e0
[   24.329714] [<ffffff8008c03a0c>] tegra186_timer_isr+0x34/0x48
[   24.335691] [<ffffff8008121940>] __handle_irq_event_percpu+0x68/0x288
[   24.342310] [<ffffff8008121b88>] handle_irq_event_percpu+0x28/0x60
[   24.348675] [<ffffff8008121c10>] handle_irq_event+0x50/0x80
[   24.354413] [<ffffff8008125aa4>] handle_fasteoi_irq+0xd4/0x1c0
[   24.360396] [<ffffff80081208f4>] generic_handle_irq+0x34/0x50
[   24.366303] [<ffffff8008120fe0>] __handle_domain_irq+0x68/0xc0
[   24.372288] [<ffffff8008080d44>] gic_handle_irq+0x5c/0xb0
[   24.377832] [<ffffff8008082c28>] el1_irq+0xe8/0x194
[   24.382855] [<ffffff80080859b0>] arch_cpu_idle+0x30/0x1b0
[   24.388419] [<ffffff80081116a8>] cpu_startup_entry+0x140/0x200
[   24.394375] [<ffffff8008091cf8>] secondary_start_kernel+0x190/0x1f8
[   24.400698] [<00000000811771a8>] 0x811771a8
[   24.405034] SMP: stopping secondary CPUs
[   25.836941] SMP: failed to stop secondary CPUs 0,2-3,5
[   25.842215] Kernel Offset: disabled
[   25.845882] Memory Limit: none
[   25.879706] Rebooting in 5 seconds..
[   30.910520] SMP: stopping secondary CPUs
[   32.133074] SMP: failed to stop secondary CPUs 0,2-3,5

The bootloop is always identical. If I flash a completely stock TX2NX image from SDK Manager on these broken Jetson’s, it works fine. I would have thought that the images are misconfigured but they do work on other 10 Jetsos’s just fine.

How can this be explained? Is it something that have killed the Jetson on specific custom pinmux pins that are different from the stock pinmux?

Which Jetpack release is in use? This seems not a easy one to debug from such remote way.

I would say you must try to reproduce issue on NV devkit first. Otherwise we may not be able to help.

Hello!

I have found the underlying issue. It’s a very interesting one. Apparently two of my carrier boards, on which the Jetson’s have been on at one or another point in time, had 2 CSI lanes shorted, which kills the Jetson pins as well - only after some time though.

Whenever I am flashing TX2NX with a DeviceTree that uses those CSI lanes, this issue happens. As stock image doesn’t use those lanes, the bootloop is not happening. Still seems weird why is this not booting at all.