After installing linuxptp on a freshly flashed Jetpack 4.2, my Xavier does not come up after reboot. Flashing with or without additional software (CUDA, tensorRT, …) does not make a difference.
Attached is the boot log captured with a serial console.
[ 11.787781] systemd[1]: Listening on Syslog Socket.
[ 11.875602] EXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
[ 11.892042] gpio tegra-gpio wake44 for gpio=192(Y:0)
[ 11.949619] nvgpu: 17000000.gv11b nvgpu_nvhost_syncpt_init:291 [INFO] syncpt_unit_base 60000000 syncpt_unit_size 400000 size 1000
[ 11.949619]
[ 12.577676] bmi160 1-0069: nvs_vregs_init vdd err=-19
[ 12.581817] bmi160 1-0069: nvs_vregs_init vdd_IO err=-19
[ 12.641324] tegra-i2c c240000.i2c: no acknowledge from address 0x69
[ 12.644897] bmi160 1-0069: bmi_i2c_rd ERR: 0x00
[ 12.648553] bmi160 1-0069: bmi_init _id_i2c ERR
[ 12.653423] bmi160 8-0069: nvs_vregs_init vdd err=-19
[ 12.659003] bmi160 8-0069: nvs_vregs_init vdd_IO err=-19
[ 12.729313] tegra-i2c 31e0000.i2c: no acknowledge from address 0x69
[ 12.732933] bmi160 8-0069: bmi_i2c_rd ERR: 0x00
[ 12.737043] bmi160 8-0069: bmi_init _id_i2c ERR
[ 17.828274] CPU2: SError detected, daif=140, spsr=0x60000000, mpidr=80000100, esr=be000000
[ 17.828290] CPU1: SError detected, daif=1c0, spsr=0x40c000c5, mpidr=80000001, esr=be000000
[ 17.828296] CPU7: SError detected, daif=1c0, spsr=0x40c000c5, mpidr=80000301, esr=be000000
[ 17.828300] CPU3: SError detected, daif=140, spsr=0x40400045, mpidr=80000101, esr=be000000
[ 17.828324] CPU4: SError detected, daif=140, spsr=0x60400145, mpidr=80000200, esr=be000000
[ 17.828341] CPU5: SError detected, daif=140, spsr=0x20400145, mpidr=80000201, esr=be000000
[ 17.828362] CPU6: SError detected, daif=1c0, spsr=0x80c000c5, mpidr=80000300, esr=be000000
[ 17.828383] **************************************
[ 17.828384] * For more Internal Decode Help
[ 17.828384] * http://nv/cbberr
[ 17.828385] * NVIDIA userID is required to access
[ 17.828386] **************************************
[ 17.828389] CPU:1, Error:CBB-NOC
[ 17.828392] Error Logger : 0
[ 17.828400] ErrLog0 : 0x80030000
[ 17.828401] Transaction Type : RD - Read, Incrementing
[ 17.828402] Error Code : SLV
[ 17.828403] Error Source : Target
[ 17.828404] Error Description : Target error detected by CBB slave
[ 17.828414] AXI2APB_4 bridge error: SFIFONE - Status FIFO Not Empty interrupt
[ 17.828417] AXI2APB_4 bridge error: TIM - Timer(Timeout) interrupt
[ 17.828418] CPU:0, Error:CBB-NOC@0x2300000,irq=486
[ 17.828421] Packet header Lock : 0
[ 17.828422] **************************************
[ 17.828424] Packet header Len1 : 3
[ 17.828425] * For more Internal Decode Help
[ 17.828426] NOC protocol version : version >= 2.7
[ 17.828427] * http://nv/cbberr
[ 17.828429] ErrLog1 : 0x313820
[ 17.828430] * NVIDIA userID is required to access
[ 17.828431] ErrLog2 : 0x0
[ 17.828432] **************************************
[ 17.828433] RouteId : 0x313820
[ 17.828434] CPU:0, Error:CBB-NOC
[ 17.828436] InitFlow : ccroc_p2ps/I/ccroc_p2ps
[ 17.828437] Error Logger : 0
[ 17.828439] Targflow : axis_satellite_axi2apb_p2pm/T/axis_satellite_axi2apb_p2pm
[ 17.828440] TargSubRange : 28
[ 17.828441] SeqId : 0
[ 17.828442] ErrLog3 : 0xb00
[ 17.828442] ErrLog4 : 0x0
[ 17.828444] ErrLog0 : 0x80030000
[ 17.828445] Transaction Type : RD - Read, Incrementing
[ 17.828446] Error Code : SLV
[ 17.828447] Error Source : Target
[ 17.828448] Error Description : Target error detected by CBB slave
[ 17.828458] AXI2APB_4 bridge error: SFIFONE - Status FIFO Not Empty interrupt
[ 17.828458] Packet header Lock : 0
[ 17.828459] Packet header Len1 : 3
[ 17.828460] NOC protocol version : version >= 2.7
[ 17.828461] ErrLog1 : 0x313820
[ 17.828462] ErrLog2 : 0x0
[ 17.828463] RouteId : 0x313820
[ 17.828464] InitFlow : ccroc_p2ps/I/ccroc_p2ps
[ 17.828465] Targflow : axis_satellite_axi2apb_p2pm/T/axis_satellite_axi2apb_p2pm
[ 17.828466] TargSubRange : 28
[ 17.828466] SeqId : 0
[ 17.828467] ErrLog3 : 0xb00
[ 17.828468] ErrLog4 : 0x0
[ 17.828474] Address : 0x2490b00 (unknown device)
[ 17.828476] ErrLog5 : 0x809f851
[ 17.828477] Non-Modify : 0x1
[ 17.828478] AXI ID : 0x10
[ 17.828479] Address : 0x2490b00 (unknown device)
[ 17.828480] Master ID : CCPLEX
[ 17.828481] ErrLog5 : 0x809f851
[ 17.828482] Security Group(GRPSEC): 0x7e
[ 17.828485] Non-Modify : 0x1
[ 17.828486] Cache : 0x1 -- Device
[ 17.828487] AXI ID : 0x10
[ 17.828489] Protection : 0x2 -- Unprivileged, Non-Secure, Data Access
[ 17.828490] Master ID : CCPLEX
[ 17.828491] FALCONSEC : 0x0
[ 17.828492] Security Group(GRPSEC): 0x7e
[ 17.828494] Virtual Queuing Channel(VQC): 0x0
[ 17.828495] Cache : 0x1 -- Device
[ 17.828497] Protection : 0x2 -- Unprivileged, Non-Secure, Data Access
[ 17.828498] **************************************
[ 17.828499] FALCONSEC : 0x0
[ 17.828501] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.828502] Virtual Queuing Channel(VQC): 0x0
[ 17.828504] **************************************
[ 17.828507] CPU0: SError detected, daif=140, spsr=0x40400145, mpidr=80000000, esr=be000000
[ 17.828521] **************************************
[ 17.828522] RAS Error in SCF:IOB:
[ 17.828523] Status = 0xf4009604
[ 17.828524] IERR = CBB Interface Error: 0x96
[ 17.828525] SERR = Assertion Failure: 0x4
[ 17.828526] Uncorrectable (this is fatal)
[ 17.828532] MISC0 = 0x40
[ 17.828533] MISC1 = 0x2646444423
[ 17.828536] ADDR = 0x8000000002490b00
[ 17.828540] **************************************
[ 17.828544] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.828561] **************************************
[ 17.828562] RAS Error in L2:
[ 17.828563] Status = 0xf4006612
[ 17.828564] IERR = SCF to L2 Slave Error Read: 0x66
[ 17.828565] SERR = Error response from slave: 0x12
[ 17.828565] Uncorrectable (this is fatal)
[ 17.828571] MISC0 = 0x100000000400000
[ 17.828571] MISC1 = 0x40240000000
[ 17.828574] ADDR = 0x8000000002490b00
[ 17.828578] **************************************
[ 17.828604] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.828692] Bad mode in Error handler detected on CPU1, code 0xbe000000 -- SError
[ 17.828696] Internal error: Oops - bad mode: 0 [#1] PREEMPT SMP
[ 17.828708] Modules linked in: binfmt_misc hid_cherry nvs_bmi160 nvs nvgpu bluedroid_pm ip_tables x_tables
[ 17.828716] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.9.140-tegra #1
[ 17.828717] Hardware name: jetson-xavier (DT)
[ 17.828719] task: ffffffc3ee3baa00 task.stack: ffffffc3ee3cc000
[ 17.828727] PC is at tegra_read_chipid+0x58/0xa8
[ 17.828729] LR is at tegra_read_chipid+0x28/0xa8
[ 17.828731] pc : [<ffffff80086a3428>] lr : [<ffffff80086a33f8>] pstate: 40c000c5
[ 17.828732] sp : ffffffc3ee3cfe50
[ 17.828739] x29: ffffffc3ee3cfe50 x28: 0000000000000001
[ 17.828741] x27: ffffff8009e26000 x26: ffffff80097fea48
[ 17.828744] x25: 0000000000000000 x24: 00000004222a98a0
[ 17.828746] x23: ffffff8009f9a318 x22: ffffffc3ffdc6a50
[ 17.828749] x21: ffffff8009f9a330 x20: ffffff800a155610
[ 17.828752] x19: ffffff800a155610 x18: 0000000000000002
[ 17.828754] x17: 000000000000000e x16: 0000000000000000
[ 17.828757] x15: 000000000000029b x14: 0000000000000019
[ 17.828759] x13: 0000000000000635 x12: 071c71c71c71c71c
[ 17.828762] x11: 000000000000000b x10: 0101010101010101
[ 17.828766] x9 : fffffffffffffffc x8 : 7f7f7f7f7f7f7f7f
[ 17.828768] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.828771] x7 : fefefeff646c606d x6 : 00170401e9e1acf4
[ 17.828774] x5 : 742c616901041700 x4 : 8080808000000000
[ 17.828776] x3 : b34b234b0963a000 x2 : 000000000000000b
[ 17.828779] x1 : ffffffc3ee3baa00 x0 : 0000000000021917
[ 17.828780]
[ 17.828783] Process swapper/1 (pid: 0, stack limit = 0xffffffc3ee3cc000)
[ 17.828784] Call trace:
[ 17.828788] [<ffffff80086a3428>] tegra_read_chipid+0x58/0xa8
[ 17.828789] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.828793] [<ffffff80086a377c>] tegra_get_platform+0x14/0x80
[ 17.828798] [<ffffff8008b8a0bc>] t19x_cpu_enter_state+0x1c/0x118
[ 17.828803] [<ffffff8008b85d64>] cpuidle_enter_state+0x84/0x380
[ 17.828806] [<ffffff8008b860d4>] cpuidle_enter+0x34/0x48
[ 17.828810] [<ffffff80081129b4>] call_cpuidle+0x44/0x70
[ 17.828812] [<ffffff8008112d30>] cpu_startup_entry+0x1b0/0x200
[ 17.828817] [<ffffff8008092ce8>] secondary_start_kernel+0x190/0x1f8
[ 17.828819] [<0000000080f441a4>] 0x80f441a4
[ 17.828824] ---[ end trace 47ef958400a7a83d ]---
[ 17.829921] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.830083] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.830102] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.830145] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.830317] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.830337] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.830380] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.836024] Kernel panic - not syncing: Attempted to kill the idle task!
[ 17.836026] SMP: stopping secondary CPUs
[ 17.912544] CPU7: SError detected, daif=1c0, spsr=0xc000c5, mpidr=80000301, esr=be000000
[ 17.994780] **************************************
[ 17.994781] * For more Internal Decode Help
[ 17.994782] * http://nv/cbberr
[ 17.994783] * NVIDIA userID is required to access
[ 17.994783] **************************************
[ 17.994784] CPU:5, Error:CBB-NOC
[ 17.994787] Error Logger : 0
[ 17.994792] ErrLog0 : 0x80030008
[ 17.994793] Transaction Type : WR - Write, Incrementing
[ 17.994795] Error Code : SLV
[ 17.994796] Error Source : Target
[ 17.994797] Error Description : Target error detected by CBB slave
[ 17.994805] AXI2APB_4 bridge error: SFIFONE - Status FIFO Not Empty interrupt
[ 17.994808] AXI2APB_4 bridge error: TIM - Timer(Timeout) interrupt
[ 17.994808] Packet header Lock : 0
[ 17.994809] Packet header Len1 : 3
[ 17.994810] NOC protocol version : version >= 2.7
[ 17.994811] ErrLog1 : 0x31380b
[ 17.994812] ErrLog2 : 0x0
[ 17.994813] RouteId : 0x31380b
[ 17.994817] InitFlow : ccroc_p2ps/I/ccroc_p2ps
[ 17.994818] Targflow : axis_satellite_axi2apb_p2pm/T/axis_satellite_axi2apb_p2pm
[ 17.994819] TargSubRange : 28
[ 17.994820] SeqId : 0
[ 17.994821] ErrLog3 : 0xb18
[ 17.994822] ErrLog4 : 0x0
[ 17.994838] Address : 0x2490b18 (unknown device)
[ 17.994839] ErrLog5 : 0x289f851
[ 17.994840] Non-Modify : 0x1
[ 17.994841] AXI ID : 0x5
[ 17.994842] Master ID : CCPLEX
[ 17.994843] Security Group(GRPSEC): 0x7e
[ 17.994844] Cache : 0x1 -- Device
[ 17.994846] Protection : 0x2 -- Unprivileged, Non-Secure, Data Access
[ 17.994847] FALCONSEC : 0x0
[ 17.994848] Virtual Queuing Channel(VQC): 0x0
[ 17.994850] **************************************
[ 17.994851] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.994867] **************************************
[ 17.994868] RAS Error in SCF:IOB:
[ 17.994869] Status = 0xfc009604
[ 17.994870] IERR = CBB Interface Error: 0x96
[ 17.994871] SERR = Assertion Failure: 0x4
[ 17.994872] Overflow (there may be more errors) - Uncorrectable
[ 17.994872] Uncorrectable (this is fatal)
[ 17.994877] MISC0 = 0x40
[ 17.994878] MISC1 = 0x1aa46404163
[ 17.994881] ADDR = 0x8000000002490b00
[ 17.994885] **************************************
[ 17.994888] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.994915] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.995040] **************************************
[ 17.995042] * For more Internal Decode Help
[ 17.995042] * http://nv/cbberr
[ 17.995043] * NVIDIA userID is required to access
[ 17.995044] **************************************
[ 17.995046] CPU:6, Error:CBB-NOC
[ 17.995048] Error Logger : 0
[ 17.995053] ErrLog0 : 0x80030000
[ 17.995055] Transaction Type : RD - Read, Incrementing
[ 17.995056] Error Code : SLV
[ 17.995057] Error Source : Target
[ 17.995058] Error Description : Target error detected by CBB slave
[ 17.995068] AXI2APB_4 bridge error: SFIFONE - Status FIFO Not Empty interrupt
[ 17.995068] Packet header Lock : 0
[ 17.995069] Packet header Len1 : 3
[ 17.995070] NOC protocol version : version >= 2.7
[ 17.995071] ErrLog1 : 0x313822
[ 17.995072] ErrLog2 : 0x0
[ 17.995072] RouteId : 0x313822
[ 17.995074] InitFlow : ccroc_p2ps/I/ccroc_p2ps
[ 17.995075] Targflow : axis_satellite_axi2apb_p2pm/T/axis_satellite_axi2apb_p2pm
[ 17.995076] TargSubRange : 28
[ 17.995077] SeqId : 0
[ 17.995078] ErrLog3 : 0xb00
[ 17.995079] ErrLog4 : 0x0
[ 17.995096] Address : 0x2490b00 (unknown device)
[ 17.995103] ErrLog5 : 0x889f851
[ 17.995104] Non-Modify : 0x1
[ 17.995105] AXI ID : 0x11
[ 17.995106] Master ID : CCPLEX
[ 17.995107] Security Group(GRPSEC): 0x7e
[ 17.995108] Cache : 0x1 -- Device
[ 17.995109] Protection : 0x2 -- Unprivileged, Non-Secure, Data Access
[ 17.995110] FALCONSEC : 0x0
[ 17.995111] Virtual Queuing Channel(VQC): 0x0
[ 17.995113] **************************************
[ 17.995115] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.995134] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.995156] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.995203] Bad mode in Error handler detected on CPU6, code 0xbe000000 -- SError
[ 17.995205] Internal error: Oops - bad mode: 0 [#2] PREEMPT SMP
[ 17.995215] Modules linked in: binfmt_misc hid_cherry nvs_bmi160 nvs nvgpu bluedroid_pm ip_tables x_tables
[ 17.995222] CPU: 6 PID: 0 Comm: swapper/6 Tainted: G D 4.9.140-tegra #1
[ 17.995223] Hardware name: jetson-xavier (DT)
[ 17.995225] task: ffffffc3ee3bf000 task.stack: ffffffc3ee3e8000
[ 17.995231] PC is at t19x_cpu_enter_state+0x4c/0x118
[ 17.995233] LR is at t19x_cpu_enter_state+0x1c/0x118
[ 17.995235] pc : [<ffffff8008b8a0ec>] lr : [<ffffff8008b8a0bc>] pstate: 80c000c5
[ 17.995236] sp : ffffffc3ee3ebe80
[ 17.995239] x29: ffffffc3ee3ebe80 x28: 0000000000000001
[ 17.995241] x27: ffffff8009e26000 x26: ffffff80097fea48
[ 17.995244] x25: 0000000000000000 x24: 000000042229f180
[ 17.995246] x23: ffffff8009f9a318 x22: ffffffc3ffe39a50
[ 17.995249] x21: ffffff8009f9a330 x20: ffffff800a164920
[ 17.995251] x19: 0000000000000000 x18: 0000000000000002
[ 17.995253] x17: 000000000000000e x16: 0000000000000000
[ 17.995256] x15: 00000000000003b2 x14: 0000000000000400
[ 17.995258] x13: 0000000000000825 x12: 071c71c71c71c71c
[ 17.995261] x11: 000000000000000b x10: 0101010101010101
[ 17.995264] x9 : fffffffffffffffc x8 : 7f7f7f7f7f7f7f7f
[ 17.995266] x7 : fefefeff646c606d x6 : 00170401e9e1acf4
[ 17.995269] x5 : 742c616901041700 x4 : 8080808000000000
[ 17.995273] x3 : b34b234b0963a000 x2 : 000000000000000b
[ 17.995274] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.995278] x1 : 0000000000000000 x0 : 0000000000000000
[ 17.995278]
[ 17.995282] Process swapper/6 (pid: 0, stack limit = 0xffffffc3ee3e8000)
[ 17.995283] Call trace:
[ 17.995286] [<ffffff8008b8a0ec>] t19x_cpu_enter_state+0x4c/0x118
[ 17.995290] [<ffffff8008b85d64>] cpuidle_enter_state+0x84/0x380
[ 17.995293] [<ffffff8008b860d4>] cpuidle_enter+0x34/0x48
[ 17.995295] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.995299] [<ffffff80081129b4>] call_cpuidle+0x44/0x70
[ 17.995302] [<ffffff8008112d30>] cpu_startup_entry+0x1b0/0x200
[ 17.995306] [<ffffff8008092ce8>] secondary_start_kernel+0x190/0x1f8
[ 17.995308] [<0000000080f441a4>] 0x80f441a4
[ 17.995311] ---[ end trace 47ef958400a7a83e ]---
[ 17.996615] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.996727] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.996747] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.996764] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 17.996799] CPU7: SError detected, daif=1c0, spsr=0xc000c5, mpidr=80000301, esr=be000000
[ 17.996865] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[ 17.996881] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[ 17.996898] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[ 18.940549] SMP: failed to stop secondary CPUs 1,6
[ 18.940551] Kernel Offset: disabled
[ 18.940553] Memory Limit: none
[ 20.333245] Rebooting in 5 seconds..
Any ideas what could cause this and how to fix it? Also, are there more “efficient” ways to rescue the system than flashing a new image? The latter approach makes debugging this problem quite time consuming …
Would be grateful for any pointers!
minicom.txt (106 KB)