Orin fails to boot after 5.1 update

I haven’t been able to update my AGX Orin Devkit from 5.0.1 preview (r34.1.1) to 5.1 (r35.2)
(It is working with a 5.0.1 install)
I’ve tried: SdkManager recovery mode / Manual recovery flash / Apt upgrade
All seem to fail a similar way, with a kernel panic (right after the fan turns on):

Internal error: Oops: 96000044 [#1] PREEMPT SMP
Unable to handle kernel paging request at virtual address 002f93562154c3b7
Kernel panic - not syncing: Oops: Fatal exception in interrupt

It looks like this doc is missing the “-s”? There is one on this page
I may not have typed in the partition name when prompted… But that wouldn’t happen in the other upgrades.

Did I get a strange/old version of Orin hardware? (32G ram, 12 cpu, 2048 CUDA cores)
In SDKManager, which option should I pick? “Jetson AGX Orin” or “Jetson AGX Orin 32GB” – what is it choosing?

In some cases the system would flash & boot (and do runtime or oem-config) but fail after the next reboot.
At some point I added a nvme drive – it appears that the 5.1 install sets the boot order to nvme first, even when that wasn’t the device it installed to. (I did change it back to the correct boot order during my last failed 5.1 attempt, but I think I didn’t need io change it back after a 5.0.1 install?)

==Hardware details

$ cat /etc/nv_boot_control.conf
TNSPEC 3701-500-0000-L.0-1-1-jetson-agx-orin-devkit-internal
COMPATIBLE_SPEC 3701----1--jetson-agx-orin-devkit-
TEGRA_CHIPID 0x23
TEGRA_OTA_BOOT_DEVICE /dev/mtdblock0
TEGRA_OTA_GPT_DEVICE /dev/mtdblock0
$ jetson_release -v
Software part of jetson-stats 4.1.4 - (c) 2023, Raffaello Bonghi
Model: NVIDIA Orin Jetson-Small Developer Kit - Jetpack 5.0.1 DP [L4T 34.1.1]
NV Power Mode: MAXN - Type: 0
Hardware:
 - 699-level Part Number: 699-13701-0000-500 L.0
 - P-Number: p3701-0000
 - Module: NVIDIA Jetson AGX Orin
 - SoC: tegra23x
 - CUDA Arch BIN: 8.7
 - Codename: Concord
 - Serial Number: 1423522021968
Platform:
 - Machine: aarch64
 - System: Linux
 - Distribution: Ubuntu 20.04 focal
 - Release: 5.10.65-tegra
 - Python: 3.8.10
jtop:
 - Version: 4.1.4
 - Service: Active
Libraries:
 - CUDA: 11.4.239
 - cuDNN: 8.3.2.49
 - TensorRT: 5.0.1
 - VPI: 2.0.14
 - Vulkan: 1.3.203
 - OpenCV: 4.5.4 - with CUDA: NO
Jetson UEFI firmware (version 2.1-32413640 built on 2023-01-24T23:12:27+00:00)
ESC   to enter Setup.
F11   to enter Boot Manager Menu.
Enter to continue boot.
**  WARNING: Test Key is used.  **
......
      L4TLauncher: Attempting Direct Boot
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...
▒▒I/TC: Secondary CPU 1 initializing
I/TC: Secondary CPU 1 switching to normal world boot
I/TC: Secondary CPU 2 initializing
I/TC: Secondary CPU 2 switching to normal world boot
I/TC: Secondary CPU 3 initializing
I/TC: Secondary CPU 3 switching to normal world boot
I/TC: Secondary CPU 4 initializing
I/TC: Secondary CPU 4 switching to normal world boot
I/TC: Secondary CPU 5 initializing
I/TC: Secondary CPU 5 switching to normal world boot
I/TC: Secondary CPU 6 initializing
I/TC: Secondary CPU 6 switching to normal world boot
I/TC: Secondary CPU 7 initializing
I/TC: Secondary CPU 7 switching to normal world boot
I/TC: Secondary CPU 8 initializing
I/TC: Secondary CPU 8 switching to normal world boot
I/TC: Secondary CPU 9 initializing
I/TC: Secondary CPU 9 switching to normal world boot
I/TC: Secondary CPU 10 initializing
I/TC: Secondary CPU 10 switching to normal world boot
I/TC: Secondary CPU 11 initializing
I/TC: Secondary CPU 11 switching to normal world boot
▒▒[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd421]
[    0.000000] Linux version 5.10.104-tegra (buildbrain@mobile-u64-6348-d7000) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2020.08) 9.3.0, GNU ld (GNU Binutils) 2.33.1) #1 SMP PREEMPT T                                            ue Jan 24 15:09:44 PST 2023
[    0.000000] OF: fdt: memory scan node memory@80000000, reg size 16,
[    0.000000] OF: fdt:  - 80000000 ,  c0000000
[    0.000000] Machine model: Jetson AGX Orin
[    0.000000] efi: EFI v2.70 by EDK II
[    0.000000] efi: RTPROP=0x808260518 SMBIOS=0xffff0000 SMBIOS 3.0=0x806be0000 MEMATTR=0x807611018 ESRT=0x806d2e298 RNG=0x82a9fdd18 MEMRESERVE=0x807612f18
[    0.000000] efi: seeding entropy pool
[    0.000000] esrt: Reserving ESRT space from 0x0000000806d2e298 to 0x0000000806d2e2d0.
[    0.000000] Reserved memory: created CMA memory pool at 0x000000081a800000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000000837ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x0000000803ffffff]
[    0.000000]   node   0: [mem 0x0000000804000000-0x00000008061bffff]
[    0.000000]   node   0: [mem 0x00000008061c0000-0x000000080653dfff]
[    0.000000]   node   0: [mem 0x000000080653e000-0x00000008066d1fff]
[    0.000000]   node   0: [mem 0x00000008066d2000-0x0000000806bcffff]
[    0.000000]   node   0: [mem 0x0000000806bd0000-0x0000000806beffff]
[    0.000000]   node   0: [mem 0x0000000806bf0000-0x0000000806dfffff]
[    0.000000]   node   0: [mem 0x0000000806e00000-0x0000000806e4ffff]
[    0.000000]   node   0: [mem 0x0000000806e50000-0x000000080751ffff]
[    0.000000]   node   0: [mem 0x0000000807520000-0x000000080756ffff]
[    0.000000]   node   0: [mem 0x0000000807570000-0x00000008075dffff]
[    0.000000]   node   0: [mem 0x00000008075e0000-0x00000008075fffff]
[    0.000000]   node   0: [mem 0x0000000807600000-0x000000080761ffff]
[    0.000000]   node   0: [mem 0x0000000807620000-0x000000080763ffff]
[    0.000000]   node   0: [mem 0x0000000807640000-0x000000080775ffff]
[    0.000000]   node   0: [mem 0x0000000807760000-0x000000080793ffff]
[    0.000000]   node   0: [mem 0x0000000807940000-0x000000080798ffff]
[    0.000000]   node   0: [mem 0x0000000807990000-0x0000000807acffff]
[    0.000000]   node   0: [mem 0x0000000807ad0000-0x0000000807b9ffff]
[    0.000000]   node   0: [mem 0x0000000807ba0000-0x0000000807d7ffff]
[    0.000000]   node   0: [mem 0x0000000807d80000-0x0000000807d9ffff]
[    0.000000]   node   0: [mem 0x0000000807da0000-0x0000000807daffff]
[    0.000000]   node   0: [mem 0x0000000807db0000-0x000000080838ffff]
[    0.000000]   node   0: [mem 0x0000000808390000-0x000000080842ffff]
[    0.000000]   node   0: [mem 0x0000000808430000-0x000000080852ffff]
[    0.000000]   node   0: [mem 0x0000000808530000-0x00000008085cffff]
[    0.000000]   node   0: [mem 0x00000008085d0000-0x00000008086cffff]
[    0.000000]   node   0: [mem 0x00000008086d0000-0x000000080876ffff]
[    0.000000]   node   0: [mem 0x0000000808770000-0x000000080903ffff]
[    0.000000]   node   0: [mem 0x0000000809040000-0x000000080921ffff]
[    0.000000]   node   0: [mem 0x0000000809220000-0x000000082a9affff]
[    0.000000]   node   0: [mem 0x000000082a9b0000-0x000000082a9fffff]
[    0.000000]   node   0: [mem 0x000000082aa00000-0x000000082e7fffff]
[    0.000000]   node   0: [mem 0x000000082ea00000-0x000000082eefffff]
[    0.000000]   node   0: [mem 0x0000000836000000-0x0000000837ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000837ffffff]
[    0.000000] On node 0, zone Normal: 256 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 32 pages/cpu s90200 r8192 d32680 u131072
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] alternatives: patching kernel code
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 7937792
[    0.000000] Kernel command line: root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 console=tty0 firmware_class.path=/etc/firmwar                                            e fbcon=map:0 net.ifnames=0 nv-auto-config
[    0.000000] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear)
[    0.000000] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000effe0000-0x00000000fffe0000] (256MB)
[    0.000000] Memory: 31048504K/32257024K available (18432K kernel code, 3168K rwdata, 7164K rodata, 3968K init, 1110K bss, 946376K reserved, 262144K cma-reserved)
[    0.000000] random: get_random_u64 called from __kmem_cache_create+0x38/0x420 with crng_init=0
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=12, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=12.
[    0.000000]  Trampoline variant of Tasks RCU enabled.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=12
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 960 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x000000000f440000
[    0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:608, num:70)
[    0.000000] GICv2m: range[mem 0x0f410000-0x0f41ffff], SPI[608:677]
[    0.000000] arch_timer: cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000003] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.000810] Console: colour dummy device 80x25
[    0.000996] printk: console [tty0] enabled
[    0.001041] Calibrating delay loop (skipped), value calculated using timer frequency.. 62.50 BogoMIPS (lpj=125000)
[    0.001051] pid_max: default: 32768 minimum: 301
[    0.001115] LSM: Security Framework initializing
[    0.001139] Yama: becoming mindful.
[    0.001158] SELinux:  Initializing.
[    0.001252] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.001302] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.003401] rcu: Hierarchical SRCU implementation.
[    0.003504] dyndbg: Ignore empty _ddebug table in a CONFIG_DYNAMIC_DEBUG_CORE build
[    0.004240] DTS File Name: /dvs/git/dirty/git-master_linux/kernel/kernel-5.10/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t23x/concord/kernel-dts/tegra234-p3701-0000-p3737                                            -0000.dts
[    0.004251] DTB Build time: Jan 24 2023 15:26:09
[    0.005678] Tegra Revision: A01 SKU: 0xd0 CPU Process: 0 SoC Process: 0
[    0.005802] Remapping and enabling EFI services.
[    0.007010] smp: Bringing up secondary CPUs ...
[    0.035713] Detected PIPT I-cache on CPU1
[    0.035753] GICv3: CPU1: found redistributor 100 region 0:0x000000000f460000
[    0.035791] CPU1: Booted secondary processor 0x0000000100 [0x410fd421]
[    0.064436] Detected PIPT I-cache on CPU2
[    0.064446] GICv3: CPU2: found redistributor 200 region 0:0x000000000f480000
[    0.064461] CPU2: Booted secondary processor 0x0000000200 [0x410fd421]
[    0.093048] Detected PIPT I-cache on CPU3
[    0.093057] GICv3: CPU3: found redistributor 300 region 0:0x000000000f4a0000
[    0.093070] CPU3: Booted secondary processor 0x0000000300 [0x410fd421]
[    0.123729] Detected PIPT I-cache on CPU4
[    0.123750] GICv3: CPU4: found redistributor 10000 region 0:0x000000000f4c0000
[    0.123777] CPU4: Booted secondary processor 0x0000010000 [0x410fd421]
[    0.152435] Detected PIPT I-cache on CPU5
[    0.152444] GICv3: CPU5: found redistributor 10100 region 0:0x000000000f4e0000
[    0.152459] CPU5: Booted secondary processor 0x0000010100 [0x410fd421]
[    0.181078] Detected PIPT I-cache on CPU6
[    0.181088] GICv3: CPU6: found redistributor 10200 region 0:0x000000000f500000
[    0.181103] CPU6: Booted secondary processor 0x0000010200 [0x410fd421]
[    0.209720] Detected PIPT I-cache on CPU7
[    0.209730] GICv3: CPU7: found redistributor 10300 region 0:0x000000000f520000
[    0.209746] CPU7: Booted secondary processor 0x0000010300 [0x410fd421]
[    0.240407] Detected PIPT I-cache on CPU8
[    0.240431] GICv3: CPU8: found redistributor 20000 region 0:0x000000000f540000
[    0.240460] CPU8: Booted secondary processor 0x0000020000 [0x410fd421]
[    0.269107] Detected PIPT I-cache on CPU9
[    0.269118] GICv3: CPU9: found redistributor 20100 region 0:0x000000000f560000
[    0.269132] CPU9: Booted secondary processor 0x0000020100 [0x410fd421]
[    0.298362] Detected PIPT I-cache on CPU10
[    0.298372] GICv3: CPU10: found redistributor 20200 region 0:0x000000000f580000
[    0.298387] CPU10: Booted secondary processor 0x0000020200 [0x410fd421]
[    0.327615] Detected PIPT I-cache on CPU11
[    0.327625] GICv3: CPU11: found redistributor 20300 region 0:0x000000000f5a0000
[    0.327640] CPU11: Booted secondary processor 0x0000020300 [0x410fd421]
[    0.327695] smp: Brought up 1 node, 12 CPUs
[    0.327776] SMP: Total of 12 processors activated.
[    0.327782] CPU features: detected: Privileged Access Never
[    0.327784] CPU features: detected: LSE atomic instructions
[    0.327787] CPU features: detected: User Access Override
[    0.327790] CPU features: detected: 32-bit EL0 Support
[    0.327795] CPU features: detected: Common not Private translations
[    0.327798] CPU features: detected: RAS Extension Support
[    0.327801] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.327804] CPU features: detected: CRC32 instructions
[    0.327807] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
[    0.363811] CPU: All CPU(s) started at EL2
[    0.366295] devtmpfs: initialized
[    0.383984] Registered cp15_barrier emulation handler
[    0.383997] Registered setend emulation handler
[    0.384003] KASLR enabled
[    0.384118] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.384134] futex hash table entries: 4096 (order: 6, 262144 bytes, linear)
[    0.385770] pinctrl core: initialized pinctrl subsystem
[    0.386329] SMBIOS 3.0.0 present.
[    0.386338] DMI: Unknown Jetson AGX Orin/Jetson AGX Orin, BIOS 2.1-32413640 01/24/2023
[    0.386622] NET: Registered protocol family 16
[    0.387462] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations
[    0.387667] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.387865] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.387892] audit: initializing netlink subsys (disabled)
[    0.387981] audit: type=2000 audit(0.384:1): state=initialized audit_enabled=0 res=1
[    0.388143] thermal_sys: Registered thermal governor 'step_wise'
[    0.388145] thermal_sys: Registered thermal governor 'power_allocator'
[    0.388150] thermal_sys: Registered thermal governor 'pid_thermal_gov'
[    0.390649] cpuidle: using governor menu
[    0.390759] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.390879] ASID allocator initialised with 32768 entries
[    0.390974] Serial: AMBA PL011 UART driver
[    0.392853] tegra234_mc_sid_probe(): tegra234 mc-sid probe successful
[    0.392878] Tegra reboot handler registered.
[    0.411430] mc: mapped MMIO address: 0xffff8000111f0000 -> 0x2c10000
[    0.411444] mc: mapped MMIO address: 0xffff800011210000 -> 0x2c20000
[    0.411451] mc: mapped MMIO address: 0xffff800011230000 -> 0x2c30000
[    0.411459] mc: mapped MMIO address: 0xffff800011250000 -> 0x2c40000
[    0.411468] mc: mapped MMIO address: 0xffff800011270000 -> 0x2c50000
[    0.411475] mc: mapped MMIO address: 0xffff800011290000 -> 0x2b80000
[    0.411482] mc: mapped MMIO address: 0xffff8000112b0000 -> 0x2b90000
[    0.411488] mc: mapped MMIO address: 0xffff8000112d0000 -> 0x2ba0000
[    0.411495] mc: mapped MMIO address: 0xffff8000112f0000 -> 0x2bb0000
[    0.411502] mc: mapped MMIO address: 0xffff800011310000 -> 0x1700000
[    0.411508] mc: mapped MMIO address: 0xffff800011330000 -> 0x1710000
[    0.411514] mc: mapped MMIO address: 0xffff800011350000 -> 0x1720000
[    0.411520] mc: mapped MMIO address: 0xffff800011370000 -> 0x1730000
[    0.411527] mc: mapped MMIO address: 0xffff800011390000 -> 0x1740000
[    0.411534] mc: mapped MMIO address: 0xffff8000113b0000 -> 0x1750000
[    0.411540] mc: mapped MMIO address: 0xffff8000113d0000 -> 0x1760000
[    0.411547] mc: mapped MMIO address: 0xffff8000113f0000 -> 0x1770000
[    0.411558] nv-tegra-mc 2c10000.mc: No mssnvlink node
[    0.411584] mc-err: mcerr ops are set to t23x
[    0.412236] 31d0000.serial: ttyAMA0 at MMIO 0x31d0000 (irq = 67, base_baud = 0) is a SBSA
[    1.670698] printk: console [ttyAMA0] enabled
[    1.679116] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    1.685822] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    1.692522] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    1.699221] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    1.707690] ACPI: Interpreter disabled.
[    1.712701] vdd-av1v1-hub: supplied by vdd-5v-sys
[    1.717589] vbusA-cvb: supplied by vdd-5v-sys
[    1.722066] vbusB-cvb: supplied by vdd-5v-sys
[    1.726547] vdd-usbc1-vbus: supplied by vdd-5v-sys
[    1.731457] vdd-usbc2-vbus: supplied by vdd-5v-sys
[    1.736526] iommu: Default domain type: Translated
[    1.741446] eventlib_kernel: keventlib is initialized, test id: 0
[    1.747714] SCSI subsystem initialized
[    1.751526] usbcore: registered new interface driver usbfs
[    1.757027] usbcore: registered new interface driver hub
[    1.762353] usbcore: registered new device driver usb
[    1.767578] mc: Linux media interface: v0.10
[    1.771856] videodev: Linux video capture interface: v2.00
[    1.777483] pps_core: LinuxPPS API ver. 1 registered
[    1.782447] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    1.791583] PTP clock support registered
[    1.795702] tegra_wdt_t18x 2190000.watchdog: Tegra WDT init timeout = 120 sec
[    1.802866] tegra_wdt_t18x 2190000.watchdog: Registered successfully
[    1.810554] Registered efivars operations
[    1.814693] FPGA manager framework
[    1.818131] Advanced Linux Sound Architecture Driver Initialized.
[    1.824475] Bluetooth: Core ver 2.22
[    1.828057] NET: Registered protocol family 31
[    1.832500] Bluetooth: HCI device and connection manager initialized
[    1.838857] Bluetooth: HCI socket layer initialized
[    1.843733] Bluetooth: L2CAP socket layer initialized
[    1.848792] Bluetooth: SCO socket layer initialized
[    1.854161] camchar: rtcpu character device driver loaded
[    1.859782] clocksource: Switched to clocksource arch_sys_counter
[    2.019302] VFS: Disk quotas dquot_6.6.0
[    2.023271] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.030303] pnp: PnP ACPI: disabled
[    2.035624] la/ptsa driver initialized.
[    2.039516] NET: Registered protocol family 2
[    2.044089] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    2.053835] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, linear)
[    2.062578] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    2.070860] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    2.078753] TCP: Hash tables configured (established 262144 bind 65536)
[    2.085463] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    2.092391] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    2.099848] NET: Registered protocol family 1
[    2.104480] RPC: Registered named UNIX socket transport module.
[    2.110418] RPC: Registered udp transport module.
[    2.115121] RPC: Registered tcp transport module.
[    2.119821] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    2.126271] PCI: CLS 0 bytes, default 64
[    2.130316] Trying to unpack rootfs image as initramfs...
[    2.245996] Freeing initrd memory: 8208K
[    2.252310] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[    2.260225] kvm [1]: IPA Size Limit: 48 bits
[    2.264695] kvm [1]: VHE mode initialized successfully
[    2.270485] Initialise system trusted keyrings
[    2.275015] workingset: timestamp_bits=46 max_order=23 bucket_order=0
[    2.283570] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.289685] NFS: Registering the id_resolver key type
[    2.294750] Key type id_resolver registered
[    2.298931] Key type id_legacy registered
[    2.302976] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    2.309685] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    2.317090] ntfs: driver 2.1.32 [Flags: R/W].
[    2.333188] NET: Registered protocol family 38
[    2.337634] Key type asymmetric registered
[    2.341729] Asymmetric key parser 'x509' registered
[    2.346622] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 240)
[    2.354079] io scheduler mq-deadline registered
[    2.358610] io scheduler kyber registered
[    2.366521] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[    2.372123] tegra-pwm 32a0000.pwm: PWM clk can sleep in ops
[    2.377715] tegra-pwm 32c0000.pwm: PWM clk can sleep in ops
[    2.383300] tegra-pwm 32f0000.pwm: PWM clk can sleep in ops
[    2.391380] tegra_dc_assign_hw_data: no matching compatible node
[    2.397393] tegradccommon module_init failed
[    2.401660] tegradc module_init failed
[    2.405659] EINJ: ACPI disabled.
[    2.410413] tegra-pmc c360000.pmc: scratch reg offset dts data not present
[    2.417303] tegra-pmc: ### PMC reset source: SYS_RESET_N
[    2.422615] tegra-pmc: ### PMC reset level: L0
[    2.427056] tegra-pmc: ### PMC reset status reg: 0x0
[    2.433396] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.446330] random: fast init done
[    2.446394] printk: console [ttyTCU0] enabled
[    4.091784] arm-smmu 12000000.iommu: probing hardware configuration...
[    4.099744] arm-smmu 12000000.iommu: SMMUv2 with:
[    4.104599] arm-smmu 12000000.iommu:         stage 1 translation
[    4.110152] arm-smmu 12000000.iommu:         stage 2 translation
[    4.115696] arm-smmu 12000000.iommu:         nested translation
[    4.121151] arm-smmu 12000000.iommu:         stream matching with 128 register groups
[    4.128576] arm-smmu 12000000.iommu:         128 context banks (0 stage-2 only)
[    4.137574] arm-smmu 12000000.iommu:         Supported page sizes: 0x00001000
[    4.144276] arm-smmu 12000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    4.150975] arm-smmu 12000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    4.158553] arm-smmu 10000000.iommu: probing hardware configuration...
[    4.165253] arm-smmu 10000000.iommu: SMMUv2 with:
[    4.170073] arm-smmu 10000000.iommu:         stage 1 translation
[    4.175612] arm-smmu 10000000.iommu:         stage 2 translation
[    4.181159] arm-smmu 10000000.iommu:         nested translation
[    4.186609] arm-smmu 10000000.iommu:         stream matching with 128 register groups
[    4.194021] arm-smmu 10000000.iommu:         128 context banks (0 stage-2 only)
[    4.202925] arm-smmu 10000000.iommu:         Supported page sizes: 0x00001000
[    4.209623] arm-smmu 10000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    4.216317] arm-smmu 10000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    4.223442] arm-smmu 8000000.iommu: probing hardware configuration...
[    4.230055] arm-smmu 8000000.iommu: SMMUv2 with:
[    4.234797] arm-smmu 8000000.iommu:  stage 1 translation
[    4.240245] arm-smmu 8000000.iommu:  stage 2 translation
[    4.245690] arm-smmu 8000000.iommu:  nested translation
[    4.251052] arm-smmu 8000000.iommu:  stream matching with 128 register groups
[    4.258382] arm-smmu 8000000.iommu:  128 context banks (0 stage-2 only)
[    4.267222] arm-smmu 8000000.iommu:  Supported page sizes: 0x00001000
[    4.273835] arm-smmu 8000000.iommu:  Stage-1: 48-bit VA -> 48-bit IPA
[    4.280449] arm-smmu 8000000.iommu:  Stage-2: 48-bit IPA -> 48-bit PA
[    4.287831] arm-smmu-suspend 12000000.smmu_suspend: arm_smmu_suspend probe successful
[    4.298194] tegra_profiler: version: 1.150, samples/io: 50/29
[    4.304180] tegra_profiler: auth: init
[    4.308324] nvsciipc: loaded module
[    4.313417] tun: Universal TUN/TAP device driver, 1.6
[    4.319052] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    4.326488] hns3: Copyright (c) 2017 Huawei Corporation.
[    4.331987] e1000e: Intel(R) PRO/1000 Network Driver
[    4.337092] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    4.343173] igb: Intel(R) Gigabit Ethernet Network Driver
[    4.348719] igb: Copyright (c) 2007-2014 Intel Corporation.
[    4.354589] PPP generic driver version 2.4.2
[    4.359021] PPP BSD Compression module registered
[    4.363853] PPP Deflate Compression module registered
[    4.369056] PPP MPPE Compression module registered
[    4.374005] usbcore: registered new interface driver r8152
[    4.379641] usbcore: registered new interface driver asix
[    4.385202] usbcore: registered new interface driver ax88179_178a
[    4.391463] usbcore: registered new interface driver cdc_ether
[    4.397462] usbcore: registered new interface driver net1080
[    4.403271] usbcore: registered new interface driver cdc_subset
[    4.409361] usbcore: registered new interface driver zaurus
[    4.415090] usbcore: registered new interface driver cdc_ncm
[    4.420906] usbcore: registered new interface driver aqc111
[    4.426731] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    4.433094] ohci-pci: OHCI PCI platform driver
[    4.437660] ohci-platform: OHCI generic platform driver
[    4.443311] tegra-xusb 3610000.xhci: Adding to iommu group 0
[    4.449732] usbcore: registered new interface driver uas
[    4.455214] usbcore: registered new interface driver usb-storage
[    4.461517] tegra-xudc 3550000.xudc: Adding to iommu group 1
[    4.467628] mousedev: PS/2 mouse device common for all mice
[    4.473375] usbcore: registered new interface driver xpad
[    4.480796] tegra_rtc c2a0000.rtc: registered as rtc1
[    4.486273] tegra_rtc c2a0000.rtc: setting system clock to 1970-01-01T00:00:31 UTC (31)
[    4.494502] tegra_rtc c2a0000.rtc: Tegra internal Real Time Clock
[    4.500910] i2c /dev entries driver
[    4.504602] tegra-i2c 3160000.i2c: Adding to iommu group 2
[    4.510427] tegra-i2c c240000.i2c: Adding to iommu group 2
[    4.516126] tegra-i2c 3180000.i2c: Adding to iommu group 2
[    4.521795] tegra-i2c 3190000.i2c: Adding to iommu group 2
[    4.527466] tegra-i2c 31b0000.i2c: Adding to iommu group 2
[    4.533138] tegra-i2c 31c0000.i2c: Adding to iommu group 2
[    4.538819] tegra-i2c c250000.i2c: Adding to iommu group 2
[    4.544497] tegra-i2c 31e0000.i2c: Adding to iommu group 2
[    4.552484] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoint.
[    4.561075] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoint.
[    4.570543] device-mapper: uevent: version 1.0.3
[    4.575376] device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@redhat.com
[    4.585179] sdhci: Secure Digital Host Controller Interface driver
[    4.591522] sdhci: Copyright(c) Pierre Ossman
[    4.595987] Synopsys Designware Multimedia Card Interface Driver
[    4.602221] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.608428] sdhci-tegra 3460000.sdhci: Adding to iommu group 3
[    4.608533] SMCCC: SOC_ID: ID = jep106:036b:0023 Revision = 0x00000401
[    4.608570] sdhci-tegra 3400000.sdhci: Adding to iommu group 4
[    4.628775] tegra-bpmp bpmp: firmware: 9a7bb16aee9be45a56e6-ccd7dae126b
[    4.636939] nvvrs_pseq 4-003c: NVVRS Vendor ID: 0x9
[    4.642505] nvvrs_pseq 4-003c: NVVRS Model Rev: 0x81
[    4.652098] nvvrs-pseq-rtc nvvrs-pseq-rtc: registered as rtc0
[    4.658003] nvvrs_pseq 4-003c: NVVRS PSEQ probe successful
▒▒debugfs initialized
▒▒[    6.906680] hid: raw HID events driver (C) Jiri Kosina
[    6.912511] usbcore: registered new interface driver usbhid
[    6.918252] usbhid: USB HID core driver
[    6.923143] tegra-dce d800000.dce: Adding to iommu group 5
[    6.930084] dce: dce_ipc_channel_init:314  Invalid Channel State [0x0] for ch_type [2]
[    6.939221] tegra234-aon c000000.aon: Adding to iommu group 6
[    6.945615] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB : [0x5]
[    6.954575] dce: dce_admin_ipc_handle_signal:90   Spurious signal on channel: [0]. Ignored...
[    6.955539]  c000000.aon:hsp: probed
[    6.963382] dce: dce_admin_send_cmd_ver:424  version : [0x2] err : [0x0]
[    6.963522] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB : [0x1]
[    6.967293] tegra234-aon c000000.aon: init done
[    6.974004] dce: dce_admin_setup_clients_ipc:553  Channel Reset Complete for Type [1] ...
[    6.974005] dce: dce_admin_ipc_handle_signal:90   Spurious signal on channel: [1]. Ignored...
[    6.974015] dce: dce_admin_ipc_handle_signal:90   Spurious signal on channel: [1]. Ignored...
[    6.983394] tegra23x-psc e860000.psc: Adding to iommu group 7
[    6.987654] dce: dce_admin_setup_clients_ipc:529  Get queue info failed for [2]
[    6.988234] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB : [0x2]
[    6.996443] tegra23x-psc e860000.psc: ext_cfg base:(____ptrval____)
[    7.004855] dce: dce_admin_setup_clients_ipc:553  Channel Reset Complete for Type [3] ...
[    7.004857] dce: dce_admin_ipc_handle_signal:90   Spurious signal on channel: [3]. Ignored...
[    7.004867] dce: dce_admin_ipc_handle_signal:90   Spurious signal on channel: [3]. Ignored...
[    7.013631] tegra23x-psc e860000.psc: init done
[    7.013778] tegra186-cam-rtcpu bc00000.rtcpu: Adding to iommu group 8
[    7.019748] dce: dce_start_boot_flow:173  DCE_BOOT_DONE
[    7.027803] tegra186-cam-rtcpu bc00000.rtcpu: Trace buffer configured at IOVA=0xbff00000
[    7.099345] tegra-ivc-bus bc00000.rtcpu:ivc-bus: region 0: iova=0xbfec0000-0xbfee01ff size=131584
[    7.108536] tegra-ivc-bus bc00000.rtcpu:ivc-bus:echo@0: echo: ver=0 grp=1 RX[16x64]=0x1000-0x1480 TX[16x64]=0x1480-0x1900
[    7.120132] tegra-ivc-bus bc00000.rtcpu:ivc-bus:dbg@1: dbg: ver=0 grp=1 RX[1x448]=0x1900-0x1b40 TX[1x448]=0x1b40-0x1d80
[    7.131527] tegra-ivc-bus bc00000.rtcpu:ivc-bus:dbg@2: dbg: ver=0 grp=1 RX[1x8192]=0x1d80-0x3e00 TX[1x8192]=0x3e00-0x5e80
[    7.143300] tegra-ivc-bus bc00000.rtcpu:ivc-bus:ivccontrol@3: ivccontrol: ver=0 grp=1 RX[64x320]=0x5e80-0xaf00 TX[64x320]=0xaf00-0xff80
[    7.155886] tegra-ivc-bus bc00000.rtcpu:ivc-bus:ivccapture@4: ivccapture: ver=0 grp=1 RX[512x64]=0xff80-0x18000 TX[512x64]=0x18000-0x20080
[    7.168737] tegra-ivc-bus bc00000.rtcpu:ivc-bus:diag@5: diag: ver=0 grp=1 RX[1x64]=0x20080-0x20140 TX[1x64]=0x20140-0x20200
[    7.180748] tegra186-cam-rtcpu bc00000.rtcpu: using cam RTCPU IRQ (117)
[    7.187570] tegra186-cam-rtcpu bc00000.rtcpu: tegra_camrtc_mon_create is successful
[    7.195926] tegra186-cam-rtcpu bc00000.rtcpu: firmware version cpu=rce cmd=6 sha1=10150ec03f8849d1537396903b70d4a48a255aba
[    7.209389] t19x_cache tegra-cache: probed
[    7.213806] scf-pmu-drv scf-pmu: Registered T23x SCF Uncore PMU
[    7.227714] optee: probing for conduit method.
[    7.232310] optee: revision 3.19 (18d0f61b)
[    7.233005] optee: dynamic shared memory is enabled
[    7.242436] optee: initialized driver
[    7.259265] nvpmodel: initialized successfully
[    7.263951] tegra_hv: get_hvd: not initialized yet
[    7.268886] user_ivc_mempool: hypervisor not present
[    7.274725] usbcore: registered new interface driver snd-usb-audio
[    7.281920] u32 classifier
[    7.284724]     input device check on
[    7.288502]     Actions configured
[    7.292131] Initializing XFRM netlink socket
[    7.296978] NET: Registered protocol family 10
[    7.302760] Segment Routing with IPv6
[    7.306593] NET: Registered protocol family 17
[    7.311185] NET: Registered protocol family 15
[    7.315801] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    7.330199] Bluetooth: RFCOMM socket layer initialized
[    7.335587] Bluetooth: RFCOMM ver 1.11
[    7.339476] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[    7.345580] Bluetooth: HIDP socket layer initialized
[    7.351359] 9pnet: Installing 9P2000 support
[    7.355833] Key type dns_resolver registered
[    7.361526] registered taskstats version 1
[    7.365764] Loading compiled-in X.509 certificates
[    7.376745] Loaded X.509 cert 'Build time autogenerated kernel key: d2e6b29ce3ba3f3d31103e414d71889f8474be7b'
[    7.387327] tegra194-pcie 14100000.pcie: Adding to iommu group 9
[    7.400165] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[    7.407308] tegra194-pcie 14160000.pcie: Adding to iommu group 10
[    7.420073] tegra194-pcie 14160000.pcie: Using GICv2m MSI allocator
[    7.426924] tegra194-pcie 141a0000.pcie: Adding to iommu group 11
[    7.439119] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[    7.471617] tegra-soc-hwpm f100000.tegra_soc_hwpm: Adding to iommu group 12
[    7.481890] host1x 13e40000.host1x: Adding to iommu group 13
[    8.499805] t23x_init_gating_regs: Timed out waiting for syncpt ram init!
[    8.519236] host1x 13e40000.host1x: initialized
[    8.524296] iommu_context_dev 13e40000.host1x:niso0_ctx0: Adding to iommu group 14
[    8.532440] iommu_context_dev 13e40000.host1x:niso0_ctx0: initialized (streamid=53, iommu=smmu.0x0000000012000000)
[    8.544539] iommu_context_dev 13e40000.host1x:niso0_ctx1: Adding to iommu group 15
[    8.552677] iommu_context_dev 13e40000.host1x:niso0_ctx1: initialized (streamid=54, iommu=smmu.0x0000000012000000)
[    8.564709] iommu_context_dev 13e40000.host1x:niso0_ctx2: Adding to iommu group 16
[    8.572813] iommu_context_dev 13e40000.host1x:niso0_ctx2: initialized (streamid=55, iommu=smmu.0x0000000012000000)
[    8.584792] iommu_context_dev 13e40000.host1x:niso0_ctx3: Adding to iommu group 17
[    8.592896] iommu_context_dev 13e40000.host1x:niso0_ctx3: initialized (streamid=56, iommu=smmu.0x0000000012000000)
[    8.604852] iommu_context_dev 13e40000.host1x:niso0_ctx4: Adding to iommu group 18
[    8.612951] iommu_context_dev 13e40000.host1x:niso0_ctx4: initialized (streamid=57, iommu=smmu.0x0000000012000000)
[    8.624930] iommu_context_dev 13e40000.host1x:niso0_ctx5: Adding to iommu group 19
[    8.633068] iommu_context_dev 13e40000.host1x:niso0_ctx5: initialized (streamid=58, iommu=smmu.0x0000000012000000)
[    8.645084] iommu_context_dev 13e40000.host1x:niso0_ctx6: Adding to iommu group 20
[    8.653185] iommu_context_dev 13e40000.host1x:niso0_ctx6: initialized (streamid=59, iommu=smmu.0x0000000012000000)
[    8.665162] iommu_context_dev 13e40000.host1x:niso0_ctx7: Adding to iommu group 21
[    8.673260] iommu_context_dev 13e40000.host1x:niso0_ctx7: initialized (streamid=60, iommu=smmu.0x0000000012000000)
[    8.685257] iommu_context_dev 13e40000.host1x:niso1_ctx0: Adding to iommu group 22
[    8.693341] iommu_context_dev 13e40000.host1x:niso1_ctx0: initialized (streamid=53, iommu=smmu.0x0000000008000000)
[    8.705331] iommu_context_dev 13e40000.host1x:niso1_ctx1: Adding to iommu group 23
[    8.713427] iommu_context_dev 13e40000.host1x:niso1_ctx1: initialized (streamid=54, iommu=smmu.0x0000000008000000)
[    8.725397] iommu_context_dev 13e40000.host1x:niso1_ctx2: Adding to iommu group 24
[    8.733489] iommu_context_dev 13e40000.host1x:niso1_ctx2: initialized (streamid=55, iommu=smmu.0x0000000008000000)
[    8.745441] iommu_context_dev 13e40000.host1x:niso1_ctx3: Adding to iommu group 25
[    8.753541] iommu_context_dev 13e40000.host1x:niso1_ctx3: initialized (streamid=56, iommu=smmu.0x0000000008000000)
[    8.765519] iommu_context_dev 13e40000.host1x:niso1_ctx4: Adding to iommu group 26
[    8.773627] iommu_context_dev 13e40000.host1x:niso1_ctx4: initialized (streamid=57, iommu=smmu.0x0000000008000000)
[    8.785613] iommu_context_dev 13e40000.host1x:niso1_ctx5: Adding to iommu group 27
[    8.793693] iommu_context_dev 13e40000.host1x:niso1_ctx5: initialized (streamid=58, iommu=smmu.0x0000000008000000)
[    8.805704] iommu_context_dev 13e40000.host1x:niso1_ctx6: Adding to iommu group 28
[    8.813782] iommu_context_dev 13e40000.host1x:niso1_ctx6: initialized (streamid=59, iommu=smmu.0x0000000008000000)
[    8.825757] iommu_context_dev 13e40000.host1x:niso1_ctx7: Adding to iommu group 29
[    8.833866] iommu_context_dev 13e40000.host1x:niso1_ctx7: initialized (streamid=60, iommu=smmu.0x0000000008000000)
[    8.845881] falcon 15340000.vic: Adding to iommu group 30
[    8.860033] falcon 15340000.vic: initialized
[    8.864721] falcon 15380000.nvjpg: Adding to iommu group 31
[    8.876275] falcon 15380000.nvjpg: initialized
[    8.880978] falcon 15540000.nvjpg1: Adding to iommu group 32
[    8.891869] falcon 15540000.nvjpg1: initialized
[    8.896662] falcon 154c0000.nvenc: Adding to iommu group 33
[    8.910302] falcon 154c0000.nvenc: initialized
[    8.915000] falcon 15a50000.ofa: Adding to iommu group 34
[    8.926502] falcon 15a50000.ofa: initialized
[    8.931026] nvdec 15480000.nvdec: Adding to iommu group 35
[    8.948762] nvdec 15480000.nvdec: initialized
[    8.953365] tsec 15500000.tsec: Adding to iommu group 36
[    8.968293] tsec 15500000.tsec: initialized
[    8.977171] gic 2a41000.agic-controller: GIC IRQ controller registered
[    8.993904] tegra-aconnect aconnect@2a41000: Tegra ACONNECT bus registered
[    9.011038] gpio-394 (camera-control-output-low): hogged as output/low
[    9.017791] gpio-397 (camera-control-output-low): hogged as output/low
[    9.024563] gpio-487 (camera-control-output-low): hogged as output/low
[    9.031301] gpio-486 (camera-control-output-low): hogged as output/low
[    9.039010] gpiochip0: registered GPIOs 348 to 511 on tegra234-gpio
[    9.046180] gpiochip1: registered GPIOs 316 to 347 on tegra234-gpio-aon
[    9.053164] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[    9.059658] tegra-pwm 32a0000.pwm: PWM clk can sleep in ops
[    9.066279] tegra-pwm 32c0000.pwm: PWM clk can sleep in ops
[    9.072556] tegra-pwm 32f0000.pwm: PWM clk can sleep in ops
[    9.080784] misc tegra_camera_ctrl: tegra_camera_isomgr_register tpg_max_iso = 3916800KBs
[    9.089353] tegra194-isp5 14800000.isp: Adding to iommu group 37
[    9.099106] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: context isolation disabled due to no IOMMU
[    9.108686] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: initialized
[    9.115273] tegra194-vi5 13e40000.host1x:vi0@15c00000: Adding to iommu group 38
[    9.123647] tegra194-vi5 13e40000.host1x:vi1@14c00000: Adding to iommu group 39
[    9.131796] scare-pigeon 13e40000.host1x:vi0-thi@15f00000: context isolation disabled due to no IOMMU
[    9.141735] scare-pigeon 13e40000.host1x:vi0-thi@15f00000: initialized
[    9.148539] scare-pigeon 13e40000.host1x:vi1-thi@14f00000: context isolation disabled due to no IOMMU
[    9.158453] scare-pigeon 13e40000.host1x:vi1-thi@14f00000: initialized
[    9.165256] scare-pigeon 13e40000.host1x:isp-thi@14b00000: context isolation disabled due to no IOMMU
[    9.175141] scare-pigeon 13e40000.host1x:isp-thi@14b00000: initialized
[    9.181955] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx0: Adding to iommu group 40
[    9.182251] random: crng init done
[    9.190703] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx0: initialized (streamid=18, iommu=smmu.0x0000000008000000)
[    9.205288] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx1: Adding to iommu group 41
[    9.214066] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx1: initialized (streamid=19, iommu=smmu.0x0000000008000000)
[    9.225440] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx2: Adding to iommu group 42
[    9.234166] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx2: initialized (streamid=20, iommu=smmu.0x0000000008000000)
[    9.245538] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx3: Adding to iommu group 43
[    9.254251] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx3: initialized (streamid=21, iommu=smmu.0x0000000008000000)
[    9.265619] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx4: Adding to iommu group 44
[    9.274372] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx4: initialized (streamid=22, iommu=smmu.0x0000000008000000)
[    9.285768] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx5: Adding to iommu group 45
[    9.294493] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx5: initialized (streamid=23, iommu=smmu.0x0000000008000000)
[    9.305855] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx6: Adding to iommu group 46
[    9.314614] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx6: initialized (streamid=24, iommu=smmu.0x0000000008000000)
[    9.325984] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx7: Adding to iommu group 47
[    9.334769] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx7: initialized (streamid=25, iommu=smmu.0x0000000008000000)
[    9.346149] pva 16000000.pva0: Adding to iommu group 48
[    9.372230] pva 16000000.pva0: initialized
[    9.384386] nvdla 15880000.nvdla0: Adding to iommu group 49
[    9.400077] nvdla 15880000.nvdla0: initialized
[    9.406335] nvdla 158c0000.nvdla1: Adding to iommu group 50
[    9.421214] nvdla 158c0000.nvdla1: initialized
[    9.427443] tegra-gpcdma 2600000.gpcdma: Adding to iommu group 2
[    9.434501] tegra-gpcdma 2600000.gpcdma: GPC DMA driver register 31 channels
[    9.442007] tegra-fuse-burn efuse-burn: shutdown limit check disabled
[    9.448669] tegra-fuse-burn efuse-burn: Fuse burn driver initialized
[    9.455415] serial-tegra 3100000.serial: Adding to iommu group 2
[    9.462367] 3100000.serial: ttyTHS0 at MMIO 0x3100000 (irq = 13, base_baud = 0) is a TEGRA_UART
[    9.472120] serial-tegra 3110000.serial: Adding to iommu group 2
[    9.478796] 3110000.serial: ttyTHS1 at MMIO 0x3110000 (irq = 65, base_baud = 0) is a TEGRA_UART
[    9.488420] serial-tegra 3130000.serial: Adding to iommu group 2
[    9.494941] 3130000.serial: ttyTHS3 at MMIO 0x3130000 (irq = 66, base_baud = 0) is a TEGRA_UART
[    9.521827] tegra-xusb 3610000.xhci: Firmware timestamp: 2021-12-01 05:27:10 UTC, Version: 80.03 release
[    9.531613] tegra-xusb 3610000.xhci: xHCI Host Controller
[    9.537209] tegra-xusb 3610000.xhci: new USB bus registered, assigned bus number 1
[    9.545861] tegra-xusb 3610000.xhci: hcc params 0x0180ff05 hci version 0x120 quirks 0x0000000000050810
[    9.555508] tegra-xusb 3610000.xhci: irq 235, io mem 0x03610000
[    9.561974] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.10
[    9.570481] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    9.577923] usb usb1: Product: xHCI Host Controller
[    9.582952] usb usb1: Manufacturer: Linux 5.10.104-tegra xhci-hcd
[    9.589246] usb usb1: SerialNumber: 3610000.xhci
[    9.594647] hub 1-0:1.0: USB hub found
[    9.598538] hub 1-0:1.0: 4 ports detected
[    9.602918] tegra-xusb 3610000.xhci: xHCI Host Controller
[    9.608483] tegra-xusb 3610000.xhci: new USB bus registered, assigned bus number 2
[    9.616299] tegra-xusb 3610000.xhci: Host supports USB 3.1 Enhanced SuperSpeed
[    9.623819] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.10
[    9.632366] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    9.639832] usb usb2: Product: xHCI Host Controller
[    9.644851] usb usb2: Manufacturer: Linux 5.10.104-tegra xhci-hcd
[    9.651134] usb usb2: SerialNumber: 3610000.xhci
[    9.656339] hub 2-0:1.0: USB hub found
[    9.660214] hub 2-0:1.0: 4 ports detected
[    9.686600] tegra23x-oc-event soctherm-oc-event: Finished tegra23x overcurrent event probing
[    9.709615] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 2008000 KHz, changing to: 2035200 KHz
[    9.723829] cpufreq: cpufreq_online: CPU4: Running at unlisted initial frequency: 1989000 KHz, changing to: 2035200 KHz
[    9.737762] cpufreq: cpufreq_online: CPU8: Running at unlisted initial frequency: 2087000 KHz, changing to: 2112000 KHz
[    9.750110] tegra194-cpufreq e000000.ccplex: probed with ICC
[    9.757036] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[    9.759722] sdhci-tegra 3400000.sdhci: Got CD GPIO
[    9.764478] tegra194-pcie 14100000.pcie: host bridge /pcie@14100000 ranges:
[    9.769281] mmc0: CQHCI version 5.10
[    9.775593] tegra194-pcie 14100000.pcie:       IO 0x0030100000..0x00301fffff -> 0x0030100000
[    9.778276] irq: IRQ302: trimming hierarchy from :interrupt-controller@f400000-1
[    9.778313] sdhci-tegra 3400000.sdhci: wakeup init done, cdirq 302
[    9.801974] tegra194-pcie 14100000.pcie:      MEM 0x20a8000000..0x20afffffff -> 0x0040000000
[    9.807757] mmc0: SDHCI controller on 3460000.sdhci [3460000.sdhci] using ADMA 64-bit
[    9.810677] tegra194-pcie 14100000.pcie:      MEM 0x2080000000..0x20a7ffffff -> 0x2080000000
[    9.863785] usb 1-3: new full-speed USB device number 2 using tegra-xusb
[    9.915783] tegra194-pcie 14100000.pcie: Link up
[    9.917277] mmc0: Command Queue Engine enabled
[    9.922457] tegra194-pcie 14100000.pcie: PCI host bridge to bus 0001:00
[    9.925103] mmc0: periodic cache flush enabled
[    9.925109] mmc0: new HS400 Enhanced strobe MMC card at address 0001
[    9.931926] pci_bus 0001:00: root bus resource [bus 00-ff]
[    9.936826] mmcblk0: mmc0:0001 G1M15M 59.3 GiB
[    9.943037] pci_bus 0001:00: root bus resource [io  0x0000-0xfffff] (bus address [0x30100000-0x301fffff])
[    9.943039] pci_bus 0001:00: root bus resource [mem 0x20a8000000-0x20afffffff] (bus address [0x40000000-0x47ffffff])
[    9.943042] pci_bus 0001:00: root bus resource [mem 0x2080000000-0x20a7ffffff pref]
[    9.948874] mmcblk0rpmb: mmc0:0001 G1M15M partition 3 4.00 MiB, chardev (511:0)
[    9.953393] pci 0001:00:00.0: [10de:229e] type 01 class 0x060400
[    9.973382]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14
[    9.974193] pci 0001:00:00.0: PME# supported from D0 D3hot
[   10.012388] pci 0001:01:00.0: [10ec:c822] type 00 class 0x028000
[   10.018726] pci 0001:01:00.0: reg 0x10: [io  0x0000-0x00ff]
[   10.024453] usb 1-3: New USB device found, idVendor=13d3, idProduct=3549, bcdDevice= 0.00
[   10.024659] pci 0001:01:00.0: reg 0x18: [mem 0x20a8000000-0x20a800ffff 64bit]
[   10.032881] usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   10.041304] pci 0001:01:00.0: supports D1 D2
[   10.047565] usb 1-3: Product: Bluetooth Radio
[   10.051962] pci 0001:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[   10.056391] pci 0001:00:00.0: BAR 14: assigned [mem 0x20a8000000-0x20a80fffff]
[   10.056447] usb 1-3: Manufacturer: Realtek
[   10.063262] pci 0001:00:00.0: BAR 13: assigned [io  0x1000-0x1fff]
[   10.070697] usb 1-3: SerialNumber: 00e04c000001
[   10.074915] pci 0001:01:00.0: BAR 2: assigned [mem 0x20a8000000-0x20a800ffff 64bit]
[   10.093903] pci 0001:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[   10.100197] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[   10.105566] pci 0001:00:00.0:   bridge window [io  0x1000-0x1fff]
[   10.111868] pci 0001:00:00.0:   bridge window [mem 0x20a8000000-0x20a80fffff]
[   10.119229] pci 0001:00:00.0: Max Payload Size set to  128/ 256 (was  256), Max Read Rq  512
[   10.127984] pci 0001:01:00.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  512
[   10.136967] pcieport 0001:00:00.0: Adding to iommu group 9
[   10.142849] pcieport 0001:00:00.0: PME: Signaling with IRQ 51
[   10.148563] usb 2-3: new SuperSpeedPlus Gen 2 USB device number 2 using tegra-xusb
[   10.149010] pcieport 0001:00:00.0: AER: enabled with IRQ 51
[   10.163120] tegra194-pcie 14160000.pcie: Using GICv2m MSI allocator
[   10.171321] tegra194-pcie 14160000.pcie: host bridge /pcie@14160000 ranges:
[   10.178510] tegra194-pcie 14160000.pcie:       IO 0x0036100000..0x00361fffff -> 0x0036100000
[   10.187194] tegra194-pcie 14160000.pcie:      MEM 0x2428000000..0x242fffffff -> 0x0040000000
[   10.195897] tegra194-pcie 14160000.pcie:      MEM 0x2140000000..0x2427ffffff -> 0x2140000000
[   10.198234] usb 2-3: New USB device found, idVendor=0bda, idProduct=0420, bcdDevice= 1.40
[   10.213025] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   10.220375] usb 2-3: Product: 4-Port USB 3.0 Hub
[   10.225125] usb 2-3: Manufacturer: Generic
[   10.240053] hub 2-3:1.0: USB hub found
[   10.246108] hub 2-3:1.0: 4 ports detected
[   10.315922] usb 1-4: new high-speed USB device number 3 using tegra-xusb
[   10.493758] usb 1-4: New USB device found, idVendor=0bda, idProduct=5420, bcdDevice= 1.40
[   10.502178] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   10.509516] usb 1-4: Product: 4-Port USB 2.0 Hub
[   10.514256] usb 1-4: Manufacturer: Generic
[   10.520602] hub 1-4:1.0: USB hub found
[   10.525704] hub 1-4:1.0: 4 ports detected
[   11.239928] usb 1-4.1: new low-speed USB device number 4 using tegra-xusb
[   11.313392] tegra194-pcie 14160000.pcie: Phy link never came up
[   11.319546] tegra194-pcie 14160000.pcie: PCI host bridge to bus 0004:00
[   11.326362] pci_bus 0004:00: root bus resource [bus 00-ff]
[   11.332001] pci_bus 0004:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x36100000-0x361fffff])
[   11.342120] pci_bus 0004:00: root bus resource [mem 0x2428000000-0x242fffffff] (bus address [0x40000000-0x47ffffff])
[   11.352950] pci_bus 0004:00: root bus resource [mem 0x2140000000-0x2427ffffff pref]
[   11.358147] usb 1-4.1: New USB device found, idVendor=0c40, idProduct=8000, bcdDevice= 0.01
[   11.360869] pci 0004:00:00.0: [10de:229c] type 01 class 0x060400
[   11.369423] usb 1-4.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   11.375767] pci 0004:00:00.0: PME# supported from D0 D3hot
[   11.383120] usb 1-4.1: Product: 2.4GHz receiver
[   11.393416] usb 1-4.1: Manufacturer: ELMCU
[   11.396554] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[   11.403015] pci 0004:00:00.0: Max Payload Size set to  256/ 256 (was  256), Max Read Rq  512
[   11.404090] input: ELMCU 2.4GHz receiver as /devices/platform/3610000.xhci/usb1/1-4/1-4.1/1-4.1:1.0/0003:0C40:8000.0001/input/input0
[   11.411955] pcieport 0004:00:00.0: Adding to iommu group 10
[   11.429933] pcieport 0004:00:00.0: PME: Signaling with IRQ 53
[   11.436075] pcieport 0004:00:00.0: AER: enabled with IRQ 53
[   11.442040] pci_bus 0004:01: busn_res: [bus 01-ff] is released
[   11.448133] pci 0004:00:00.0: Removing from iommu group 10
[   11.453784] pci_bus 0004:00: busn_res: [bus 00-ff] is released
[   11.461218] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[   11.467875] vdd-3v3-pcie: supplied by vdd-3v3-sys
[   11.472910] avdd-cam-2v8: supplied by vdd-3v3-sys
[   11.477840] vdd-dp: supplied by vdd-3v3-sys
[   11.482302] vdd-3v3-sd: supplied by vdd-3v3-sys
[   11.487261] dsi-vdd-1v8-bl-en: supplied by vdd-1v8-sys
[   11.488296] hid-generic 0003:0C40:8000.0001: input,hidraw0: USB HID v1.11 Keyboard [ELMCU 2.4GHz receiver] on usb-3610000.xhci-4.1/input0
[   11.506848] tegra194-isp5 14800000.isp: initialized
[   11.510565] input: ELMCU 2.4GHz receiver Mouse as /devices/platform/3610000.xhci/usb1/1-4/1-4.1/1-4.1:1.1/0003:0C40:8000.0002/input/input1
[   11.524738] input: ELMCU 2.4GHz receiver System Control as /devices/platform/3610000.xhci/usb1/1-4/1-4.1/1-4.1:1.1/0003:0C40:8000.0002/input/input2
[   11.538840] tegra194-vi5 13e40000.host1x:vi0@15c00000: initialized
[   11.545276] (NULL device *): fops function table already registered
[   11.552521] tegra194-vi5 13e40000.host1x:vi1@14c00000: initialized
[   11.559497] sdhci-tegra 3400000.sdhci: Got CD GPIO
[   11.564505] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[   11.573696] sdhci-tegra 3400000.sdhci: wakeup init done, cdirq 302
[   11.595998] input: ELMCU 2.4GHz receiver Consumer Control as /devices/platform/3610000.xhci/usb1/1-4/1-4.1/1-4.1:1.1/0003:0C40:8000.0002/input/input3
[   11.609998] hid-generic 0003:0C40:8000.0002: input,hidraw1: USB HID v1.11 Mouse [ELMCU 2.4GHz receiver] on usb-3610000.xhci-4.1/input1
[   11.677088] tegra194-pcie 141a0000.pcie: host bridge /pcie@141a0000 ranges:
[   11.684268] tegra194-pcie 141a0000.pcie:       IO 0x003a100000..0x003a1fffff -> 0x003a100000
[   11.692966] tegra194-pcie 141a0000.pcie:      MEM 0x2b28000000..0x2b2fffffff -> 0x0040000000
[   11.701641] tegra194-pcie 141a0000.pcie:      MEM 0x2740000000..0x2b27ffffff -> 0x2740000000
[   12.618553] mmc1: SDHCI controller on 3400000.sdhci [3400000.sdhci] using ADMA 64-bit
[   12.817115] tegra194-pcie 141a0000.pcie: Phy link never came up
[   12.823267] tegra194-pcie 141a0000.pcie: PCI host bridge to bus 0005:00
[   12.830078] pci_bus 0005:00: root bus resource [bus 00-ff]
[   12.835714] pci_bus 0005:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x3a100000-0x3a1fffff])
[   12.845813] pci_bus 0005:00: root bus resource [mem 0x2b28000000-0x2b2fffffff] (bus address [0x40000000-0x47ffffff])
[   12.856654] pci_bus 0005:00: root bus resource [mem 0x2740000000-0x2b27ffffff pref]
[   12.864571] pci 0005:00:00.0: [10de:229a] type 01 class 0x060400
[   12.870898] pci 0005:00:00.0: PME# supported from D0 D3hot
[   12.884480] pci 0005:00:00.0: PCI bridge to [bus 01-ff]
[   12.889878] pci 0005:00:00.0: Max Payload Size set to  256/ 256 (was  256), Max Read Rq  512
[   12.898814] pcieport 0005:00:00.0: Adding to iommu group 11
[   12.904785] pcieport 0005:00:00.0: PME: Signaling with IRQ 55
[   12.910925] pcieport 0005:00:00.0: AER: enabled with IRQ 55
[   12.916875] pci_bus 0005:01: busn_res: [bus 01-ff] is released
[   12.922954] pci 0005:00:00.0: Removing from iommu group 11
[   12.928594] pci_bus 0005:00: busn_res: [bus 00-ff] is released
[   12.936442] nvethernet 6810000.ethernet: Adding to iommu group 51
[   12.943076] nvethernet 6810000.ethernet: failed to read skip mac reset flag, default 0
[   12.951234] nvethernet 6810000.ethernet: failed to read MDIO address
[   12.957781] nvethernet 6810000.ethernet: setting to default DMA bit mask
[   12.964706] nvethernet 6810000.ethernet: Failed to read DMA Tx ring size, using default [4096]
[   12.973570] nvethernet 6810000.ethernet: Failed to read DMA Tx ring size, using default [4096]
[   12.982438] nvethernet 6810000.ethernet: failed to read UPHY GBE mode- default to 10G
[   13.379266] nvethernet 6810000.ethernet: Ethernet MAC address: 48:b0:2d:a4:b5:60
[   13.387506] nvethernet 6810000.ethernet: Macsec not enabled
[   13.393245] nvethernet 6810000.ethernet: Macsec: Reduced MTU: 1466 Max: 16383
[   13.404194] nvethernet 6810000.ethernet: eth0 (HW ver: 31) created with 10 DMA channels
[   13.412880] irq: IRQ304: trimming hierarchy from :pmc@c360000
[   13.418844] irq: IRQ305: trimming hierarchy from :interrupt-controller@f400000-1
[   13.426518] irq: IRQ306: trimming hierarchy from :pmc@c360000
[   13.432499] input: gpio-keys as /devices/platform/gpio-keys/input/input4
[   13.440502] tegra-se-nvhost 15810000.se: Adding to iommu group 52
[   13.447150] tegra-se-nvhost 15810000.se: initialized
[   13.463270] tegra-se-nvhost 15810000.se: tegra_se_probe: complete
[   13.469663] tegra-se-nvhost 15820000.se: Adding to iommu group 53
[   13.476238] tegra-se-nvhost 15820000.se: initialized
[   13.483506] tegra-se-nvhost 15820000.se: tegra_se_probe: complete
[   13.489865] tegra-se-nvhost 15840000.se: Adding to iommu group 54
[   13.496411] tegra-se-nvhost 15840000.se: initialized
[   13.503896] tegra-se-nvhost 15840000.se: tegra_se_probe: complete
[   13.510749] tegra_actmon d230000.actmon: in actmon_register()...
[   13.517103] tegra_actmon d230000.actmon: bwmgr_disable = 1
[   13.523109] tegra_actmon d230000.actmon: initialization Completed for the device mc_all
[   13.531487] hvc_sysfs: hypervisor is not present
▒▒WARNING: clock_disable: clk_power_ungate on gated domain 35 for gpusysclk
WARNING: clock_disable: clk_power_ungate on gated domain 35 for gpc1clk
WARNING: clock_disable: clk_power_ungate on gated domain 35 for gpc0clk
▒▒[   13.666951] ALSA device list:
[   13.670011]   No soundcards found.
[   13.674087] Freeing unused kernel memory: 3968K
[   13.678822] Run /init as init process
[   13.694208] Root device found: mmcblk0p1
[   13.698707] Found dev node: /dev/mmcblk0p1
[   13.747765] EXT4-fs (mmcblk0p1): 1 orphan inode deleted
[   13.753151] EXT4-fs (mmcblk0p1): recovery complete
[   13.761300] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)
[   13.770436] Rootfs mounted over mmcblk0p1
[   13.789498] Switching from initrd to actual rootfs
[   13.913534] systemd[1]: System time before build time, advancing clock.
[   13.956851] systemd[1]: systemd 245.4-4ubuntu3.19 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +B                                            LKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid)
[   13.980476] systemd[1]: Detected architecture arm64.
[   14.024675] systemd[1]: Set hostname to <ubuntu>.
[   14.200522] systemd[1]: system-modprobe.slice: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling.
[   14.213573] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)
[   14.223149] systemd[1]: Created slice system-modprobe.slice.
[   14.229344] systemd[1]: Created slice system-serial\x2dgetty.slice.
[   14.236042] systemd[1]: Created slice User and Session Slice.
[   14.242063] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[   14.250168] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[   14.259926] systemd[1]: Reached target User and Group Name Lookups.
[   14.266428] systemd[1]: Reached target Slices.
[   14.271055] systemd[1]: Reached target Swap.
[   14.282156] systemd[1]: Listening on RPCbind Server Activation Socket.
[   14.289010] systemd[1]: Listening on Syslog Socket.
[   14.294194] systemd[1]: Listening on fsck to fsckd communication Socket.
[   14.301221] systemd[1]: Listening on initctl Compatibility Named Pipe.
[   14.308202] systemd[1]: Listening on Journal Audit Socket.
[   14.313966] systemd[1]: Listening on Journal Socket (/dev/log).
[   14.320221] systemd[1]: Listening on Journal Socket.
[   14.325482] systemd[1]: Listening on udev Control Socket.
[   14.331152] systemd[1]: Listening on udev Kernel Socket.
[   14.337629] systemd[1]: Mounting Huge Pages File System...
[   14.344269] systemd[1]: Mounting POSIX Message Queue File System...
[   14.351566] systemd[1]: Mounting RPC Pipe File System...
[   14.357999] systemd[1]: Mounting Kernel Debug File System...
[   14.364758] systemd[1]: Mounting Kernel Trace File System...
[   14.371743] systemd[1]: Starting Journal Service...
[   14.377912] systemd[1]: Starting Set the console keyboard layout...
[   14.385374] systemd[1]: Starting Create list of static device nodes for the current kernel...
[   14.395142] systemd[1]: Starting Load Kernel Module chromeos_pstore...
[   14.402702] systemd[1]: Condition check resulted in Load Kernel Module drm being skipped.
[   14.412106] systemd[1]: Starting Load Kernel Module efi_pstore...
[   14.420044] systemd[1]: Starting Load Kernel Module pstore_blk...
[   14.427623] systemd[1]: Starting Load Kernel Module pstore_zone...
[   14.435238] systemd[1]: Starting Load Kernel Module ramoops...
[   14.442271] systemd[1]: Starting NVIDIA specific first-boot udev script...
[   14.450419] systemd[1]: Started Nameserver information manager.
[   14.456721] systemd[1]: Reached target Network (Pre).
[   14.462729] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
[   14.472100] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[   14.489166] systemd[1]: Starting Load Kernel Modules...
[   14.495480] systemd[1]: Starting Remount Root and Kernel File Systems...
[   14.503124] EXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
[   14.503499] systemd[1]: Starting udev Coldplug all Devices...
[   14.504516] nvmap_heap_init: nvmap_heap_init: created heap block cache
[   14.504614] tegra-carveouts tegra-carveouts: fsi :dma coherent mem declare 0x0000000833000000,16777216
[   14.504617] tegra-carveouts tegra-carveouts: assigned reserved memory node fsi-carveout
[   14.504624] tegra-carveouts tegra-carveouts: vpr :dma coherent mem declare 0x0000000849800000,914358272
[   14.504625] tegra-carveouts tegra-carveouts: assigned reserved memory node vpr-carveout
[   14.504630] nvmap_page_pool_init: Total RAM pages: 7830706
[   14.504632] nvmap_page_pool_init: nvmap page pool size: 978838 pages (3823 MB)
[   14.504689] nvmap_background_zero_thread: PP zeroing thread starting.
[   14.504771] misc nvmap: created heap vpr base 0x0000000849800000 size (892928KiB)
[   14.507757] misc nvmap: created heap fsi base 0x0000000833000000 size (16384KiB)
[   14.574437] nvgpu: 17000000.ga10b          nvgpu_nvhost_syncpt_init:135  [INFO]  syncpt_unit_base 60000000 syncpt_unit_size 4000000 size 10000
[   14.574437]
[   14.608500] systemd[1]: Started Journal Service.
[   14.636951] systemd-journald[315]: Received client request to flush runtime journal.
[   14.924180] using random self ethernet address
[   14.930105] using random host ethernet address
[   14.950303] nvidia: loading out-of-tree module taints kernel.
[   15.286343] OF: graph: no port node found in /i2c@c240000/ucsi_ccg@8/connector@0
[   15.380687] OF: graph: no port node found in /i2c@c240000/ucsi_ccg@8/connector@0
[   15.388326] OF: graph: no port node found in /i2c@c240000/ucsi_ccg@8/connector@0
[   15.701741] using random self ethernet address
[   15.706394] using random host ethernet address
[   15.735824] Unable to handle kernel paging request at virtual address 005a18ef007b58ac
[   15.744000] Mem abort info:
[   15.746867]   ESR = 0x96000004
[   15.750028]   EC = 0x25: DABT (current EL), IL = 32 bits
[   15.751941] Unable to handle kernel paging request at virtual address 000b3fed4b2cedb5
[   15.755493]   SET = 0, FnV = 0
[   15.755795] Unable to handle kernel paging request at virtual address 000b3fed4b2cedcd
[   15.755797] Mem abort info:
[   15.755798]   ESR = 0x96000021
[   15.755800]   EC = 0x25: DABT (current EL), IL = 32 bits
[   15.755801]   SET = 0, FnV = 0
[   15.755801]   EA = 0, S1PTW = 0
[   15.755802] Data abort info:
[   15.755803]   ISV = 0, ISS = 0x00000021
[   15.755804]   CM = 0, WnR = 0
[   15.755805] [000b3fed4b2cedcd] address between user and kernel address ranges
[   15.755808] Internal error: Oops: 96000021 [#1] PREEMPT SMP
[   15.755811] Modules linked in: aes_ce_blk crypto_simd rtl8822ce rtk_btusb cryptd btusb input_leds aes_ce_cipher snd_soc_tegra186_asrc snd_soc_tegra210_ope snd_soc_tegra186_dspk btrtl ghash_c                                            e snd_soc_tegra186_arad btbcm snd_soc_tegra210_iqc sha2_ce snd_soc_tegra210_afc snd_soc_tegra210_mvc snd_soc_tegra210_dmic snd_soc_tegra210_adx snd_soc_tegra210_adsp snd_soc_tegra210_admaif bti                                            ntel sha256_arm64 snd_soc_tegra210_amx snd_soc_tegra210_mixer snd_soc_tegra210_sfc snd_soc_tegra210_i2s ucsi_ccg cfg80211 snd_soc_tegra_pcm snd_soc_tegra_machine_driver snd_hda_codec_hdmi sha1_                                            ce snd_soc_tegra_utils nvadsp snd_hda_tegra typec_ucsi nct1008 snd_soc_simple_card_utils snd_soc_tegra210_ahub snd_soc_spdif_tx snd_hda_codec typec snd_hda_core tegra210_adma loop snd_soc_rt564                                            0 tegra_bpmp_thermal snd_soc_rl6231 userspace_alert spi_tegra114 nvidia(O) binfmt_misc ina3221 pwm_fan nvgpu nvmap ip_tables x_tables
[   15.755877] CPU: 1 PID: 855 Comm: apport Tainted: G           O      5.10.104-tegra #1
[   15.755878] Hardware name: Unknown Jetson AGX Orin/Jetson AGX Orin, BIOS 2.1-32413640 01/24/2023
[   15.755881] pstate: a0400009 (NzCv daif +PAN -UAO -TCO BTYPE=--)
[   15.755891] pc : propagate_protected_usage+0xcc/0xf0
[   15.755893] lr : propagate_protected_usage+0x20/0xf0
[   15.755894] sp : ffff80001000bcd0
[   15.755894] x29: ffff80001000bcd0 x28: ffffd9194ae67000
[   15.755896] x27: ffff53674a1f0000 x26: ffff53674a1f0000
[   15.755898] x25: ffff80001000be90 x24: ffff536e6ebc1af0
[   15.755899] x23: ffffd9194b1eae40 x22: ffff53674a1f0000
[   15.755900] x21: ffffd9194b158998 x20: b862689f511f9eef
[   15.755902] x19: ffff536744d18140 x18: 0000000000000000
[   15.755903] x17: 0000000000000000 x16: 0000000000000000
[   15.755904] x15: 0000000000000000 x14: 0000000000000000
[   15.755905] x13: 00000000fffee9dc x12: ffffffffffffff19
[   15.755907] x11: ffffd9194b156000 x10: ffffd9194b1e7a80
[   15.755908] x9 : 0000aa1fe84f6238 x8 : 04a43eb94560c27c
[   15.755910] x7 : 00000000010b23ab x6 : ffffd9194aad6c48
[   15.755911] x5 : 000000000000000b x4 : 0000000000000001
[   15.755912] x3 : ffff536744d18190 x2 : 4e0b3fed4b2cedcd
[   15.755914] x1 : b862689f511f9eef x0 : 4e0b3fed4b2ced75
[   15.755915] Call trace:
[   15.755917]  propagate_protected_usage+0xcc/0xf0
[   15.755918]  page_counter_cancel+0x40/0x80
[   15.755919]  page_counter_uncharge+0x34/0x50
[   15.755923]  uncharge_batch+0xd8/0x130
[   15.755924]  mem_cgroup_uncharge+0x58/0x80
[   15.755928]  __put_page+0x54/0x70
[   15.755931]  free_page_and_swap_cache+0xf8/0x110
[   15.755934]  tlb_remove_table_rcu+0x38/0x60
[   15.755937]  rcu_core+0x274/0x980
[   15.755939]  rcu_core_si+0x18/0x20
[   15.755941]  __do_softirq+0x140/0x3e8
[   15.755943]  irq_exit+0xc0/0xe0
[   15.755946]  __handle_domain_irq+0x74/0xd0
[   15.755947]  gic_handle_irq+0x68/0x134
[   15.755948]  el0_irq_naked+0x4c/0x54
[   15.755951] Code: f9401660 91016002 d503201f d503201f (f821005f)
[   15.755962] ---[ end trace 84c6b9d8fe9c96ae ]---
[   15.755963] Kernel panic - not syncing: Oops: Fatal exception in interrupt
[   15.755965] SMP: stopping secondary CPUs
[   15.763644] Kernel Offset: 0x591939450000 from 0xffff800010000000
[   15.763645] PHYS_OFFSET: 0xffffac99c0000000
[   15.763647] CPU features: 0x0040006,4a80aa38
[   15.763648] Memory Limit: none
[   16.118416] ---[ end Kernel panic - not syncing: Oops: Fatal exception in interrupt ]---

Hi ddrogahn1,

Are you using standalone Ubuntu 18.04 or 20.04 as your host PC for SDK Manager?

For you case, please select “Jetson AGX Orin”.

To narrow down the issue, please remove NVMe drive and use SDKM to flash your devkit again.

I am facing the similar issue. The nvidia Orin is failed to boot after 5.1 update.

What is the purpose of SdkManager recovery mode flash? It will fix the crashed kernel or it will reformat the whole Orin with a new kernel?

Any suggestion how can I investigate my Orin and recover it back?

Hi chunjye088,

It would re-flash the whole Orin to make sure the clean image inside.

Please get a standalone Ubuntu18.04 or 20.04 as your host PC and refer to the following instructions to flash the board.
Install Jetson Software with SDK Manager :: NVIDIA SDK Manager Documentation

So there is no chance the fix and keep the data inside after fail to boot from 5.1 update?

What is the purpose of flash.sh in Linux_for_Tegra?

If you could not boot up, you might not access the rootfs in internal eMMC.
Or you could try clone them first.

This is the tool for flash the board.
You could refer to the following guide for detail.
Flashing Support — Jetson Linux Developer Guide documentation (nvidia.com)

hello,I have a very confusing question. I am looking at the code. There is something related to tegra dce in the code. What is the main purpose of dce?

 tegra-dce d800000.dce: Adding to iommu group 5
[    6.930084] dce: dce_ipc_channel_init:314  Invalid Channel State [0x0] for ch_type [2]
[    6.939221] tegra234-aon c000000.aon: Adding to iommu group 6
[    6.945615] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB : [0x5]
[    6.954575] dce: dce_admin_ipc_handle_signal:90   Spurious signal on channel: [0]. Ignored...
[    6.955539]  c000000.aon:hsp: probed
[    6.963382] dce: dce_admin_send_cmd_ver:424  version : [0x2] err : [0x0]
[    6.963522] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB : [0x1]
[    6.967293] tegra234-aon c000000.aon: init done

Display Controller Engine(DCE) is used to run display related code on CPU with some functionalities.
The logs provided from you seems fine,.

I was previously using Lubuntu, which would explain the dependency and qemu version problems (dependencies were an easy fix, qemu was harder 2.12 works)… (and also had tried WSL2)

I’ve now tried 5.1 (r35.2) with fresh installs of Ubuntu 18.04 and 20.04, and it still fails the same way. After flash: it reboots, loads the GUI and runs OEM pre-config, but all future reboots crash

–Often in the log before the crash:

  • always right after the fan spins up, before it spins down (it’s stuck on high until a manual or watchdog reboot) (could be the fan causing a power dip?)
  • “using random self ethernet address”
  • “systemd: Started Journal Service.”
  • “systemd-journald[303]: Received client request to flush runtime journal.”

–A few contributing quirks & things to note:

  • If the rootfs creation is aborted - the folder is not removed. The command mknod returns an error if its target exists – and you get a fail loop until the rootfs folder is manually removed. Config=Runtime lets me choose when to reboot, so I can run some things, but still fails.

  • On a USB2 host, Orin may fail to reconnect during the flash process. It may work better if connected through a usb3 hub.

  • DVI to HDMI adapter: the boot screen displays on HDMI, but the X11 GUI will not?!

  • NFS rcm-boot (Recovery Mode) works! It boots the 5.1 rootfs (same one that I’m flashing) and gets to the GUI. But, it can still crash mysteriously (like after pressing tab/autocomplete in the serial console). Maybe it works because it is slower due to network delay? (maybe everything would work if installed on SD or USB?)
    sudo ./flash.sh --rcm-boot -r -N 192.168.1.2:/nfs/jetson_rootfs jetson-agx-orin-devkit eth0

  • NFS Boot: If oem config is set to runtime, but l4t_create_default_user.sh is not run – the log appears to hang on this. It’s doing something on the network, but nfsstats says it’s all “getattr”.
    Please complete system configuration setup on the serial port provided by Jetson's USB device mode connection. e.g. /dev/ttyUSBx where x can 0, 1, 2 etc.
    (Note: it says ttyUSB instead of ttyACM - which seems to be wrong? I have also seen it in the log with the correct device name.)

  • From NFS boot: I tried “apt dist-upgrade” and then flashed the updated rootfs – normal boot still crashes.

  • From NFS boot: I tried extracting a tgz of rootfs into mmcblk0p1 (that was previously all flashed normally) – normal boot still crashes.

  • Tried increasing systemd loglevel, if it worked, it didn’t add much

  • Tried TX1 power adapter (barrel plug instead of USB PD): 19V 90W – no change

  • Does the order in the SDK Manager “detected device” dialog reflect anything? Sometimes the Jetson is first, sometimes it isn’t.

–More version notes:

Hardware name: Unknown Jetson AGX Orin/Jetson AGX Orin, BIOS 2.1-32413640 01/24/2023
;
Jetson AGX Orin
 Not Specified                                       2.00 GHz
 2.1-32413640                                        0 MB RAM
;
699-13701-0000-500 L.0
;
./nvautoflash.sh --print_boardid
--- Reading board information succeeded.
Parsing module EEPROM:
--- Parsing board ID (3701) succeeded.
--- Parsing board version (500) succeeded.
--- Parsing board SKU (0000) succeeded.
--- Parsing board REV (L.0) succeeded.
jetson-agx-orin-devkit found.

–Another exception type:

[   14.728223] Internal error: SP/PC alignment exception: 8a000000 [#1] PREEMPT SMP
[   14.728226] Modules linked in: bnep aes_ce_blk crypto_simd cryptd rtk_btusb rtl8822ce aes_ce_cipher btusb ghash_ce btrtl sha2_ce snd_soc_tegra186_asrc snd_soc_tegra210_adsp snd_soc_tegra210_ope btbcm snd_soc_tegra186_dspk ucsi_ccg loop sha256_arm64 snd_soc_tegra186_arad snd_soc_tegra210_mvc snd_soc_tegra210_iqc snd_soc_tegra210_afc snd_soc_tegra210_admaif snd_soc_tegra210_dmic btintel snd_soc_tegra210_adx typec_ucsi snd_hda_codec_hdmi snd_soc_tegra_machine_driver sha1_ce nvadsp snd_soc_tegra_utils snd_soc_tegra210_i2s snd_soc_tegra210_mixer snd_soc_tegra210_amx snd_soc_tegra210_sfc typec snd_hda_tegra snd_soc_tegra_pcm snd_soc_simple_card_utils snd_soc_tegra210_ahub snd_soc_spdif_tx snd_hda_codec nct1008 cfg80211 tegra210_adma snd_soc_rt5640 tegra_bpmp_thermal snd_hda_core userspace_alert snd_soc_rl6231 spi_tegra114 nvidia(O) ina3221 pwm_fan binfmt_misc nvgpu nvmap ip_tables x_tables
[   14.728286] CPU: 4 PID: 935 Comm: udevadm Tainted: G           O      5.10.104-tegra #1
[   14.728287] Hardware name: Unknown Jetson AGX Orin/Jetson AGX Orin, BIOS 2.1-32413640 01/24/2023
[   14.728289] pstate: 60400009 (nZCv daif +PAN -UAO -TCO BTYPE=--)
[   14.728292] pc : 0x7ae1462c9e9f5dc3
[   14.728300] lr : rcu_core+0x274/0x980
[   14.728301] sp : ffff800010023e00
[   14.728302] x29: ffff800010023e00 x28: ffffce0e40e67000
[   14.728304] x27: ffff5662c1324880 x26: ffff5662c1324880
[   14.728305] x25: ffff800010023e90 x24: ffff5669eec20af0
[   14.728306] x23: ffffce0e411eae40 x22: ffff5662c1324880
[   14.728308] x21: ffffce0e41158998 x20: ffffce0e40e67000
[   14.728309] x19: ffff5669eec20a80 x18: 0000000000000010
[   14.728310] x17: 0000000000000000 x16: 0000000000000000
[   14.728311] x15: ffff5662c1324df0 x14: 0720072007200720
[   14.728313] x13: 00000000fffee8df x12: ffffffffffffff31
[   14.728314] x11: ffffce0e41156000 x10: 0000000000000640
[   14.728315] x9 : ffff885badda4000 x8 : ffff5662c1324880
[   14.728317] x7 : 0000000000000020 x6 : ffff885badda4000
[   14.728318] x5 : 0000000000000001 x4 : ffffff598b23bb20
[   14.728319] x3 : 0000000080200005 x2 : ffff5662d0eed000
[   14.728321] x1 : 7ae1462c9e9f5dc3 x0 : ffff5662d0eed000
[   14.728322] Call trace:
[   14.728323]  0x7ae1462c9e9f5dc3
[   14.728326]  rcu_core_si+0x18/0x20
[   14.728328]  __do_softirq+0x140/0x3e8
[   14.728332]  irq_exit+0xc0/0xe0
[   14.728335]  __handle_domain_irq+0x74/0xd0
[   14.728336]  gic_handle_irq+0x68/0x134
[   14.728337]  el1_irq+0xd0/0x180
[   14.728338]  console_unlock+0x36c/0x540
[   14.728339]  vprintk_emit+0x124/0x2a0
[   14.728341]  vprintk_default+0x48/0x60
[   14.728342]  vprintk_func+0xfc/0x2b0
[   14.728346]  printk+0x70/0x94
[   14.728347]  mem_abort_decode+0x9c/0x12c
[   14.728350]  die_kernel_fault+0x50/0x80
[   14.728351]  __do_kernel_fault+0x8c/0x1c0
[   14.728353]  do_bad_area+0x84/0xa0
[   14.728355]  do_translation_fault+0x64/0x90
[   14.728356]  do_mem_abort+0x54/0xb0
[   14.728358]  el1_abort+0x44/0x70
[   14.728359]  el1_sync_handler+0x98/0xb0
[   14.728360]  el1_sync+0x8c/0x140
[   14.728364]  kernfs_find_ns+0x60/0x130
[   14.728366]  kernfs_iop_lookup+0x5c/0xd0
[   14.728369]  __lookup_slow+0x90/0x150
[   14.728370]  walk_component+0x120/0x1f0
[   14.728371]  path_lookupat.isra.0+0x84/0x180
[   14.728372]  filename_lookup+0x9c/0x170
[   14.728373]  user_path_at_empty+0x5c/0x70
[   14.728376]  do_faccessat+0x98/0x2b0
[   14.728377]  __arm64_sys_faccessat+0x30/0x40
[   14.728380]  el0_svc_common.constprop.0+0x80/0x1d0
[   14.728381]  do_el0_svc+0x38/0xb0
[   14.728382]  el0_svc+0x1c/0x30
[   14.728383]  el0_sync_handler+0xa8/0xb0
[   14.728384]  el0_sync+0x16c/0x180
[   14.728387] Code: bad PC value
[   14.728398] ---[ end trace c734c3072143a90a ]---
[   14.728399] Kernel panic - not syncing: SP/PC alignment exception: Fatal exception in interrupt
[   14.728401] SMP: stopping secondary CPUs
[   14.728787] Kernel Offset: 0x4e0e2f450000 from 0xffff800010000000
[   14.728788] PHYS_OFFSET: 0xffffa99e40000000
[   14.728789] CPU features: 0x0040006,4a80aa38
[   14.728790] Memory Limit: none
[   15.107258] ---[ end Kernel panic - not syncing: SP/PC alignment exception: Fatal exception in interrupt ]---

Could you help to re-flash Jetpack5.1(r35.2.1) with SDK Manager again and provide the full flash log at host side? (you could just provide the SDKM log after flash).

And please also provide the serial console log from the board when you hit the issue of reboot crash.
NVIDIA Jetson Orin - Serial Console

Updated to sdkManager 1.9.2.10884 and tried 5.1 rev 1.
It crashes on the 1st boot - doesn’t make it to the system configuration.
Here are the sdkm and serial logs.

SDKM_logs_2023-03-17_10-47-42.zip (134.6 KB)
putty.log (117.9 KB)

It seems fine in the flash log.

Does it crash only on the 1st boot? Or also on the further reboot?

Do you have other Orin devkit with the similar kernel panic issue?

It fails 1st boot and all later boots, even after reset or power cycle.

But, 5.1 rev 1 appears to work with NFS boot.
(I did multiple reboots: set maxn, run apt dist-upgrade, autoremove, but havent installed cuda)
The previous (v5 before rev 1) would get through the first boot – but fail after OEM pre-config.
4.x has no problem booting.

No, I only have one Orin, and an old TX1.

Do you have any peripherals connected on your devkit?

JP4.x is not compatible with AGX Orin…

Peripherals:
Sometimes usb to wireless keyboard/mouse.
Usb console to a windows pc, usb3 to Linux pc, ethernet, DP, power.
(all devices on the same power strip – sometimes had usb console going to the same linux pc, but harder to capture)

I meant JP 5.0.2 boots fine.

Could you help to disconnect all peripherals and just keep MicroUSB connected (to capture log) and capture the serial console log for further check?

I tried also removing the M.2 wireless card (only remaining connections are: power, usb console, usb for flash)
(both usb are connected to the same host machine – no ethernet, no DP, no usb keyboard)
Then the first boot after flash succeeded, and later boots failed.

Then, I added kernel option logelvel=8 – so these logs may be more detailed.
I tried a flash & few boots with the wireless card, all failed.
putty 0321.log (207.0 KB)

I tried flashing again without the wireless card: it failed the 1st boot, and succeeded on the 2nd or 3rd (it completed oem-config, I could login), then failed a few boots…
putty 0321b.log (726.5 KB)

Now I always get a recovery boot – logged as “Mount initrd as rootfs and enter recovery mode”
(I don’t know if there is anything I can to try from there, “reboot” or Ctrl-D consistently triggers a kernel panic)

The recovery kernel provides a friendly bash for user to do some corrections.
If you don’t know what to do, manual action to enter force recovery mode (RCM) to re-flash the board is required.

It seems something wrong with your wireless card and cause kernel panic, which wireless are you using?

It’s the stock wireless/bluetooth card in the devkit: Realtek RTL8822CE
The wireless connection works fine in JP 5.0.2

Would it help to disable the driver? (Can the driver affect the system when inactive?)

JP 5.1.1 [Mar 28] also fails on first boot
Is something maybe wrong with or around systemd-journald flush?

–To recap:
Jetson Orin Dev kit 32G & SDK Manager on Ubuntu 20.04.6

  • JP 5.0 works
  • JP 5.1 fails (kernel panic) on 2nd boot (after oem-config):
    ++ tried: internal emmc, NVMe, usb
  • JP 5.1.0.1 (5.1 rev 1) & 5.1.1 fail on 1st boot
  • JP 5.x works with NFS boot: persistent or -rcm-boot

–Updates:

  • After a lot of logged boots… In most cases, the error is nearly or immediately after “systemd-journald[#]: Received client request to flush runtime journal.”

  • increasing the number of lines in the systemd log level seems to make the crash happen earlier (kernel command line: “systemd.log_level=debug” vs =err)

  • removing the wifi device shortens the log by several lines, so the boot works for a little longer (after a crash, fsck adds a few log lines and makes the next crash happen earlier?)

  • adding “1” (runlevel 1) to the kernel command line (with a low log level) will pretty reliably get to a prompt on the usb debug connection
    ++ then about 1/4 of the time “init 3” will work (lower the fan speed, enable networking, start sshd)
    ++ init 5 would start the x11 gui - but I dont think that has ever worked (except testing in NFS boot, or 5.0)
    ++ it may still randomly crash, when the log eventually grows big enough for journald to flush?

  • possible bug: if the kernel command line is too long (about 100 characters) - the boot firmware is corrupted? Shortening the cmd will not fix it – boot will fail until reflashed.
    (also: editing the kernel command line, when over a certain length of text, displays the cursor incorrectly)

  • Why are there are corrupt characters like “▒” in the console log? I see some other users’ logs have the same thing.

–More debugging I tried:

  • I can’t set the root filesystem to readonly “RO” with the kernel command line, because it has “RW” (read-write) prepended?

  • I tried configuring journald to not use disk – but see no change in the crashes…

/etc/systemd/journald.conf

Storage=volatile
RuntimeMaxUse=64M

What else can I do?