Can Jetson orin nx 8g disable CONFIG_PCI_IOV in kernel config?

[bbear953308023]

I am working with a custom carrier board with Jetson orin nx 8g.
In my carrier board, I used a vl805. And it’s connected to PCIE2(C7).
And I can make it’s basic function works.But there is a stability issue after I plugin a u-disk(usb3.0) to vl805.
After a few minutes, the PCIE2(C2) seems to blocked and I can do anything with it anymore.

I tried to post this problem to the fae of vl805. They told us to disable the IOV configuration to fix it. I searched the kernel config of Jetson kernel(R36.3.0 kernel version:5.15.136), and I find a the CONFIG_PCI_IOV.

Then I tried the recompile the kernel and drivers with the CONFIG_PCI_IOV disabled. But the system can’t boot up after I reflash the device. Here is the detail log of it:

[    7.294286] pcie_tegra194: module verification failed: signature and/or required key missing - tainting kernel
[    7.294409] pcie_tegra194: disagrees about version of symbol pci_generic_config_write
[    7.294412] pcie_tegra194: Unknown symbol pci_generic_config_write (err -22)
[    7.294420] pcie_tegra194: disagrees about version of symbol dw_pcie_host_deinit
[    7.294421] pcie_tegra194: Unknown symbol dw_pcie_host_deinit (err -22)
[    7.294431] pcie_tegra194: disagrees about version of symbol dw_pcie_find_ext_capability
[    7.294432] pcie_tegra194: Unknown symbol dw_pcie_find_ext_capability (err -22)
[    7.294438] pcie_tegra194: disagrees about version of symbol dw_pcie_setup_rc
[    7.294439] pcie_tegra194: Unknown symbol dw_pcie_setup_rc (err -22)
[    7.294442] pcie_tegra194: disagrees about version of symbol dw_pcie_wait_for_link
[    7.294443] pcie_tegra194: Unknown symbol dw_pcie_wait_for_link (err -22)
[    7.294451] pcie_tegra194: disagrees about version of symbol pci_generic_config_read
[    7.294452] pcie_tegra194: Unknown symbol pci_generic_config_read (err -22)
[    7.294468] pcie_tegra194: disagrees about version of symbol dw_pcie_own_conf_map_bus
[    7.294469] pcie_tegra194: Unknown symbol dw_pcie_own_conf_map_bus (err -22)
[    7.294476] pcie_tegra194: disagrees about version of symbol dw_pcie_write_dbi
[    7.294477] pcie_tegra194: Unknown symbol dw_pcie_write_dbi (err -22)
[    7.294478] pcie_tegra194: disagrees about version of symbol dw_pcie_read_dbi
[    7.294479] pcie_tegra194: Unknown symbol dw_pcie_read_dbi (err -22)
[    7.294489] pcie_tegra194: disagrees about version of symbol dw_pcie_find_capability
[    7.294490] pcie_tegra194: Unknown symbol dw_pcie_find_capability (err -22)
[    7.294506] pcie_tegra194: disagrees about version of symbol pci_rescan_bus
[    7.294507] pcie_tegra194: Unknown symbol pci_rescan_bus (err -22)
[    7.294513] pcie_tegra194: disagrees about version of symbol dw_pcie_host_init
[    7.294514] pcie_tegra194: Unknown symbol dw_pcie_host_init (err -22)
[    7.393342] nvme: disagrees about version of symbol pci_save_state
[    7.393348] nvme: Unknown symbol pci_save_state (err -22)
[    7.393353] nvme: disagrees about version of symbol pcie_aspm_enabled
[    7.393354] nvme: Unknown symbol pcie_aspm_enabled (err -22)
[    7.393368] nvme: disagrees about version of symbol pci_request_selected_regions
[    7.393369] nvme: Unknown symbol pci_request_selected_regions (err -22)
[    7.393371] nvme: disagrees about version of symbol pci_release_selected_regions
[    7.393371] nvme: Unknown symbol pci_release_selected_regions (err -22)
[    7.393375] nvme: disagrees about version of symbol pci_enable_device_mem
[    7.393376] nvme: Unknown symbol pci_enable_device_mem (err -22)
[    7.393402] nvme: disagrees about version of symbol __pci_register_driver
[    7.393402] nvme: Unknown symbol __pci_register_driver (err -22)
[    7.393405] nvme: disagrees about version of symbol pcibios_resource_to_bus
[    7.393406] nvme: Unknown symbol pcibios_resource_to_bus (err -22)
[    7.393421] nvme: disagrees about version of symbol pci_disable_pcie_error_reporting
[    7.393423] nvme: Unknown symbol pci_disable_pcie_error_reporting (err -22)
[    7.393427] nvme: disagrees about version of symbol pci_irq_vector
[    7.393428] nvme: Unknown symbol pci_irq_vector (err -22)
[    7.393436] nvme: disagrees about version of symbol pci_unregister_driver
[    7.393437] nvme: Unknown symbol pci_unregister_driver (err -22)
[    7.393450] nvme: disagrees about version of symbol blk_mq_pci_map_queues
[    7.393450] nvme: Unknown symbol blk_mq_pci_map_queues (err -22)
[    7.393466] nvme: disagrees about version of symbol pci_free_irq
[    7.393466] nvme: Unknown symbol pci_free_irq (err -22)
[    7.393477] nvme: disagrees about version of symbol pci_select_bars
[    7.393478] nvme: Unknown symbol pci_select_bars (err -22)
[    7.393484] nvme: disagrees about version of symbol pci_request_irq
[    7.393485] nvme: Unknown symbol pci_request_irq (err -22)
[    7.393490] nvme: disagrees about version of symbol pci_load_saved_state
[    7.393491] nvme: Unknown symbol pci_load_saved_state (err -22)
[    7.393495] nvme: disagrees about version of symbol pci_device_is_present
[    7.393496] nvme: Unknown symbol pci_device_is_present (err -22)
[    7.393499] nvme: disagrees about version of symbol pci_read_config_word
[    7.393499] nvme: Unknown symbol pci_read_config_word (err -22)
[    7.393511] nvme: disagrees about version of symbol pci_restore_state
[    7.393512] nvme: Unknown symbol pci_restore_state (err -22)
[    7.393518] nvme: disagrees about version of symbol pci_enable_pcie_error_reporting
[    7.393519] nvme: Unknown symbol pci_enable_pcie_error_reporting (err -22)
[    7.393521] nvme: disagrees about version of symbol pci_alloc_irq_vectors_affinity
[    7.393522] nvme: Unknown symbol pci_alloc_irq_vectors_affinity (err -22)
[    7.393523] nvme: disagrees about version of symbol pci_set_master
[    7.393524] nvme: Unknown symbol pci_set_master (err -22)
[    7.393553] nvme: disagrees about version of symbol pci_disable_device
[    7.393553] nvme: Unknown symbol pci_disable_device (err -22)
[    7.393560] nvme: Unknown symbol pci_sriov_configure_simple (err -2)
[    7.393563] nvme: disagrees about version of symbol pci_free_irq_vectors
[    7.393564] nvme: Unknown symbol pci_free_irq_vectors (err -22)
[    7.576364] tegra-xudc 3550000.usb: Adding to iommu group 3
[   17.895249] ERROR: PARTUUID=5d580646-7fc8-4d37-ae70-40ff9a87c831 mount fail...

So can I disabled the CONFIG_PCI_IOV in jetson’s kernel config. If I can, what did I missed to fix the bootup error?

And I want to know if there is anyone who had used vl805 before.
If the vl805 is not supported by jetson officially. Can you provide better choices for PCIE-to-USB3.0 chip.

Looking forward to you reply~~~

[WayneWWW]

Did you remember to rebuild every kernel module after you modify the kernel image?

[bbear953308023]

Yes, I recompile the kernel with command ./nvbuild.sh.

[WayneWWW]

Yeah… but did you update those kernel modules or not…

Building and whether you updated the driver correctly are two separate things…

You know the difference of kernel image and kernel modules, right…?

[bbear953308023]

Yeah, I know the difference of kernel image and kernel modules.

Here is the detail steps I compile the kernel and flash the device:
1 remove CONFIG_PCI_IOV=y from Linux_for_Tegra/source/kernel/kernel-jammy-src/arch/arm64/configs/defconfig
2 compile the kernel and driver with

cd source
./nvbuild.sh

3 install the kernel and kernel module

cd source
cp  ./kernel_out/kernel/kernel-jammy-src/arch/arm64/boot/Image ../kernel/Image
export INSTALL_MOD_PATH=`realpath ../rootfs/`
./nvbuild.sh -i

3 flash the device with new kernel and kernel module

cd Linux_for_Tegra
sudo  ./tools/kernel_flash/l4t_initrd_flash.sh --external-device nvme0n1p1 -c tools/kernel_flash/flash_l4t_nvme.xml -S 80GiB  -p "-c bootloader/generic/cfg/flash_t234_qspi.xml --no-systemimg"  --network usb0 jetson-orin-nano-devkit external

[WayneWWW]

Some kernel modules are located in initrd and your command won’t update it.

Please unpack the initrd image and put new kernel modules into it too.

[bbear953308023]

Oh, I see.
Is there any document which can show me how to update the new kernel modules in initrd?

[WayneWWW]

No, there is no document. Only some modules are located in initrd.

initrd image is under Linux_for_Tegra/bootloader

mkdir initrd; cd initrd;
sudo gunzip -c …/l4t_initrd.img.orig | sudo cpio -i
→ update the files you want to update
sudo find . | sudo cpio --quiet -H newc -o | sudo gzip -9 -n > …/l4t_initrd.img

[bbear953308023]

Thank you. It seems works. I will test it and feedback to you~

[bbear953308023]

I tried to update all the kernel modules I found in initrd.
The kernel log did go further. But it go into the bash in the end.
Here is my bootup kernel log:

[    0.000000] Linux version 5.15.136-tegra (root@stu) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2022.08) 11.3.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT Tue Sep 3 11:19:33 UTC 2024 ()
[    0.000000] Machine model: NVIDIA Jetson Orin NX Engineering Reference Developer Kit
[    0.000000] efi: EFI v2.70 by EDK II
[    0.000000] efi: RTPROP=0x26d83f198 SMBIOS=0xffff0000 SMBIOS 3.0=0x26d210000 MEMATTR=0x267790018 ESRT=0x267209318 RNG=0x25a920018 MEMRESERVE=0x25b9acf18
[    0.000000] random: crng init done
[    0.000000] secureboot: Secure boot disabled
[    0.000000] esrt: Reserving ESRT space from 0x0000000267209318 to 0x0000000267209350.
[    0.000000] Reserved memory: created CMA memory pool at 0x000000024a000000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000080000000-0x0000000277ffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x2712f9800-0x2712fbfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000000277ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000bdffffff]
[    0.000000]   node   0: [mem 0x00000000c2000000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000025e24cfff]
[    0.000000]   node   0: [mem 0x000000025e24d000-0x000000025e41bfff]
[    0.000000]   node   0: [mem 0x000000025e41c000-0x000000026b8fffff]
[    0.000000]   node   0: [mem 0x000000026b900000-0x000000026d83ffff]
[    0.000000]   node   0: [mem 0x000000026d840000-0x0000000271dfffff]
[    0.000000]   node   0: [mem 0x0000000271e00000-0x0000000271ffffff]
[    0.000000]   node   0: [mem 0x0000000272000000-0x000000027259ffff]
[    0.000000]   node   0: [mem 0x0000000272f00000-0x0000000272ffffff]
[    0.000000]   node   0: [mem 0x0000000276000000-0x0000000277ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000277ffffff]
[    0.000000] On node 0, zone DMA: 16384 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 2400 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 12288 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 29 pages/cinsmod /l408 r8module d3.154 u118784egra/u  0.00/dr0] rs/net/eted PIPnet-caaltek/r8168. on CP

[    0.000000] CPU features: detected: Address authentication (architected algorithm)
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: detected: Hardware dirty bit management
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU featuresmodprobe:: dcteRORKer: c page d not insert 'rion (K In)
valid .00000nt0]
alternatives: patching kernel code
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 2001056
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/initrd rw rootwait mminit_loglevel=4 console=ttyTCU0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 bl_prof_dataptr=2031616@0x271E10000 bl_prof_ro_ptr=65536@0x271E00000
[    0.000000] Unknown kernel command line parameters "bl_prof_dataptr=2031616@0x271E10000 bl_prof_ro_ptr=65536@0x271E00000", will be passed to user space.
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000fbfe0000-0x00000000fffe0000] (64MB)
[    0.000000] Memory: 7517248K/8133248K available (19072K kernel code, 4060K rwdata, 10032K rodata, 7616K init, 529K bss, 353856K reserved, 262144K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=6.
[    0.000000]  Trampoline variant of Tasks RCU enabled.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=6
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 960 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x000000000f440000
[    0.000000] arch_timer: cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000001] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.000374] Console: colour dummy device 80x25
[    0.000440] Calibrating delay loop (skipped), value calculated using timer frequency.. 62.50 BogoMIPS (lpj=125000)
[    0.000448] pid_max: default: 32768 minimum: 301
[    0.000487] LSM: Security Framework initializing
[    0.000507] Yama: becoming mindful.
[    0.000522] SELinux:  Initializing.
[    0.000606] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.000620] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.001726] rcu: Hierarchical SRCU implementation.
[    0.003953] Tegra Revision: A01 SKU: 212 CPU Process: 0 SoC Process: 0
[    0.004219] Remapping and enabling EFI services.
[    0.005020] smp: Bringing up secondary CPUs ...
[    0.005505] Detected PIPT I-cache on CPU1
[    0.005546] GICv3: CPU1: found redistributor 100 region 0:0x000000000f460000
[    0.005579] CPU1: Booted secondary processor 0x0000000100 [0x410fd421]
[    0.006029] Detected PIPT I-cache on CPU2
[    0.006043] GICv3: CPU2: found redistributor 200 region 0:0x000000000f480000
[    0.006058] CPU2: Booted secondary processor 0x0000000200 [0x410fd421]
[    0.006472] Detected PIPT I-cache on CPU3
[    0.006481] GICv3: CPU3: found redistributor 300 region 0:0x000000000f4a0000
[    0.006495] CPU3: Booted secondary processor 0x0000000300 [0x410fd421]
[    0.008933] Detected PIPT I-cache on CPU4
[    0.008959] GICv3: CPU4: found redistributor 10200 region 0:0x000000000f500000
[    0.008984] CPU4: Booted secondary processor 0x0000010200 [0x410fd421]
[    0.009452] Detected PIPT I-cache on CPU5
[    0.009463] GICv3: CPU5: found redistributor 10300 region 0:0x000000000f520000
[    0.009476] CPU5: Booted secondary processor 0x0000010300 [0x410fd421]
[    0.009530] smp: Brought up 1 node, 6 CPUs
[    0.009534] SMP: Total of 6 processors activated.
[    0.009537] CPU features: detected: 32-bit EL0 Support
[    0.009538] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.009540] CPU features: detected: Common not Private translations
[    0.009541] CPU features: detected: CRC32 instructll CPU(s) started at EL2
[    0.039987] devtmpfs: initialized
[    0.059553] KASLR enabled
[    0.059746] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.059775] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.064843] pinctrl core: initialized pinctrl subsystem
[    0.065852] SMBIOS 3.6.0 present.
[    0.065867] DMI: NVIDIA NVIDIA Jetson Orin NX Engineering Reference Developer Kit/Jetson, BIOS 36.3.0-gcid-36106755 04/25/2024
[    0.066416] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.068127] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.068252] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.068339] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.068455] audit: initializing netlink subsys (disabled)
[    0.068602] audit: type=2000 audit(0.068:1): state=initialized audit_enabled=0 res=1
[    0.070443] thermal_sys: Registered thermal governor 'step_wise'
[    0.070448] thermal_sys: Registered thermal governor 'user_space'
[    0.070450] thermal_sys: Registered thermal governor 'power_allocator'
[    0.074009] cpuidle: using governor menu
[    0.074295] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.074402] ASID allocator initialised with 32768 entries
[    0.076113] Serial: AMBA PL011 UART driver
[    0.086660] 31d0000.serial: ttyAMA0 at MMIO 0x31d0000 (irq = 117, base_baud = 0) is a SBSA
[    0.097512] platform bpmp: Fixing up cyclic dependency with 2c60000.external-memory-controller
[    0.106458] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.106468] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.106469] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.106470] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.107300] cryptd: max_cpu_qlen set to 1000
[    0.108937] ACPI: Interpreter disabled.
[    0.110183] VDD_1V8_AO: supplied by VDD_5V0_SYS
[    0.110352] VDD_3V3_AO: supplied by VDD_5V0_SYS
[    0.110752] VDD_AV10_HUB: supplied by VDD_5V0_SYS
[    0.111126] iommu: Default domain type: Translated
[    0.111133] iommu: DMA domain TLB invalidation policy: strict mode
[    0.111448] SCSI subsystem initialized
[    0.111704] vgaarb: loaded
[    0.111940] usbcore: registered new interface driver usbfs
[    0.111969] usbcore: registered new interface driver hub
[    0.111984] usbcore: registered new device driver usb
[    0.112381] pps_core: LinuxPPS API ver. 1 registered
[    0.112385] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.112394] PTP clock support registered
[    0.112473] EDAC MC: Ver: 3.0.0
[    0.113758] Registered efivars operations
[    0.114423] FPGA manager framework
[    0.114505] Advanced Linux Sound Architecture Driver Initialized.
[    0.115486] clocksource: Switched to clocksource arch_sys_counter
[    0.149245] VFS: Disk quotas dquot_6.6.0
[    0.149316] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.149549] pnp: PnP ACPI: disabled
[    0.152562] NET: Registered PF_INET protocol family
[    0.152893] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.154291] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)
[    0.154312] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.154328] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.154432] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    0.155821] TCP: Hash tables configured (established 65536 bind 65536)
[    0.156020] UDP hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.156048] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.156257smmu 8000000.iommu:        stream matching with 128 register groups
[    0.209807] arm-smmu 8000000.iommu:  128 context banks (0 stage-2 only)
[    0.209823] arm-smmu 8000000.iommu:  Supported page sizes: 0x61311000
[    0.209825] arm-smmu 8000000.iommu:  Stage-1: 48-bit VA -> 48-bit IPA
[    0.209826] arm-smmu 8000000.iommu:  Stage-2: 48-bit IPA -> 48-bit PA
[    0.210824] arm-smmu 10000000.iommu: probing hardware configuration...
[    0.210826] arm-smmu 10000000.iommu: SMMUv2 with:
[    0.210828] arm-smmu 10000000.iommu:         stage 1 translation
[    0.210829] arm-smmu 10000000.iommu:         stage 2 translation
[    0.210829] arm-smmu 10000000.iommu:         nested translation
[    0.210832] arm-smmu 10000000.iommu:         stream matching with 128 register groups
[    0.210834] arm-smmu 10000000.iommu:         128 context banks (0 stage-2 only)
[    0.210840] arm-smmu 10000000.iommu:         Supported page sizes: 0x61311000
[    0.210842] arm-smmu 10000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    0.210843] arm-smmu 10000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    0.211190] arm-smmu 12000000.iommu: probing hardware configuration...
[    0.211194] arm-smmu 12000000.iommu: SMMUv2 with:
[    0.211195] arm-smmu 12000000.iommu:         stage 1 translation
[    0.211196] arm-smmu 12000000.iommu:         stage 2 translation
[    0.211197] arm-smmu 12000000.iommu:         nested translation
[    0.211199] arm-smmu 12000000.iommu:         stream matching with 128 register groups
[    0.211201] arm-smmu 12000000.iommu:         128 context banks (0 stage-2 only)
[    0.211206] arm-smmu 12000000.iommu:         Supported page sizes: 0x61311000
[    0.211207] arm-smmu 12000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    0.211208] arm-smmu 12000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    0.218688] loop: module loaded
[    0.219549] megasas: 07.717.02.00-rc1
[    0.223549] tun: Universal TUN/TAP device driver, 1.6
[    0.224094] thunder_xcv, ver 1.0
[    0.224118] thunder_bgx, ver 1.0
[    0.224143] nicpf, ver 1.0
[    0.224936] hclge is initializing
[    0.224969] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    0.224975] hns3: Copyright (c) 2017 Huawei Corporation.
[    0.225025] e1000: Intel(R) PRO/1000 Network Driver
[    0.225033] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    0.225061] e1000e: Intel(R) PRO/1000 Network Driver
[    0.225066] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.225088] igb: Intel(R) Gigabit Ethernet Network Driver
[    0.225093] igb: Copyright (c) 2007-2014 Intel Corporation.
[    0.225106] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    0.225112] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    0.225299] sky2: driver version 1.30
[    0.225843] PPP generic driver version 2.4.2
[    0.226028] VFIO - User Level meta-driver version: 0.3
[    0.227556] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.227566] ehci-pci: EHCI PCI platform driver
[    0.227596] ehci-platform: EHCI generic platform driver
[    0.227668] ehci-orion: EHCI orion driver
[    0.227731] ehci-exynos: EHCI Exynos driver
[    0.227782] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.227811] ohci-pci: OHCI PCI platform driver
[    0.227834] ohci-platform: OHCI generic platform driver
[    0.227907] ohci-exynos: OHCI Exynos driver
[    0.228323] usbcore: registered new interface driver usb-storage
[    0.230020] i2c_dev: i2c /dev entries driver
[    0.231218] pps pps0: new PPS source ktimer
[    0.231229] pps pps0: ktimer PPS source registered
[    0.231237] pps_ldisc: PPS line discipline registered
[    0.233264] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[    0.235274] sdhci: Secure Digital Host Controller Interface driver
[    0.235279] sdhci: Copyright(c) Pierre Ossman
[    0.235713] Synopsys Designware Multimedia Card Interface Driver
[    0.236310] sdhci-pltfm: SDHCI platform and ware timestamp: 2023-02-10 03:48:10 UTC
[    3.544489] tegra-xusb 3610000.usb: xHCI Host Controller
[    3.544499] tegra-xusb 3610000.usb: new USB bus registered, assigned bus number 1
[    3.545228] tegra-xusb 3610000.usb: hcc params 0x0180ff05 hci version 0x120 quirks 0x0000000000010810
[    3.545250] tegra-xusb 3610000.usb: irq 123, io mem 0x03610000
[    3.545362] tegra-xusb 3610000.usb: xHCI Host Controller
[    3.545366] tegra-xusb 3610000.usb: new USB bus registered, assigned bus number 2
[    3.545371] tegra-xusb 3610000.usb: Host supports USB 3.1 Enhanced SuperSpeed
[    3.545785] hub 1-0:1.0: USB hub found
[    3.545799] hub 1-0:1.0: 4 ports detected
[    3.546042] hub 2-0:1.0: USB hub found
[    3.546055] hub 2-0:1.0: 4 ports detected
[    3.548127] irq: IRQ224: trimming hierarchy from :bus@0:pmc@c360000
[    3.548183] irq: IRQ225: trimming hierarchy from :bus@0:interrupt-controller@f400000-1
[    3.548223] irq: IRQ226: trimming hierarchy from :bus@0:pmc@c360000
[    3.548282] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    3.586607] ALSA device list:
[    3.586611]   No soundcards found.
[    3.588953] Freeing unused kernel memory: 7616K
[    3.589023] Run /init as init process
[    3.604784] Root device found: initrd
[    3.617195] r8168: loading out-of-tree module taints kernel.
[    3.617499] r8168: module verification failed: signature and/or required key missing - tainting kernel
[    3.617520] r8168: disagrees about version of symbol pci_save_state
[    3.617521] r8168: Unknown symbol pci_save_state (err -22)
[    3.617549] r8168: disagrees about version of symbol pci_wake_from_d3
[    3.617550] r8168: Unknown symbol pci_wake_from_d3 (err -22)
[    3.617552] r8168: disagrees about version of symbol pci_enable_device
[    3.617553] r8168: Unknown symbol pci_enable_device (err -22)
[    3.617570] r8168: disagrees about version of symbol pci_choose_state
[    3.617571] r8168: Unknown symbol pci_choose_state (err -22)
[    3.617580] r8168: disagrees about version of symbol __pci_register_driver
[    3.617581] r8168: Unknown symbol __pci_register_driver (err -22)
[    3.617587] r8168: disagrees about version of symbol pci_disable_msi
[    3.617587] r8168: Unknown symbol pci_disable_msi (err -22)
[    3.617589] r8168: disagrees about version of symbol pci_request_regions
[    3.617590] r8168: Unknown symbol pci_request_regions (err -22)
[    3.617596] r8168: disagrees about version of symbol pci_clear_mwi
[    3.617597] r8168: Unknown symbol pci_clear_mwi (err -22)
[    3.617605] r8168: disagrees about version of symbol pci_set_power_state
[    3.617606] r8168: Unknown symbol pci_set_power_state (err -22)
[    3.617613] r8168: disagrees about version of symbol pci_unregister_driver
[    3.617614] r8168: Unknown symbol pci_unregister_driver (err -22)
[    3.617626] r8168: disagrees about version of symbol pci_read_config_dword
[    3.617626] r8168: Unknown symbol pci_read_config_dword (err -22)
[    3.617641] r8168: disagrees about version of symbol pci_enable_msix_range
[    3.617642] r8168: Unknown symbol pci_enable_msix_range (err -22)
[    3.617645] r8168: disagrees about version of symbol pci_disable_link_state
[    3.617646] r8168: Unknown symbol pci_disable_link_state (err -22)
[    3.617653] r8168: disagrees about version of symbol pci_set_mwi
[    3.617654] r8168: Unknown symbol pci_set_mwi (err -22)
[    3.617656] r8168: disagrees about version of symbol pci_find_capability
[    3.617656] r8168: Unknown symbol pci_find_capability (err -22)
[    3.617658] r8168: disagrees about version of symbol pci_clear_master
[    3.617659] r8168: Unknown symbol pci_clear_master (err -22)
[    3.617660] r8168: disagrees about version of symbol pci_enable_msi
[    3.617661] r8168: Unknown symbol pci_enable_msi (err -22)
[    3.617677] r8168: disagrees about version of symbol pci_read_config_word
[    3.617677] r8168l pci_write_config_byte
[    3.617709] r8168: Unknown symbol pci_write_config_byte (err -22)
[    3.617715] r8168: disagrees about version of symbol pci_disable_msix
[    3.617716] r8168: Unknown symbol pci_disable_msix (err -22)
[    3.617717] r8168: disagrees about version of symbol pci_disable_device
[    3.617718] r8168: Unknown symbol pci_disable_device (err -22)
[    3.617720] r8168: disagrees about version of symbol pci_get_slot
[    3.617721] r8168: Unknown symbol pci_get_slot (err -22)
[    3.617726] r8168: disagrees about version of symbol pci_read_config_byte
[    3.617727] r8168: Unknown symbol pci_read_config_byte (err -22)
[    3.617732] r8168: disagrees about version of symbol pci_write_config_word
[    3.617733] r8168: Unknown symbol pci_write_config_word (err -22)
[    3.617738] r8168: disagrees about version of symbol pci_write_config_dword
[    3.617738] r8168: Unknown symbol pci_write_config_dword (err -22)
[    3.666833] Mount initrd as rootfs and enter recovery mode
[    3.879666] usb 1-3: new high-speed USB device number 2 using tegra-xusb
[    4.031701] hub 1-3:1.0: USB hub found
[    4.032146] hub 1-3:1.0: 4 ports detected
insmod /lib/modules/5.15.136-tegra/kernel/drivers/phy/tegra/phy-tegra194-p2u.ko
insmod /lib/modules/5.15.136-tegra/kernel/drivers/pci/controller/dwc/pcie-tegra194.ko
[   13.690392] tegra194-pcie 140a0000.pcie: Adding to iommu group 3
[   13.801345] tegra194-pcie 140a0000.pcie: host bridge /bus@0/pcie@140a0000 ranges:
[   13.801366] tegra194-pcie 140a0000.pcie:      MEM 0x3240000000..0x3527ffffff -> 0x3240000000
[   13.801374] tegra194-pcie 140a0000.pcie:      MEM 0x3528000000..0x352fffffff -> 0x0040000000
[   13.801377] tegra194-pcie 140a0000.pcie:       IO 0x002a100000..0x002a1fffff -> 0x002a100000
[   13.801866] tegra194-pcie 140a0000.pcie: iATU unroll: enabled
[   13.801869] tegra194-pcie 140a0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   13.907651] tegra194-pcie 140a0000.pcie: Link up
[   13.910376] tegra194-pcie 140a0000.pcie: Link up
[   13.910428] tegra194-pcie 140a0000.pcie: PCI host bridge to bus 0008:00
[   13.910432] pci_bus 0008:00: root bus resource [io  0x0000-0xfffff] (bus address [0x2a100000-0x2a1fffff])
[   13.910435] pci_bus 0008:00: root bus resource [mem 0x3528000000-0x352fffffff] (bus address [0x40000000-0x47ffffff])
[   13.910438] pci_bus 0008:00: root bus resource [bus 00-ff]
[   13.910440] pci_bus 0008:00: root bus resource [mem 0x3240000000-0x3527ffffff pref]
[   13.910487] pci 0008:00:00.0: [10de:229c] type 01 class 0x060400
[   13.910647] pci 0008:00:00.0: PME# supported from D0 D3hot
[   13.913905] pci 0008:01:00.0: [10ec:8168] type 00 class 0x020000
[   13.914062] pci 0008:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid
[   13.914064] pci 0008:01:00.0: reg 0x10: [io  size 0x0100]
[   13.914245] pci 0008:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[   13.914361] pci 0008:01:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
[   13.915340] pci 0008:01:00.0: supports D1 D2
[   13.915341] pci 0008:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[   13.930106] pci 0008:00:00.0: BAR 8: assigned [mem 0x3528000000-0x35280fffff]
[   13.930110] pci 0008:00:00.0: BAR 7: assigned [io  0x1000-0x1fff]
[   13.930113] pci 0008:01:00.0: BAR 4: assigned [mem 0x3528000000-0x3528003fff 64bit]
[   13.930205] pci 0008:01:00.0: BAR 2: assigned [mem 0x3528004000-0x3528004fff 64bit]
[   13.930297] pci 0008:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[   13.930326] pci 0008:00:00.0: PCI bridge to [bus 01-ff]
[   13.930329] pci 0008:00:00.0:   bridge window [io  0x1000-0x1fff]
[   13.930332] pci 0008:00:00.0:   bridge window [mem 0x3528000000-0x35280fffff]
[   13.930419] pcieport 0008:00:00.0: Adding to iommu group 3
[   13.930502] pcieport 0008:00:00.0: PME: Signaling with IRQ 189
[   13.930800] pcieport 0008:00:00.0: AER: enabled with IRQ 189
[   13.931077] tegra194-pcie 140c0000.pcie: Adding to iommu group 4
[   13.933561] tegra194-pcie 140c0000.pcie: host bridge /bus@0/pcie@140c0000 ranges:
[   13.933579] tegra194-pcie 140c0000.pcie:      MEM 0x3540000000..0x37ffffffff -> 0x3540000000
[   13.933585] tegra194-pcie 140c0000.pcie:      MEM 0x3828000000..0x382fffffff -> 0x0040000000
[   13.933588] tegra194-pcie 140c0000.pcie:       IO 0x002c100000..0x002c1fffff -> 0x002c100000
[   13.934037] tegra194-pcie 140c0000.pcie: iATU unroll: enabled
[   13.934040] tegra194-pcie 140c0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   15.036716] tegra194-pcie 140c0000.pcie: Phy link never came up
[   16.038492] tegra194-pcie 140c0000.pcie: Phy link never came up
[   16.038544] tegra194-pcie 140c0000.pcie: PCI host bridge to bus 0009:00
[   16.038548] pci_bus 0009:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x2c100000-0x2c1fffff])
[   16.038551] pci_bus 0009:00: root bus resource [mem 0x3828000000-0x382fffffff] (bus address [0x40000000-0x47ffffff])
[   16.038553] pci_bus 0009:00: root bus resource [bus 00-ff]
[   16.038555] pci_bus 0009:00: root bus resource [mem 0x3540000000-0x37ffffffff pref]
[   16.038598] pci 0009:00:00.0: [10de:229c] type 01 class 0x060400
[   16.038745] pci 0009:00:00.0: PME# supported from D0 D3hot
[   16.043915] pci 0009:00:00.0: PCI bridge to [bus 01-ff]
[   16.044000] pcieport 0009:00:00.0: Adding to iommu group 4
[   16.044081] pcieport 0009:00:00.0: PME: Signaling with IRQ 191
[   16.044613] pcieport 0009:00:00.0: AER: enabled with IRQ 191
[   16.045064] pci_bus 0009:01: busn_res: [bus 01-ff] is released
[   16.045138] pci 0009:00:00.0: Removing from iommu group 4
[   16.045147] pci_bus 0009:00: busn_res: [bus 00-ff] is released
[   16.046032] tegra194-pcie 14100000.pcie: Adding to iommu group 5
[   16.047488] tegra194-pcie 14100000.pcie: host bridge /bus@0/pcie@14100000 ranges:
[   16.047499] tegra194-pcie 14100000.pcie:      MEM 0x2080000000..0x20a7ffffff -> 0x2080000000
[   16.047504] tegra194-pcie 14100000.pcie:      MEM 0x20a8000000..0x20afffffff -> 0x0040000000
[   16.047507] tegra194-pcie 14100000.pcie:       IO 0x0030100000..0x00301fffff -> 0x0030100000
[   16.047786] tegra194-pcie 14100000.pcie: iATU unroll: enabled
[   16.047788] tegra194-pcie 14100000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   17.157421] tegra194-pcie 14100000.pcie: Phy link never came up
[   18.159203] tegra194-pcie 14100000.pcie: Phy link never came up
[   18.159244] tegra194-pcie 14100000.pcie: PCI host bridge to bus 0001:00
[   18.159247] pci_bus 0001:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x30100000-0x301fffff])
[   18.159250] pci_bus 0001:00: root bus resource [mem 0x20a8000000-0x20afffffff] (bus address [0x40000000-0x47ffffff])
[   18.159253] pci_bus 0001:00: root bus resource [bus 00-ff]
[   18.159254] pci_bus 0001:00: root bus resource [mem 0x2080000000-0x20a7ffffff pref]
[   18.159298] pci 0001:00:00.0: [10de:229e] type 01 class 0x060400
[   18.159459] pci 0001:00:00.0: PME# supported from D0 D3hot
[   18.164637] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[   18.164717] pcieport 0001:00:00.0: Adding to iommu group 5
[   18.164794] pcieport 0001:00:00.0: PME: Signaling with IRQ 193
[   18.165317] pcieport 0001:00:00.0: AER: enabled with IRQ 193
[   18.165772] pci_bus 0001:01: busn_res: [bus 01-ff] is released
[   18.165837] pci 0001:00:00.0: Removing from iommu group 5
[   18.165845] pci_bus 0001:00: busn_res: [bus 00-ff] is released
[   18.166912] tegra194-pcie 14160000.pcie: Adding to iommu group 6
[   18.168358] tegra194-pcie 14160000.pcie: host bridge /bus@0/pcie@14160000 ranges:
[   18.168369] tegra194-pcie 14160000.pcie:      MEM 0x2140000000..0x2427ffffff -> 0x2140000000
[   18.168375] tegra194-pcie 14160000.pcie:      MEM 0x2428000000..0x242fffffff -> 0x0040000000
[   18.168378] tegra194-pcie 14160000.pcie:       IO 0x0036100000..0x00361fffff -> 0x0036100000
[   18.168674] tegra194-pcie 14160000.pcie: iATU unroll: enabled
[   18.168676] tegra194-pcie 14160000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   18.275651] tegra194-pcie 14160000.pcie: Link up
[   18.278250] tegra194-pcie 14160000.pcie: Link up
[   18.278289] tegra194-pcie 14160000.pcie: PCI host bridge to bus 0004:00
[   18.278292] pci_bus 0004:00: root bus resource [io  0x300000-0x3fffff] (bus address [0x36100000-0x361fffff])
[   18.278294] pci_bus 0004:00: root bus resource [mem 0x2428000000-0x242fffffff] (bus address [0x40000000-0x47ffffff])
[   18.278296] pci_bus 0004:00: root bus resource [bus 00-ff]
[   18.278297] pci_bus 0004:00: root bus resource [mem 0x2140000000-0x2427ffffff pref]
[   18.278336] pci 0004:00:00.0: [10de:229c] type 01 class 0x060400
[   18.278481] pci 0004:00:00.0: PME# supported from D0 D3hot
[   18.281387] pci 0004:01:00.0: [144d:a809] type 00 class 0x010802
[   18.281579] pci 0004:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
[   18.294073] pci 0004:00:00.0: BAR 8: assigned [mem 0x2428000000-0x24280fffff]
[   18.294077] pci 0004:01:00.0: BAR 0: assigned [mem 0x2428000000-0x2428003fff 64bit]
[   18.294162] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[   18.294166] pci 0004:00:00.0:   bridge window [mem 0x2428000000-0x24280fffff]
[   18.294237] pcieport 0004:00:00.0: Adding to iommu group 6
[   18.294306] pcieport 0004:00:00.0: PME: Signaling with IRQ 195
[   18.294594] pcieport 0004:00:00.0: AER: enabled with IRQ 195
[   18.294806] tegra194-pcie 141e0000.pcie: Adding to iommu group 7
[   18.296853] tegra194-pcie 141e0000.pcie: host bridge /bus@0/pcie@141e0000 ranges:
[   18.296870] tegra194-pcie 141e0000.pcie:      MEM 0x3000000000..0x3227ffffff -> 0x3000000000
[   18.296875] tegra194-pcie 141e0000.pcie:      MEM 0x3228000000..0x322fffffff -> 0x0040000000
[   18.296878] tegra194-pcie 141e0000.pcie:       IO 0x003e100000..0x003e1fffff -> 0x003e100000
[   18.297320] tegra194-pcie 141e0000.pcie: iATU unroll: enabled
[   18.297323] tegra194-pcie 141e0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   19.405422] tegra194-pcie 141e0000.pcie: Phy link never came up
[   20.407207] tegra194-pcie 141e0000.pcie: Phy link never came up
[   20.407260] tegra194-pcie 141e0000.pcie: PCI host bridge to bus 0007:00
[   20.407263] pci_bus 0007:00: root bus resource [io  0x400000-0x4fffff] (bus address [0x3e100000-0x3e1fffff])
[   20.407265] pci_bus 0007:00: root bus resource [mem 0x3228000000-0x322fffffff] (bus address [0x40000000-0x47ffffff])
[   20.407268] pci_bus 0007:00: root bus resource [bus 00-ff]
[   20.407270] pci_bus 0007:00: root bus resource [mem 0x3000000000-0x3227ffffff pref]
[   20.407308] pci 0007:00:00.0: [10de:229a] type 01 class 0x060400
[   20.407450] pci 0007:00:00.0: PME# supported from D0 D3hot
[   20.412572] pci 0007:00:00.0: PCI bridge to [bus 01-ff]
[   20.412648] pcieport 0007:00:00.0: Adding to iommu group 7
[   20.412719] pcieport 0007:00:00.0: PME: Signaling with IRQ 197
[   20.413253] pcieport 0007:00:00.0: AER: enabled with IRQ 197
[   20.413692] pci_bus 0007:01: busn_res: [bus 01-ff] is released
[   20.413756] pci 0007:00:00.0: Removing from iommu group 7
modules/5.15.136-tegra/kernel/drivers/nvme/host/nvme-core.ko s eased
insmod /lib/modules/5.15.136-tegra/kernel/drivers/nvme/host/nvme.ko
[   20.428506] nvme 0004:01:00.0: Adding to iommu group 6
[   20.428687] nvme nvme0: pci function 0004:01:00.0
[   20.428912] nvme 0004:01:00.0: enabling device (0000 -> 0002)
[   20.433568] nvme nvme0: Shutdown timeout set to 8 seconds
[   20.450132] nvme nvme0: allocated 64 MiB host memory buffer.
[   23.238177] nvme nvme0: 6/0/0 default/read/poll queues
[   24.522682]  nvme0n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15
insmod /lib/modules/5.15.136-tegra/kernel/drivers/thermal/tegra/tegra-bpmp-thermal.ko
insmod /lib/modules/5.15.136-tegra/kernel/drivers/pwm/pwm-tegra.ko
insmod /lib/modules/5.15.136-tegra/kernel/drivers/hwmon/pwm-fan.ko
Checking whether device /dev/mmcblk?p1 exist
Device /dev/mmcblk?p1 does not exist
Checking whether device /dev/sd?1 exist
[   24.549445] Finding OTA work dir on external storage devices
Device /dev/sd?1 does not exist
Checking whether device /dev/nvme?n1p1 exist
Looking for OTA work directory on the device(s): /dev/nvme0n1p1
[   24.556711] mount /dev/nvme0n1p1 /mnt
[   24.572539] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[   24.574101] is_boot_only_partition /mnt
[   24.582372] OTA work directory /mnt/ota_work is not found on /dev/nvme0n1p1
[   24.587392] Finding OTA work dir on internal storage device
mount: /mnt: special device /dev/mmcblk0 does not exist.
[   24.588540] mount /dev/mmcblk0 /mnt
[   24.591033] Failed to mount /dev/mmcblk0 on the /mnt
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
bash-5.1# [   24.592254] Failed to run "mount_ota_work_partition /dev/mmcblk0 /mnt"
[   24.593367] OTA work directory is not found on internal and external storage devices

bash-5.1# lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
nvme0n1      259:0    0 238.5G  0 disk
|-nvme0n1p1  259:1    0   237G  0 part
|-nvme0n1p2  259:2    0   128M  0 part
|-nvme0n1p3  259:3    0   768K  0 part
|-nvme0n1p4  259:4    0  31.6M  0 part
|-nvme0n1p5  259:5    0   128M  0 part
|-nvme0n1p6  259:6    0   768K  0 part
|-nvme0n1p7  259:7    0  31.6M  0 part
|-nvme0n1p8  259:8    0    80M  0 part
|-nvme0n1p9  259:9    0   512K  0 part
|-nvme0n1p10 259:10   0    64M  0 part
|-nvme0n1p11 259:11   0    80M  0 part
|-nvme0n1p12 259:12   0   512K  0 part
|-nvme0n1p13 259:13   0    64M  0 part
|-nvme0n1p14 259:14   0   400M  0 part
`-nvme0n1p15 259:15   0 479.5M  0 part

In this kernel log. I don’t know why the driver of r8168 is still output signature failure. I am sure I replace it correctly.

Do you have any idea about this?

[WayneWWW]

This looks like recovery image log. Please dump your full log (with bootloader) and it will clarify.

[bbear953308023]

You are right. The log before kernel log shows it attempt to bootup in recovery:

Jetson System firmware version 36.3.0-gcid-36106755 date 2024-04-25T02:38:25+00:
00
ESC   to enter Setup.
F11   to enter Boot Manager Menu.
Enter to continue boot.
......
      L4TLauncher: Attempting Recovery Boot
EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services...
▒▒debugfs initialized
▒▒I/TC: Reserved shared memory is disabled
I/TC: Dynamic shared memory is enabled
I/TC: Normal World virtualization support is disabled
I/TC: Asynchronous notifications are disabled
▒▒[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd421]
[    0.000000] Linux version 5.15.136-tegra (root@stu) (aarch64-buildroot-linux-gnu-gcc.br_real (Buildroot 2022.08) 11.3.0, GNU ld (GNU Binutils) 2.38) #1 SMP PREEMPT Tue Sep 3 11:19:33 UTC 2024 ()
[    0.000000] Machine model: NVIDIA Jetson Orin NX Engineering Reference Developer Kit
[    0.000000] efi: EFI v2.70 by EDK II
[    0.000000] efi: RTPROP=0x26d83f198 SMBIOS=0xffff0000 SMBIOS 3.0=0x26d210000 MEMATTR=0x267790018 ESRT=0x267209318 RNG=0x25a920018 MEMRESERVE=0x25b9acf18
[    0.000000] random: crng init done
[    0.000000] secureboot: Secure boot disabled
[    0.000000] esrt: Reserving ESRT space from 0x0000000267209318 to 0x0000000267209350.
[    0.000000] Reserved memory: created CMA memory pool at 0x000000024a000000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000080000000-0x0000000277ffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x2712f9800-0x2712fbfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000000277ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000bdffffff]
[    0.000000]   node   0: [mem 0x00000000c2000000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000025e24cfff]
[    0.000000]   node   0: [mem 0x000000025e24d000-0x000000025e41bfff]
[    0.000000]   node   0: [mem 0x000000025e41c000-0x000000026b8fffff]
[    0.000000]   node   0: [mem 0x000000026b900000-0x000000026d83ffff]
[    0.000000]   node   0: [mem 0x000000026d840000-0x0000000271dfffff]
[    0.000000]   node   0: [mem 0x0000000271e00000-0x0000000271ffffff]
[    0.000000]   node   0: [mem 0x0000000272000000-0x000000027259ffff]
[    0.000000]   node   0: [mem 0x0000000272f00000-0x0000000272ffffff]
[    0.000000]   node   0: [mem 0x0000000276000000-0x0000000277ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000277ffffff]
[    0.000000] On node 0, zone DMA: 16384 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 2400 pages in unavailable ranges
[    0.000000] On node 0, zone Normal: 12288 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 29 pages/cinsmod /l408 r8module d3.154 u118784egra/u  0.00/dr0] rs/net/eted PIPnet-caaltek/r8168. on CP

[    0.000000] CPU features: detected: Address authentication (architected algorithm)
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: detected: Hardware dirty bit management
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU featuresmodprobe:: dcteRORKer: c page d not insert 'rion (K In)
valid .00000nt0]
alternatives: patching kernel code
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 2001056
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/initrd rw rootwait mminit_loglevel=4 console=ttyTCU0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 bl_prof_dataptr=2031616@0x271E10000 bl_prof_ro_ptr=65536@0x271E00000
[    0.000000] Unknown kernel command line parameters "bl_prof_dataptr=2031616@0x271E10000 bl_prof_ro_ptr=65536@0x271E00000", will be passed to user space.
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000fbfe0000-0x00000000fffe0000] (64MB)
[    0.000000] Memory: 7517248K/8133248K available (19072K kernel code, 4060K rwdata, 10032K rodata, 7616K init, 529K bss, 353856K reserved, 262144K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=6.
[    0.000000]  Trampoline variant of Tasks RCU enabled.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=6
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 960 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x000000000f440000
[    0.000000] arch_timer: cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000001] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.000374] Console: colour dummy device 80x25
[    0.000440] Calibrating delay loop (skipped), value calculated using timer frequency.. 62.50 BogoMIPS (lpj=125000)
[    0.000448] pid_max: default: 32768 minimum: 301
[    0.000487] LSM: Security Framework initializing
[    0.000507] Yama: becoming mindful.
[    0.000522] SELinux:  Initializing.
[    0.000606] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.000620] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.001726] rcu: Hierarchical SRCU implementation.
[    0.003953] Tegra Revision: A01 SKU: 212 CPU Process: 0 SoC Process: 0
[    0.004219] Remapping and enabling EFI services.
[    0.005020] smp: Bringing up secondary CPUs ...
[    0.005505] Detected PIPT I-cache on CPU1
[    0.005546] GICv3: CPU1: found redistributor 100 region 0:0x000000000f460000
[    0.005579] CPU1: Booted secondary processor 0x0000000100 [0x410fd421]
[    0.006029] Detected PIPT I-cache on CPU2
[    0.006043] GICv3: CPU2: found redistributor 200 region 0:0x000000000f480000
[    0.006058] CPU2: Booted secondary processor 0x0000000200 [0x410fd421]
[    0.006472] Detected PIPT I-cache on CPU3
[    0.006481] GICv3: CPU3: found redistributor 300 region 0:0x000000000f4a0000
[    0.006495] CPU3: Booted secondary processor 0x0000000300 [0x410fd421]
[    0.008933] Detected PIPT I-cache on CPU4
[    0.008959] GICv3: CPU4: found redistributor 10200 region 0:0x000000000f500000
[    0.008984] CPU4: Booted secondary processor 0x0000010200 [0x410fd421]
[    0.009452] Detected PIPT I-cache on CPU5
[    0.009463] GICv3: CPU5: found redistributor 10300 region 0:0x000000000f520000
[    0.009476] CPU5: Booted secondary processor 0x0000010300 [0x410fd421]
[    0.009530] smp: Brought up 1 node, 6 CPUs
[    0.009534] SMP: Total of 6 processors activated.
[    0.009537] CPU features: detected: 32-bit EL0 Support
[    0.009538] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.009540] CPU features: detected: Common not Private translations
[    0.009541] CPU features: detected: CRC32 instructll CPU(s) started at EL2
[    0.039987] devtmpfs: initialized
[    0.059553] KASLR enabled
[    0.059746] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.059775] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.064843] pinctrl core: initialized pinctrl subsystem
[    0.065852] SMBIOS 3.6.0 present.
[    0.065867] DMI: NVIDIA NVIDIA Jetson Orin NX Engineering Reference Developer Kit/Jetson, BIOS 36.3.0-gcid-36106755 04/25/2024
[    0.066416] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.068127] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations
[    0.068252] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.068339] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.068455] audit: initializing netlink subsys (disabled)
[    0.068602] audit: type=2000 audit(0.068:1): state=initialized audit_enabled=0 res=1
[    0.070443] thermal_sys: Registered thermal governor 'step_wise'
[    0.070448] thermal_sys: Registered thermal governor 'user_space'
[    0.070450] thermal_sys: Registered thermal governor 'power_allocator'
[    0.074009] cpuidle: using governor menu
[    0.074295] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.074402] ASID allocator initialised with 32768 entries
[    0.076113] Serial: AMBA PL011 UART driver
[    0.086660] 31d0000.serial: ttyAMA0 at MMIO 0x31d0000 (irq = 117, base_baud = 0) is a SBSA
[    0.097512] platform bpmp: Fixing up cyclic dependency with 2c60000.external-memory-controller
[    0.106458] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.106468] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.106469] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.106470] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.107300] cryptd: max_cpu_qlen set to 1000
[    0.108937] ACPI: Interpreter disabled.
[    0.110183] VDD_1V8_AO: supplied by VDD_5V0_SYS
[    0.110352] VDD_3V3_AO: supplied by VDD_5V0_SYS
[    0.110752] VDD_AV10_HUB: supplied by VDD_5V0_SYS
[    0.111126] iommu: Default domain type: Translated
[    0.111133] iommu: DMA domain TLB invalidation policy: strict mode
[    0.111448] SCSI subsystem initialized
[    0.111704] vgaarb: loaded
[    0.111940] usbcore: registered new interface driver usbfs
[    0.111969] usbcore: registered new interface driver hub
[    0.111984] usbcore: registered new device driver usb
[    0.112381] pps_core: LinuxPPS API ver. 1 registered
[    0.112385] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.112394] PTP clock support registered
[    0.112473] EDAC MC: Ver: 3.0.0
[    0.113758] Registered efivars operations
[    0.114423] FPGA manager framework
[    0.114505] Advanced Linux Sound Architecture Driver Initialized.
[    0.115486] clocksource: Switched to clocksource arch_sys_counter
[    0.149245] VFS: Disk quotas dquot_6.6.0
[    0.149316] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.149549] pnp: PnP ACPI: disabled
[    0.152562] NET: Registered PF_INET protocol family
[    0.152893] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.154291] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)
[    0.154312] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.154328] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.154432] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    0.155821] TCP: Hash tables configured (established 65536 bind 65536)
[    0.156020] UDP hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.156048] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.156257smmu 8000000.iommu:        stream matching with 128 register groups
[    0.209807] arm-smmu 8000000.iommu:  128 context banks (0 stage-2 only)
[    0.209823] arm-smmu 8000000.iommu:  Supported page sizes: 0x61311000
[    0.209825] arm-smmu 8000000.iommu:  Stage-1: 48-bit VA -> 48-bit IPA
[    0.209826] arm-smmu 8000000.iommu:  Stage-2: 48-bit IPA -> 48-bit PA
[    0.210824] arm-smmu 10000000.iommu: probing hardware configuration...
[    0.210826] arm-smmu 10000000.iommu: SMMUv2 with:
[    0.210828] arm-smmu 10000000.iommu:         stage 1 translation
[    0.210829] arm-smmu 10000000.iommu:         stage 2 translation
[    0.210829] arm-smmu 10000000.iommu:         nested translation
[    0.210832] arm-smmu 10000000.iommu:         stream matching with 128 register groups
[    0.210834] arm-smmu 10000000.iommu:         128 context banks (0 stage-2 only)
[    0.210840] arm-smmu 10000000.iommu:         Supported page sizes: 0x61311000
[    0.210842] arm-smmu 10000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    0.210843] arm-smmu 10000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    0.211190] arm-smmu 12000000.iommu: probing hardware configuration...
[    0.211194] arm-smmu 12000000.iommu: SMMUv2 with:
[    0.211195] arm-smmu 12000000.iommu:         stage 1 translation
[    0.211196] arm-smmu 12000000.iommu:         stage 2 translation
[    0.211197] arm-smmu 12000000.iommu:         nested translation
[    0.211199] arm-smmu 12000000.iommu:         stream matching with 128 register groups
[    0.211201] arm-smmu 12000000.iommu:         128 context banks (0 stage-2 only)
[    0.211206] arm-smmu 12000000.iommu:         Supported page sizes: 0x61311000
[    0.211207] arm-smmu 12000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    0.211208] arm-smmu 12000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    0.218688] loop: module loaded
[    0.219549] megasas: 07.717.02.00-rc1
[    0.223549] tun: Universal TUN/TAP device driver, 1.6
[    0.224094] thunder_xcv, ver 1.0
[    0.224118] thunder_bgx, ver 1.0
[    0.224143] nicpf, ver 1.0
[    0.224936] hclge is initializing
[    0.224969] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    0.224975] hns3: Copyright (c) 2017 Huawei Corporation.
[    0.225025] e1000: Intel(R) PRO/1000 Network Driver
[    0.225033] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    0.225061] e1000e: Intel(R) PRO/1000 Network Driver
[    0.225066] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.225088] igb: Intel(R) Gigabit Ethernet Network Driver
[    0.225093] igb: Copyright (c) 2007-2014 Intel Corporation.
[    0.225106] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    0.225112] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    0.225299] sky2: driver version 1.30
[    0.225843] PPP generic driver version 2.4.2
[    0.226028] VFIO - User Level meta-driver version: 0.3
[    0.227556] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.227566] ehci-pci: EHCI PCI platform driver
[    0.227596] ehci-platform: EHCI generic platform driver
[    0.227668] ehci-orion: EHCI orion driver
[    0.227731] ehci-exynos: EHCI Exynos driver
[    0.227782] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.227811] ohci-pci: OHCI PCI platform driver
[    0.227834] ohci-platform: OHCI generic platform driver
[    0.227907] ohci-exynos: OHCI Exynos driver
[    0.228323] usbcore: registered new interface driver usb-storage
[    0.230020] i2c_dev: i2c /dev entries driver
[    0.231218] pps pps0: new PPS source ktimer
[    0.231229] pps pps0: ktimer PPS source registered
[    0.231237] pps_ldisc: PPS line discipline registered
[    0.233264] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[    0.235274] sdhci: Secure Digital Host Controller Interface driver
[    0.235279] sdhci: Copyright(c) Pierre Ossman
[    0.235713] Synopsys Designware Multimedia Card Interface Driver
[    0.236310] sdhci-pltfm: SDHCI platform and ware timestamp: 2023-02-10 03:48:10 UTC
[    3.544489] tegra-xusb 3610000.usb: xHCI Host Controller
[    3.544499] tegra-xusb 3610000.usb: new USB bus registered, assigned bus number 1
[    3.545228] tegra-xusb 3610000.usb: hcc params 0x0180ff05 hci version 0x120 quirks 0x0000000000010810
[    3.545250] tegra-xusb 3610000.usb: irq 123, io mem 0x03610000
[    3.545362] tegra-xusb 3610000.usb: xHCI Host Controller
[    3.545366] tegra-xusb 3610000.usb: new USB bus registered, assigned bus number 2
[    3.545371] tegra-xusb 3610000.usb: Host supports USB 3.1 Enhanced SuperSpeed
[    3.545785] hub 1-0:1.0: USB hub found
[    3.545799] hub 1-0:1.0: 4 ports detected
[    3.546042] hub 2-0:1.0: USB hub found
[    3.546055] hub 2-0:1.0: 4 ports detected
[    3.548127] irq: IRQ224: trimming hierarchy from :bus@0:pmc@c360000
[    3.548183] irq: IRQ225: trimming hierarchy from :bus@0:interrupt-controller@f400000-1
[    3.548223] irq: IRQ226: trimming hierarchy from :bus@0:pmc@c360000
[    3.548282] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    3.586607] ALSA device list:
[    3.586611]   No soundcards found.
[    3.588953] Freeing unused kernel memory: 7616K
[    3.589023] Run /init as init process
[    3.604784] Root device found: initrd
[    3.617195] r8168: loading out-of-tree module taints kernel.
[    3.617499] r8168: module verification failed: signature and/or required key missing - tainting kernel
[    3.617520] r8168: disagrees about version of symbol pci_save_state
[    3.617521] r8168: Unknown symbol pci_save_state (err -22)
[    3.617549] r8168: disagrees about version of symbol pci_wake_from_d3
[    3.617550] r8168: Unknown symbol pci_wake_from_d3 (err -22)
[    3.617552] r8168: disagrees about version of symbol pci_enable_device
[    3.617553] r8168: Unknown symbol pci_enable_device (err -22)
[    3.617570] r8168: disagrees about version of symbol pci_choose_state
[    3.617571] r8168: Unknown symbol pci_choose_state (err -22)
[    3.617580] r8168: disagrees about version of symbol __pci_register_driver
[    3.617581] r8168: Unknown symbol __pci_register_driver (err -22)
[    3.617587] r8168: disagrees about version of symbol pci_disable_msi
[    3.617587] r8168: Unknown symbol pci_disable_msi (err -22)
[    3.617589] r8168: disagrees about version of symbol pci_request_regions
[    3.617590] r8168: Unknown symbol pci_request_regions (err -22)
[    3.617596] r8168: disagrees about version of symbol pci_clear_mwi
[    3.617597] r8168: Unknown symbol pci_clear_mwi (err -22)
[    3.617605] r8168: disagrees about version of symbol pci_set_power_state
[    3.617606] r8168: Unknown symbol pci_set_power_state (err -22)
[    3.617613] r8168: disagrees about version of symbol pci_unregister_driver
[    3.617614] r8168: Unknown symbol pci_unregister_driver (err -22)
[    3.617626] r8168: disagrees about version of symbol pci_read_config_dword
[    3.617626] r8168: Unknown symbol pci_read_config_dword (err -22)
[    3.617641] r8168: disagrees about version of symbol pci_enable_msix_range
[    3.617642] r8168: Unknown symbol pci_enable_msix_range (err -22)
[    3.617645] r8168: disagrees about version of symbol pci_disable_link_state
[    3.617646] r8168: Unknown symbol pci_disable_link_state (err -22)
[    3.617653] r8168: disagrees about version of symbol pci_set_mwi
[    3.617654] r8168: Unknown symbol pci_set_mwi (err -22)
[    3.617656] r8168: disagrees about version of symbol pci_find_capability
[    3.617656] r8168: Unknown symbol pci_find_capability (err -22)
[    3.617658] r8168: disagrees about version of symbol pci_clear_master
[    3.617659] r8168: Unknown symbol pci_clear_master (err -22)
[    3.617660] r8168: disagrees about version of symbol pci_enable_msi
[    3.617661] r8168: Unknown symbol pci_enable_msi (err -22)
[    3.617677] r8168: disagrees about version of symbol pci_read_config_word
[    3.617677] r8168l pci_write_config_byte
[    3.617709] r8168: Unknown symbol pci_write_config_byte (err -22)
[    3.617715] r8168: disagrees about version of symbol pci_disable_msix
[    3.617716] r8168: Unknown symbol pci_disable_msix (err -22)
[    3.617717] r8168: disagrees about version of symbol pci_disable_device
[    3.617718] r8168: Unknown symbol pci_disable_device (err -22)
[    3.617720] r8168: disagrees about version of symbol pci_get_slot
[    3.617721] r8168: Unknown symbol pci_get_slot (err -22)
[    3.617726] r8168: disagrees about version of symbol pci_read_config_byte
[    3.617727] r8168: Unknown symbol pci_read_config_byte (err -22)
[    3.617732] r8168: disagrees about version of symbol pci_write_config_word
[    3.617733] r8168: Unknown symbol pci_write_config_word (err -22)
[    3.617738] r8168: disagrees about version of symbol pci_write_config_dword
[    3.617738] r8168: Unknown symbol pci_write_config_dword (err -22)
[    3.666833] Mount initrd as rootfs and enter recovery mode
[    3.879666] usb 1-3: new high-speed USB device number 2 using tegra-xusb
[    4.031701] hub 1-3:1.0: USB hub found
[    4.032146] hub 1-3:1.0: 4 ports detected
insmod /lib/modules/5.15.136-tegra/kernel/drivers/phy/tegra/phy-tegra194-p2u.ko
insmod /lib/modules/5.15.136-tegra/kernel/drivers/pci/controller/dwc/pcie-tegra194.ko
[   13.690392] tegra194-pcie 140a0000.pcie: Adding to iommu group 3
[   13.801345] tegra194-pcie 140a0000.pcie: host bridge /bus@0/pcie@140a0000 ranges:
[   13.801366] tegra194-pcie 140a0000.pcie:      MEM 0x3240000000..0x3527ffffff -> 0x3240000000
[   13.801374] tegra194-pcie 140a0000.pcie:      MEM 0x3528000000..0x352fffffff -> 0x0040000000
[   13.801377] tegra194-pcie 140a0000.pcie:       IO 0x002a100000..0x002a1fffff -> 0x002a100000
[   13.801866] tegra194-pcie 140a0000.pcie: iATU unroll: enabled
[   13.801869] tegra194-pcie 140a0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   13.907651] tegra194-pcie 140a0000.pcie: Link up
[   13.910376] tegra194-pcie 140a0000.pcie: Link up
[   13.910428] tegra194-pcie 140a0000.pcie: PCI host bridge to bus 0008:00
[   13.910432] pci_bus 0008:00: root bus resource [io  0x0000-0xfffff] (bus address [0x2a100000-0x2a1fffff])
[   13.910435] pci_bus 0008:00: root bus resource [mem 0x3528000000-0x352fffffff] (bus address [0x40000000-0x47ffffff])
[   13.910438] pci_bus 0008:00: root bus resource [bus 00-ff]
[   13.910440] pci_bus 0008:00: root bus resource [mem 0x3240000000-0x3527ffffff pref]
[   13.910487] pci 0008:00:00.0: [10de:229c] type 01 class 0x060400
[   13.910647] pci 0008:00:00.0: PME# supported from D0 D3hot
[   13.913905] pci 0008:01:00.0: [10ec:8168] type 00 class 0x020000
[   13.914062] pci 0008:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid
[   13.914064] pci 0008:01:00.0: reg 0x10: [io  size 0x0100]
[   13.914245] pci 0008:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[   13.914361] pci 0008:01:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
[   13.915340] pci 0008:01:00.0: supports D1 D2
[   13.915341] pci 0008:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[   13.930106] pci 0008:00:00.0: BAR 8: assigned [mem 0x3528000000-0x35280fffff]
[   13.930110] pci 0008:00:00.0: BAR 7: assigned [io  0x1000-0x1fff]
[   13.930113] pci 0008:01:00.0: BAR 4: assigned [mem 0x3528000000-0x3528003fff 64bit]
[   13.930205] pci 0008:01:00.0: BAR 2: assigned [mem 0x3528004000-0x3528004fff 64bit]
[   13.930297] pci 0008:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[   13.930326] pci 0008:00:00.0: PCI bridge to [bus 01-ff]
[   13.930329] pci 0008:00:00.0:   bridge window [io  0x1000-0x1fff]
[   13.930332] pci 0008:00:00.0:   bridge window [mem 0x3528000000-0x35280fffff]
[   13.930419] pcieport 0008:00:00.0: Adding to iommu group 3
[   13.930502] pcieport 0008:00:00.0: PME: Signaling with IRQ 189
[   13.930800] pcieport 0008:00:00.0: AER: enabled with IRQ 189
[   13.931077] tegra194-pcie 140c0000.pcie: Adding to iommu group 4
[   13.933561] tegra194-pcie 140c0000.pcie: host bridge /bus@0/pcie@140c0000 ranges:
[   13.933579] tegra194-pcie 140c0000.pcie:      MEM 0x3540000000..0x37ffffffff -> 0x3540000000
[   13.933585] tegra194-pcie 140c0000.pcie:      MEM 0x3828000000..0x382fffffff -> 0x0040000000
[   13.933588] tegra194-pcie 140c0000.pcie:       IO 0x002c100000..0x002c1fffff -> 0x002c100000
[   13.934037] tegra194-pcie 140c0000.pcie: iATU unroll: enabled
[   13.934040] tegra194-pcie 140c0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   15.036716] tegra194-pcie 140c0000.pcie: Phy link never came up
[   16.038492] tegra194-pcie 140c0000.pcie: Phy link never came up
[   16.038544] tegra194-pcie 140c0000.pcie: PCI host bridge to bus 0009:00
[   16.038548] pci_bus 0009:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x2c100000-0x2c1fffff])
[   16.038551] pci_bus 0009:00: root bus resource [mem 0x3828000000-0x382fffffff] (bus address [0x40000000-0x47ffffff])
[   16.038553] pci_bus 0009:00: root bus resource [bus 00-ff]
[   16.038555] pci_bus 0009:00: root bus resource [mem 0x3540000000-0x37ffffffff pref]
[   16.038598] pci 0009:00:00.0: [10de:229c] type 01 class 0x060400
[   16.038745] pci 0009:00:00.0: PME# supported from D0 D3hot
[   16.043915] pci 0009:00:00.0: PCI bridge to [bus 01-ff]
[   16.044000] pcieport 0009:00:00.0: Adding to iommu group 4
[   16.044081] pcieport 0009:00:00.0: PME: Signaling with IRQ 191
[   16.044613] pcieport 0009:00:00.0: AER: enabled with IRQ 191
[   16.045064] pci_bus 0009:01: busn_res: [bus 01-ff] is released
[   16.045138] pci 0009:00:00.0: Removing from iommu group 4
[   16.045147] pci_bus 0009:00: busn_res: [bus 00-ff] is released
[   16.046032] tegra194-pcie 14100000.pcie: Adding to iommu group 5
[   16.047488] tegra194-pcie 14100000.pcie: host bridge /bus@0/pcie@14100000 ranges:
[   16.047499] tegra194-pcie 14100000.pcie:      MEM 0x2080000000..0x20a7ffffff -> 0x2080000000
[   16.047504] tegra194-pcie 14100000.pcie:      MEM 0x20a8000000..0x20afffffff -> 0x0040000000
[   16.047507] tegra194-pcie 14100000.pcie:       IO 0x0030100000..0x00301fffff -> 0x0030100000
[   16.047786] tegra194-pcie 14100000.pcie: iATU unroll: enabled
[   16.047788] tegra194-pcie 14100000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   17.157421] tegra194-pcie 14100000.pcie: Phy link never came up
[   18.159203] tegra194-pcie 14100000.pcie: Phy link never came up
[   18.159244] tegra194-pcie 14100000.pcie: PCI host bridge to bus 0001:00
[   18.159247] pci_bus 0001:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x30100000-0x301fffff])
[   18.159250] pci_bus 0001:00: root bus resource [mem 0x20a8000000-0x20afffffff] (bus address [0x40000000-0x47ffffff])
[   18.159253] pci_bus 0001:00: root bus resource [bus 00-ff]
[   18.159254] pci_bus 0001:00: root bus resource [mem 0x2080000000-0x20a7ffffff pref]
[   18.159298] pci 0001:00:00.0: [10de:229e] type 01 class 0x060400
[   18.159459] pci 0001:00:00.0: PME# supported from D0 D3hot
[   18.164637] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[   18.164717] pcieport 0001:00:00.0: Adding to iommu group 5
[   18.164794] pcieport 0001:00:00.0: PME: Signaling with IRQ 193
[   18.165317] pcieport 0001:00:00.0: AER: enabled with IRQ 193
[   18.165772] pci_bus 0001:01: busn_res: [bus 01-ff] is released
[   18.165837] pci 0001:00:00.0: Removing from iommu group 5
[   18.165845] pci_bus 0001:00: busn_res: [bus 00-ff] is released
[   18.166912] tegra194-pcie 14160000.pcie: Adding to iommu group 6
[   18.168358] tegra194-pcie 14160000.pcie: host bridge /bus@0/pcie@14160000 ranges:
[   18.168369] tegra194-pcie 14160000.pcie:      MEM 0x2140000000..0x2427ffffff -> 0x2140000000
[   18.168375] tegra194-pcie 14160000.pcie:      MEM 0x2428000000..0x242fffffff -> 0x0040000000
[   18.168378] tegra194-pcie 14160000.pcie:       IO 0x0036100000..0x00361fffff -> 0x0036100000
[   18.168674] tegra194-pcie 14160000.pcie: iATU unroll: enabled
[   18.168676] tegra194-pcie 14160000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   18.275651] tegra194-pcie 14160000.pcie: Link up
[   18.278250] tegra194-pcie 14160000.pcie: Link up
[   18.278289] tegra194-pcie 14160000.pcie: PCI host bridge to bus 0004:00
[   18.278292] pci_bus 0004:00: root bus resource [io  0x300000-0x3fffff] (bus address [0x36100000-0x361fffff])
[   18.278294] pci_bus 0004:00: root bus resource [mem 0x2428000000-0x242fffffff] (bus address [0x40000000-0x47ffffff])
[   18.278296] pci_bus 0004:00: root bus resource [bus 00-ff]
[   18.278297] pci_bus 0004:00: root bus resource [mem 0x2140000000-0x2427ffffff pref]
[   18.278336] pci 0004:00:00.0: [10de:229c] type 01 class 0x060400
[   18.278481] pci 0004:00:00.0: PME# supported from D0 D3hot
[   18.281387] pci 0004:01:00.0: [144d:a809] type 00 class 0x010802
[   18.281579] pci 0004:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
[   18.294073] pci 0004:00:00.0: BAR 8: assigned [mem 0x2428000000-0x24280fffff]
[   18.294077] pci 0004:01:00.0: BAR 0: assigned [mem 0x2428000000-0x2428003fff 64bit]
[   18.294162] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[   18.294166] pci 0004:00:00.0:   bridge window [mem 0x2428000000-0x24280fffff]
[   18.294237] pcieport 0004:00:00.0: Adding to iommu group 6
[   18.294306] pcieport 0004:00:00.0: PME: Signaling with IRQ 195
[   18.294594] pcieport 0004:00:00.0: AER: enabled with IRQ 195
[   18.294806] tegra194-pcie 141e0000.pcie: Adding to iommu group 7
[   18.296853] tegra194-pcie 141e0000.pcie: host bridge /bus@0/pcie@141e0000 ranges:
[   18.296870] tegra194-pcie 141e0000.pcie:      MEM 0x3000000000..0x3227ffffff -> 0x3000000000
[   18.296875] tegra194-pcie 141e0000.pcie:      MEM 0x3228000000..0x322fffffff -> 0x0040000000
[   18.296878] tegra194-pcie 141e0000.pcie:       IO 0x003e100000..0x003e1fffff -> 0x003e100000
[   18.297320] tegra194-pcie 141e0000.pcie: iATU unroll: enabled
[   18.297323] tegra194-pcie 141e0000.pcie: Detected iATU regions: 8 outbound, 2 inbound
[   19.405422] tegra194-pcie 141e0000.pcie: Phy link never came up
[   20.407207] tegra194-pcie 141e0000.pcie: Phy link never came up
[   20.407260] tegra194-pcie 141e0000.pcie: PCI host bridge to bus 0007:00
[   20.407263] pci_bus 0007:00: root bus resource [io  0x400000-0x4fffff] (bus address [0x3e100000-0x3e1fffff])
[   20.407265] pci_bus 0007:00: root bus resource [mem 0x3228000000-0x322fffffff] (bus address [0x40000000-0x47ffffff])
[   20.407268] pci_bus 0007:00: root bus resource [bus 00-ff]
[   20.407270] pci_bus 0007:00: root bus resource [mem 0x3000000000-0x3227ffffff pref]
[   20.407308] pci 0007:00:00.0: [10de:229a] type 01 class 0x060400
[   20.407450] pci 0007:00:00.0: PME# supported from D0 D3hot
[   20.412572] pci 0007:00:00.0: PCI bridge to [bus 01-ff]
[   20.412648] pcieport 0007:00:00.0: Adding to iommu group 7
[   20.412719] pcieport 0007:00:00.0: PME: Signaling with IRQ 197
[   20.413253] pcieport 0007:00:00.0: AER: enabled with IRQ 197
[   20.413692] pci_bus 0007:01: busn_res: [bus 01-ff] is released
[   20.413756] pci 0007:00:00.0: Removing from iommu group 7
modules/5.15.136-tegra/kernel/drivers/nvme/host/nvme-core.ko s eased
insmod /lib/modules/5.15.136-tegra/kernel/drivers/nvme/host/nvme.ko
[   20.428506] nvme 0004:01:00.0: Adding to iommu group 6
[   20.428687] nvme nvme0: pci function 0004:01:00.0
[   20.428912] nvme 0004:01:00.0: enabling device (0000 -> 0002)
[   20.433568] nvme nvme0: Shutdown timeout set to 8 seconds
[   20.450132] nvme nvme0: allocated 64 MiB host memory buffer.
[   23.238177] nvme nvme0: 6/0/0 default/read/poll queues
[   24.522682]  nvme0n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15
insmod /lib/modules/5.15.136-tegra/kernel/drivers/thermal/tegra/tegra-bpmp-thermal.ko
insmod /lib/modules/5.15.136-tegra/kernel/drivers/pwm/pwm-tegra.ko
insmod /lib/modules/5.15.136-tegra/kernel/drivers/hwmon/pwm-fan.ko
Checking whether device /dev/mmcblk?p1 exist
Device /dev/mmcblk?p1 does not exist
Checking whether device /dev/sd?1 exist
[   24.549445] Finding OTA work dir on external storage devices
Device /dev/sd?1 does not exist
Checking whether device /dev/nvme?n1p1 exist
Looking for OTA work directory on the device(s): /dev/nvme0n1p1
[   24.556711] mount /dev/nvme0n1p1 /mnt
[   24.572539] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[   24.574101] is_boot_only_partition /mnt
[   24.582372] OTA work directory /mnt/ota_work is not found on /dev/nvme0n1p1
[   24.587392] Finding OTA work dir on internal storage device
mount: /mnt: special device /dev/mmcblk0 does not exist.
[   24.588540] mount /dev/mmcblk0 /mnt
[   24.591033] Failed to mount /dev/mmcblk0 on the /mnt
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
bash-5.1# [   24.592254] Failed to run "mount_ota_work_partition /dev/mmcblk0 /mnt"
[   24.593367] OTA work directory is not found on internal and external storage devices

bash-5.1# lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
nvme0n1      259:0    0 238.5G  0 disk
|-nvme0n1p1  259:1    0   237G  0 part
|-nvme0n1p2  259:2    0   128M  0 part
|-nvme0n1p3  259:3    0   768K  0 part
|-nvme0n1p4  259:4    0  31.6M  0 part
|-nvme0n1p5  259:5    0   128M  0 part
|-nvme0n1p6  259:6    0   768K  0 part
|-nvme0n1p7  259:7    0  31.6M  0 part
|-nvme0n1p8  259:8    0    80M  0 part
|-nvme0n1p9  259:9    0   512K  0 part
|-nvme0n1p10 259:10   0    64M  0 part
|-nvme0n1p11 259:11   0    80M  0 part
|-nvme0n1p12 259:12   0   512K  0 part
|-nvme0n1p13 259:13   0    64M  0 part
|-nvme0n1p14 259:14   0   400M  0 part
`-nvme0n1p15 259:15   0 479.5M  0 part

What I did wrong?

[WayneWWW]

If the board fails to boot into file system for consecutive 3 times, then it will enter recovery image.

It is pointless to check a log from recovery image because it is all the same for everyone. What needs to check is what happened before recovery image so that you failed to boot 3 times.

Please recover from this situation by this method and check what is happened in the log.

AGX Orin Recovery Boot

Hi, Please try below method and it will go back to direct Boot again. Press ESC to enter UEFI Menu, then choose Device Manager → NVIDIA Configuration → L4T Configuration → OS chain A status → (The value is Unbootable if UEFI attemps recovery kernel) choose Normal → Save and exit, reboot, UEFI will try Direct Boot.

[bbear953308023]

Nice, It works. Thank you so much for you help~