VPN connection cause Orin system hang

Hi,

Sorry that I still need to do some clarifications.

  1. I feel the cause you are talking about does not match to the topic. Your topic sounds like “disconnecting display and keyboard” is a must-have step to reproduce issue. But you tell us the method to crash device is through VPN. So which one is correct? Or both of them will lead device crash?

  2. Do you have any devices on your side that no need the help from a 3rd guy (your customer) ? Based on past experience, this debug way is not a good idea if we need to check situation with another guy who cannot fully conduct our debug process.

  3. If you really don’t have a device on your side, could you share a method so that we can reproduce same issue on our devkit?

Incidentally, this does suggest a driver issue (perhaps due to the driver, but also perhaps due to data passed to the driver), but is not nearly enough of the message to debug:

kernel:[  446.328294] Internal error: Oops: 8600000d [#1] PREEMPT SMP

If you get the full dmesg with content leading up to (and including) the kernel issue, and can post it here, this would help.

OK guys allow me to clarify since the story has evolved and not everything has been communicated here:

  1. We initially thought that the Jetpack Linux was going to sleep / getting stuck after removing peripherals or monitor. This was happening on the client’s device.
  2. After setting up a secondary Orin (local, in-house) and reproducing the issue we have decided that it was the traffic on L2TP VPN that was causing it to freeze, not peripherals, on our local Orin.
  3. dmesg and syslog have been published above, but I’ll link them here again:
    3.1. Surveily
    3.2. Surveily
    3.3. password: 7*P6py^&Tk@eHj
  4. I am now trying to set up OpenVPN to see if it will work.
  5. We normally use Wireguard but Jetpack Linux has the kernel module disabled.

If the error is the system just went off and even ssh is gone, then it could be kernel panic.

Then your log may not help here. dmesg won’t be able to capture kernel panic. Syslog won’t have it either.

You need the setup in my previous comment to dump from UART serial console. If this is too complicated for your customer to operate, then please do it on your side.

I am trying both the micro usb and usb-c connection options but neither registers any ACM* in /dev. Do I need to boot it up in recovery mode?

Hi,

Devices name are /dev/ttyACM* and on your x86 host machines. Not jetson.

No need to be in recovery mode.

I am checking on my machine, not jetson. I did flash the device via SDKManager to JP 5.0.2.

Could you connect the micro usb cable between your host and jetson and check if the “dmesg” on your “host PC” ever showing anything new?

Tried three different micro usb cables and it finally worked. Now I get this. How can I get the logs?

OK after reboot I see logs



Welcome to minicom 2.8

OPTIONS: I18n 
Port /dev/ttyACM0, 09:00:58

Press CTRL-A Z for help on special keys
                                                              
��                                                            
[0000.064] I> MB1 (version: 0.23.0.0-t234-54845784-92215740)  
[0000.070] I> t234-A01-0-Silicon (0x12347) Prod               
[0000.074] I> Boot-mode : Coldboot                            
[0000.077] I> Emulation: 
[0000.079] I> Entry timestamp: 0x00000000
[0000.083] I> last_boot_error: 0x0
[0000.086] I> BR-BCT: preprod_dev_sign: 0
[0000.090] I> rst_source: 0x0, rst_level: 0x0
[0000.094] I> Task: CRC check (0x5001cf71)
[0000.098] I> Skip CRC check as records_integrity fuse is not burned
[0000.104] I> Task: Crypto init (0x5001c0d9)
[0000.108] I> Task: Secure debug controls (0x5000cb19)
[0000.113] I> Task: strap war set (0x5000b739)
[0000.117] I> strap value(0x4000401) set to 0x4000401
[0000.122] I> Task: Initialize SOC Therm (0x5001a509)
[0000.127] I> Task: Program NV master stream id (0x5000cad1)
[0000.132] I> Task: Verify boot mode (0xd482021)
[0000.138] I> Task: Alias fuses (0x50010749)
[0000.143] E> FUSE_ALIAS: Fuse alias on production fused part is not supported.
[0000.150] I> Task: Print SKU type (0x5000fba9)
[0000.155] I> FUSE_OPT_CCPLEX_CLUSTER_DISABLE = 0x00000000
[0000.160] I> FUSE_OPT_GPC_DISABLE = 0x00000000
[0000.164] I> FUSE_OPT_TPC_DISABLE = 0x00000000
[0000.168] I> FUSE_OPT_DLA_DISABLE = 0x00000000
[0000.173] I> FUSE_OPT_EMC_DISABLE = 0x00000000
[0000.177] I> FUSE_BOOTROM_PATCH_VERSION = 0x7
[0000.181] I> FUSE_PSCROM_PATCH_VERSION = 0x7
[0000.185] I> FUSE_OPT_ADC_CAL_FUSE_REV = 0x2
[0000.189] I> FUSE_SKU_INFO_0 = 0xd0
[0000.192] I> FUSE_OPT_SAMPLE_TYPE_0 = 0x3 PS 
[0000.197] I> FUSE_PACKAGE_INFO_0 = 0x2
[0000.200] I> SKU: Prod
[0000.202] I> Task: Boost clocks (0x50013db1)
[0000.207] I> Initializing PLLC2 for AXI_CBB.
[0000.211] I> AXI_CBB : src = 35, divisor = 0
[0000.215] I> Task: Voltage monitor (0x50013dbd)
[0000.219] I> VMON: Vmon re-calibration and fine tuning done
[0000.225] I> Task: UPHY init (0x5000da39)
[0000.229] E> UPHY: UPHY lane info table is empty in MB1 BCT.
[0000.234] I> Task: Boot device init (0x50000a3d)
[0000.239] I> Boot_device: QSPI_FLASH instance: 0
[0000.243] I> Qspi clock source : pllc_out0
[0000.247] I> QSPI Flash: Macronix 64MB
[0000.251] I> QSPI-0l initialized successfully
[0000.255] I> Task: TSC init (0x5001ed35)
[0000.259] I> Task: Load membct (0x5001190d)
[0000.263] I> RAM_CODE 0x4000401
[0000.266] I> Loading MEM-BCT to SysRAM
[0000.270] I> Scratch RSV109: 0x00000001    active chain: 0
[0000.275] I> Slot: 0
[0000.277] I> Binary[0] block-0 (partition size: 0x40000)
[0000.282] I>  get_binary_info: Binary name: MEM-BCT-0
[0000.287] I> Size of crypto header is 8192
[0000.291] I> BCH load address is : 0x40050000
[0000.295] I> Size of crypto header is 8192
[0000.299] I> BCH of MEM-BCT-0 read from storage
[0000.304] I> BCH address is : 0x40050000
[0000.308] I> MEM-BCT-0 header integrity check is success
[0000.313] I> Binary magic in BCH component 0 is MEM0
[0000.318] I> component binary type is 0
[0000.322] I> MEM-BCT-0 binary is read from storage
[0000.327] I> MEM-BCT-0 binary integrity check is success
[0000.332] I> Binary MEM-BCT-0 loaded successfully at 0x40040000 (0xe580)
[0000.338] I> RAM_CODE 0x4000401
[0000.344] I> RAM_CODE 0x4000401
[0000.348] I> Task: Load Page retirement list (0x50011139)
[0000.353] I> Task: SDRAM params override (0x50011b79)
[0000.358] I> Task: Save mem-bct info (0x50014071)
[0000.362] I> Task: Carveout allocate (0x500140d5)
[0000.367] I> RCM blob carveout will not be allocated
[0000.372] I> ECC region[0]: Start:0x0, End:0x0
[0000.376] I> ECC region[1]: Start:0x0, End:0x0
[0000.380] I> ECC region[2]: Start:0x0, End:0x0
[0000.384] I> ECC region[3]: Start:0x0, End:0x0
[0000.389] I> ECC region[4]: Start:0x0, End:0x0
[0000.393] I> Non-ECC region[0]: Start:0x80000000, End:0x880000000
[0000.399] I> Non-ECC region[1]: Start:0x0, End:0x0
[0000.404] I> Non-ECC region[2]: Start:0x0, End:0x0
[0000.408] I> Non-ECC region[3]: Start:0x0, End:0x0
[0000.413] I> Non-ECC region[4]: Start:0x0, End:0x0
[0000.422] I> allocated(CO:44) base:0x849800000 size:0x36800000 align: 0x100000
[0000.430] I> allocated(CO:31) base:0x840000000 size:0x8000000 align: 0x8000000
[0000.437] I> allocated(CO:39) base:0x83a000000 size:0x6000000 align: 0x200000
[0000.444] I> allocated(CO:20) base:0x838000000 size:0x2000000 align: 0x2000000
[0000.451] I> allocated(CO:24) base:0x836000000 size:0x2000000 align: 0x2000000
[0000.458] I> allocated(CO:28) base:0x834000000 size:0x2000000 align: 0x2000000
[0000.465] I> allocated(CO:29) base:0x832000000 size:0x2000000 align: 0x2000000
[0000.472] I> allocated(CO:43) base:0x830000000 size:0x2000000 align: 0x200000
[0000.479] I> allocated(CO:22) base:0x848000000 size:0x1000000 align: 0x1000000
[0000.486] I> allocated(CO:41) base:0x82f000000 size:0x1000000 align: 0x100000
[0000.493] I> allocated(CO:02) base:0x849000000 size:0x800000 align: 0x800000
[0000.500] I> allocated(CO:03) base:0x82e800000 size:0x800000 align: 0x800000
[0000.507] I> allocated(CO:06) base:0x82e000000 size:0x800000 align: 0x800000
[0000.514] I> allocated(CO:10) base:0x82d800000 size:0x800000 align: 0x800000
[0000.521] I> allocated(CO:56) base:0x82d000000 size:0x800000 align: 0x200000
[0000.528] I> allocated(CO:07) base:0x82cc00000 size:0x400000 align: 0x400000
[0000.535] I> allocated(CO:33) base:0x82c800000 size:0x400000 align: 0x200000
[0000.542] I> allocated(CO:47) base:0x82c400000 size:0x400000 align: 0x200000
[0000.549] I> allocated(CO:72) base:0x82c200000 size:0x200000 align: 0x10000
[0000.555] I> allocated(CO:01) base:0x82c100000 size:0x100000 align: 0x100000
[0000.562] I> allocated(CO:04) base:0x82c000000 size:0x100000 align: 0x100000
[0000.569] I> allocated(CO:05) base:0x82bf00000 size:0x100000 align: 0x100000
[0000.576] I> allocated(CO:08) base:0x82be00000 size:0x100000 align: 0x100000
[0000.583] I> allocated(CO:09) base:0x82bd00000 size:0x100000 align: 0x100000
[0000.590] I> allocated(CO:15) base:0x82bc00000 size:0x100000 align: 0x100000
[0000.597] I> allocated(CO:17) base:0x82bb00000 size:0x100000 align: 0x100000
[0000.604] I> allocated(CO:23) base:0x82ba00000 size:0x100000 align: 0x100000
[0000.611] I> allocated(CO:27) base:0x82b900000 size:0x100000 align: 0x100000
[0000.618] I> allocated(CO:42) base:0x82b800000 size:0x100000 align: 0x100000
[0000.625] I> allocated(CO:54) base:0x82b780000 size:0x80000 align: 0x80000
[0000.631] I> allocated(CO:26) base:0x82b740000 size:0x40000 align: 0x2000
[0000.638] I> allocated(CO:48) base:0x82b720000 size:0x20000 align: 0x10000
[0000.645] I> allocated(CO:69) base:0x82b700000 size:0x20000 align: 0x10000
[0000.652] I> allocated(CO:34) base:0x82b6f0000 size:0x10000 align: 0x10000
[0000.658] I> allocated(CO:35) base:0x82b6e0000 size:0x10000 align: 0x10000
[0000.665] I> allocated(CO:49) base:0x82b6d0000 size:0x10000 align: 0x10000
[0000.672] I> allocated(CO:50) base:0x82b6c0000 size:0x10000 align: 0x10000
[0000.679] I> allocated(CO:52) base:0x82b6b0000 size:0x10000 align: 0x10000
[0000.685] I> Task: Thermal check (0x5001fba1)
[0000.689] I> max_chip_limit = 115
[0000.693] I> min_chip_limit = -30
[0000.696] I> max temp read = 33
[0000.699] I> min temp read = 32
[0000.702] I> Task: Update FSI SCR with thermal fuse data (0x5001fc3d)
[0000.708] I> Task: Enable WDT 5th expiry (0x5001f929)
[0000.713] I> Task: Disable WDT globally (0x5001f965)
[0000.718] I> Task: I2C register (0x50000a31)
[0000.722] I> Task: Reset FSI (0x50013db9)
[0000.726] I> Task: Pinmux init (0x50013219)
[0000.730] I> Task: Prod config init (0x50013631)
[0000.735] I> Task: Pad voltage init (0x500132a5)
[0000.739] I> Task: Prod init (0x500136d5)
[0000.743] I> Task: Common rail init (0x50013bd1)
[0000.747] W>  Thermal config not found in BCT
[0000.752] W>  Generic i2c config not found in BCT
[0000.756] W>  Platform rail config not found in BCT
[0000.762] W> DEVICE_PROD: module = 13, instance = 4 not found in device prod.
[0000.770] W>  MEMIO rail config not found in BCT
[0000.774] I> Task: Mem clock src (0x500119c5)
[0000.779] I> Task: Misc. board config (0x50013c29)
[0000.783] W>  Platform config not found in BCT
[0000.788] I> Task: SDRAM init (0x500119c9)
[0000.794] I> SDRAM-params @ 0xd485000
[0000.797] I> MSS code-drop: NvBootSdramInit
[0000.801] I> MSS CAR: PLLM/HUB programming for MemoryType: 4 and MemBctRevision: 8
[0000.809] I> MSS CAR: Init PLLM
[0000.812] I> MSS CAR: Init PLLHUB
[0000.817] I> Encryption:   MTS: en, TX: dis, VPR: dis, GSC: en
[0000.829] I> mb1_sdram_init Done !
[0000.832] I> SDRAM Size in Total 0x800000000
[0000.836] I> Task: Dram Ecc scrub (0x50011261)
[0000.840] I> Task: DRAM alias check (0x50011b71)
[0000.871] I> Task: Program NSDRAM carveout (0x50014921)
[0000.878] I> NSDRAM carveout encryption is disabled
[0000.883] I> Program NSDRAM carveout
[0000.887] I> Task: Register checker (0x50011b75)
[0000.896] I> Task: Enable clock-mon (0x5001ed25)
[0000.912] I> FMON: Fmon re-programming done
[0000.917] I> Task: Mapper init (0x5001d451)
[0000.922] I> Task: SC7 Context Init (0x5001f02d)
[0000.928] I> Task: CCPLEX IST init (0x5000d335)
[0000.934] I> Task: CPU WP0 (0x5000c635)
[0000.939] I> Loading MCE
[0000.942] I> Scratch RSV109: 0x00000001    active chain: 0
[0000.948] I> Slot: 0
[0000.950] I> Binary[8] block-0 (partition size: 0x80000)
[0000.956] I>  get_binary_info: Binary name: MCE
[0000.961] I> Size of crypto header is 8192
[0000.965] I> BCH load address is : 0x4003e000
[0000.971] I> Size of crypto header is 8192
[0000.976] I> BCH of MCE read from storage
[0000.980] I> BCH address is : 0x4003e000
[0000.985] I> MCE header integrity check is success
[0000.990] I> Binary magic in BCH component 0 is MTSM
[0000.996] I> component binary type is 8
[0001.000] I> Size of crypto header is 8192
[0001.006] I> MCE binary is read from storage
[0001.012] I> MCE binary integrity check is success
[0001.017] I> Binary MCE loaded successfully at 0x40000000 (0x2c080)
[0001.024] I> Size of crypto header is 8192
[0001.039] I> Size of crypto header is 8192
[0001.044] I> Sending WP0 mailbox command to PSC
[0001.054] I> Task: MB1 fixed firewalls (0x5001e55d)
[0001.074] I> Task: Load bpmp-fw (0x5001e801)
[0001.079] I> Scratch RSV109: 0x00000001    active chain: 0
[0001.085] I> Slot: 0
[0001.087] I> Binary[15] block-0 (partition size: 0x180000)
[0001.094] I>  get_binary_info: Binary name: BPMP_FW
[0001.099] I> Size of crypto header is 8192
[0001.103] I> BCH load address is : 0x807fe000
[0001.109] I> Size of crypto header is 8192
[0001.114] I> BCH of BPMP_FW read from storage
[0001.118] I> BCH address is : 0x807fe000
[0001.124] I> BPMP_FW header integrity check is success
[0001.129] I> Binary magic in BCH component 0 is BPMF
[0001.134] I> component binary type is 15
[0001.139] I> Size of crypto header is 8192
[0001.154] I> BPMP_FW binary is read from storage
[0001.161] I> BPMP_FW binary integrity check is success
[0001.167] I> Binary BPMP_FW loaded successfully at 0x80000000 (0xfb3c0)
[0001.175] I> Scratch RSV109: 0x00000001    active chain: 0
[0001.181] I> Slot: 0
[0001.183] I> Binary[16] block-10000 (partition size: 0x400000)
[0001.190] I>  get_binary_info: Binary name: BPMP_FW_DTB
[0001.195] I> Size of crypto header is 8192
[0001.200] I> BCH load address is : 0x807fc000
[0001.205] I> Size of crypto header is 8192
[0001.210] I> BCH of BPMP_FW_DTB read from storage
[0001.215] I> BCH address is : 0x807fc000
[0001.221] I> BPMP_FW_DTB header integrity check is success
[0001.227] I> Binary magic in BCH component 0 is BPMD
[0001.232] I> component binary type is 16
[0001.236] I> Size of crypto header is 8192
[0001.242] I> BPMP_FW_DTB binary is read from storage
[0001.248] I> BPMP_FW_DTB binary integrity check is success
[0001.254] I> Binary BPMP_FW_DTB loaded successfully at 0x807db9f0 (0x20440)
[0001.263] I> Task: Load psc-fw (0x5001ea0d)
[0001.269] I> Scratch RSV109: 0x00000001    active chain: 0
��Ƥ�冄�ńG��DŽd��䥇��Ć���b憆Ƈ���b憆Ƈ��b憆��m����������DC#ㄥ������GC#�����b憆Ƈ��ƃ��b憆Ƈ��e��F��̇b椆                                                                                          t
L4TLauncher: Attempting DireEFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Exiting boot services and installing virtual address map...
��I/TC: Secondary CPU 1 initializing
I/TC: Secondary CPU 1 switching to normal world boot
I/TC: Secondary CPU 2 initializing
I/TC: Secondary CPU 2 switching to normal world boot
I/TC: Secondary CPU 3 initializing
I/TC: Secondary CPU 3 switching to normal world boot
I/TC: Secondary CPU 4 initializing
I/TC: Secondary CPU 4 switching to normal world boot
I/TC: Secondary CPU 5 initializing
I/TC: Secondary CPU 5 switching to normal world boot
I/TC: Secondary CPU 6 initializing
I/TC: Secondary CPU 6 switching to normal world boot
I/TC: Secondary CPU 7 initializing
I/TC: Secondary CPU 7 switching to normal world boot
I/TC: Secondary CPU 8 initializing
I/TC: Secondary CPU 8 switching to normal world boot
I/TC: Secondary CPU 9 initializing
I/TC: Secondary CPU 9 switching to normal world boot
I/TC: Secondary CPU 10 initializing
I/TC: Secondary CPU 10 switching to normal world boot
I/TC: Secondary CPU 11 initializing
I/TC: Secondary CPU 11 switching to normal world boot
��[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd421]
[    0.000000] Linux version 5.10.104-tegra (buildbrain@mobile-u64-5273-d7000) (aarch64-buil2
[    0.000000] OF: fdt: memory scan node memory@80000000, reg size 16,
[    0.000000] OF: fdt:  - 80000000 ,  c0000000
[    0.000000] Machine model: Jetson AGX Orin
[    0.000000] efi: EFI v2.70 by EDK II
[    0.000000] efi: RTPROP=0x805405a98 SMBIOS=0xffff0000 SMBIOS 3.0=0x8039a0000 MEMATTR=0x80 
[    0.000000] efi: seeding entropy pool
[    0.000000] esrt: Reserving ESRT space from 0x0000000803790a18 to 0x0000000803790a50.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000817800000, size 256 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x0000000835ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x00000007ffffffff]
[    0.000000]   node   0: [mem 0x0000000800000000-0x00000008021bffff]
[    0.000000]   node   0: [mem 0x00000008021c0000-0x0000000803570fff]
[    0.000000]   node   0: [mem 0x0000000803571000-0x000000080370cfff]
[    0.000000]   node   0: [mem 0x000000080370d000-0x000000080398ffff]
[    0.000000]   node   0: [mem 0x0000000803990000-0x00000008039affff]
[    0.000000]   node   0: [mem 0x00000008039b0000-0x000000080453ffff]
[    0.000000]   node   0: [mem 0x0000000804540000-0x00000008046cffff]
[    0.000000]   node   0: [mem 0x00000008046d0000-0x000000080473ffff]
[    0.000000]   node   0: [mem 0x0000000804740000-0x000000080475ffff]
[    0.000000]   node   0: [mem 0x0000000804760000-0x000000080477ffff]
[    0.000000]   node   0: [mem 0x0000000804780000-0x000000080499ffff]
[    0.000000]   node   0: [mem 0x00000008049a0000-0x00000008049bffff]
[    0.000000]   node   0: [mem 0x00000008049c0000-0x0000000804a0ffff]
[    0.000000]   node   0: [mem 0x0000000804a10000-0x0000000804a9ffff]
[    0.000000]   node   0: [mem 0x0000000804aa0000-0x0000000804e4ffff]
[    0.000000]   node   0: [mem 0x0000000804e50000-0x0000000804e5ffff]
[    0.000000]   node   0: [mem 0x0000000804e60000-0x0000000804e6ffff]
[    0.000000]   node   0: [mem 0x0000000804e70000-0x0000000804e7ffff]
[    0.000000]   node   0: [mem 0x0000000804e80000-0x0000000804ebffff]
[    0.000000]   node   0: [mem 0x0000000804ec0000-0x000000080534ffff]
[    0.000000]   node   0: [mem 0x0000000805350000-0x000000080539ffff]
[    0.000000]   node   0: [mem 0x00000008053a0000-0x00000008053cffff]
[    0.000000]   node   0: [mem 0x00000008053d0000-0x00000008053effff]
[    0.000000]   node   0: [mem 0x00000008053f0000-0x000000080553ffff]
[    0.000000]   node   0: [mem 0x0000000805540000-0x00000008055dffff]
[    0.000000]   node   0: [mem 0x00000008055e0000-0x000000080568ffff]
[    0.000000]   node   0: [mem 0x0000000805690000-0x000000080572ffff]
[    0.000000]   node   0: [mem 0x0000000805730000-0x00000008057bffff]
[    0.000000]   node   0: [mem 0x00000008057c0000-0x000000080585ffff]
[    0.000000]   node   0: [mem 0x0000000805860000-0x000000080612ffff]
[    0.000000]   node   0: [mem 0x0000000806130000-0x000000080630ffff]
[    0.000000]   node   0: [mem 0x0000000806310000-0x00000008278bffff]
[    0.000000]   node   0: [mem 0x00000008278c0000-0x000000082790ffff]
[    0.000000]   node   0: [mem 0x0000000827910000-0x000000082b6affff]
[    0.000000]   node   0: [mem 0x000000082c400000-0x000000082c7fffff]
[    0.000000]   node   0: [mem 0x0000000834000000-0x0000000835ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000835ffffff]
[    0.000000] On node 0, zone Normal: 336 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] percpu: Embedded 32 pages/cpu s90136 r8192 d32744 u131072
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] alternatives: patching kernel code
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 7925040
[    0.000000] Kernel command line: root=/dev/nvme0n1p1 rw rootwait rootfstype=ext4 mminit_l 
[    0.000000] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear)
[    0.000000] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x00000000effe0000-0x00000000fffe0000] (256MB)
[    0.000000] Memory: 30994412K/32205504K available (18432K kernel code, 3172K rwdata, 7156)
[    0.000000] random: get_random_u64 called from __kmem_cache_create+0x38/0x420 with crng_i0
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=12, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu:     RCU event tracing is enabled.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=12.
[    0.000000]  Trampoline variant of Tasks RCU enabled.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=12
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 960 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x000000000f440000
[    0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:608, num:70)
[    0.000000] GICv2m: range[mem 0x0f410000-0x0f41ffff], SPI[608:677]
[    0.000000] arch_timer: cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046s
[    0.000004] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.000729] Console: colour dummy device 80x25
[    0.000921] printk: console [tty0] enabled
[    0.000955] Calibrating delay loop (skipped), value calculated using timer frequency.. 62)
[    0.000965] pid_max: default: 32768 minimum: 301
[    0.001025] LSM: Security Framework initializing
[    0.001046] Yama: becoming mindful.
[    0.001062] SELinux:  Initializing.
[    0.001142] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.001186] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.002987] rcu: Hierarchical SRCU implementation.
[    0.003073] dyndbg: Ignore empty _ddebug table in a CONFIG_DYNAMIC_DEBUG_CORE build
[    0.003600] DTS File Name: /dvs/git/dirty/git-master_linux/kernel/kernel-5.10/arch/arm64/s
[    0.003609] DTB Build time: Aug 10 2022 20:31:40
[    0.005022] Tegra Revision: A01 SKU: 0xd0 CPU Process: 0 SoC Process: 0
[    0.005134] Remapping and enabling EFI services.
[    0.006347] smp: Bringing up secondary CPUs ...
[    0.035010] Detected PIPT I-cache on CPU1
[    0.035045] GICv3: CPU1: found redistributor 100 region 0:0x000000000f460000
[    0.035072] CPU1: Booted secondary processor 0x0000000100 [0x410fd421]
[    0.063676] Detected PIPT I-cache on CPU2
[    0.063686] GICv3: CPU2: found redistributor 200 region 0:0x000000000f480000
[    0.063700] CPU2: Booted secondary processor 0x0000000200 [0x410fd421]
[    0.092272] Detected PIPT I-cache on CPU3
[    0.092281] GICv3: CPU3: found redistributor 300 region 0:0x000000000f4a0000
[    0.092295] CPU3: Booted secondary processor 0x0000000300 [0x410fd421]
[    0.122912] Detected PIPT I-cache on CPU4
[    0.122933] GICv3: CPU4: found redistributor 10000 region 0:0x000000000f4c0000
[    0.122957] CPU4: Booted secondary processor 0x0000010000 [0x410fd421]
[    0.151588] Detected PIPT I-cache on CPU5
[    0.151599] GICv3: CPU5: found redistributor 10100 region 0:0x000000000f4e0000
[    0.151613] CPU5: Booted secondary processor 0x0000010100 [0x410fd421]
[    0.180207] Detected PIPT I-cache on CPU6
[    0.180217] GICv3: CPU6: found redistributor 10200 region 0:0x000000000f500000
[    0.180231] CPU6: Booted secondary processor 0x0000010200 [0x410fd421]
[    0.208832] Detected PIPT I-cache on CPU7
[    0.208842] GICv3: CPU7: found redistributor 10300 region 0:0x000000000f520000
[    0.208856] CPU7: Booted secondary processor 0x0000010300 [0x410fd421]
[    0.239482] Detected PIPT I-cache on CPU8
[    0.239505] GICv3: CPU8: found redistributor 20000 region 0:0x000000000f540000
[    0.239530] CPU8: Booted secondary processor 0x0000020000 [0x410fd421]
[    0.268153] Detected PIPT I-cache on CPU9
[    0.268163] GICv3: CPU9: found redistributor 20100 region 0:0x000000000f560000
[    0.268178] CPU9: Booted secondary processor 0x0000020100 [0x410fd421]
[    0.297387] Detected PIPT I-cache on CPU10
[    0.297398] GICv3: CPU10: found redistributor 20200 region 0:0x000000000f580000
[    0.297412] CPU10: Booted secondary processor 0x0000020200 [0x410fd421]
[    0.326614] Detected PIPT I-cache on CPU11
[    0.326625] GICv3: CPU11: found redistributor 20300 region 0:0x000000000f5a0000
[    0.326639] CPU11: Booted secondary processor 0x0000020300 [0x410fd421]
[    0.326695] smp: Brought up 1 node, 12 CPUs
[    0.326758] SMP: Total of 12 processors activated.
[    0.326763] CPU features: detected: Privileged Access Never
[    0.326766] CPU features: detected: LSE atomic instructions
[    0.326768] CPU features: detected: User Access Override
[    0.326771] CPU features: detected: 32-bit EL0 Support
[    0.326774] CPU features: detected: Common not Private translations
[    0.326777] CPU features: detected: RAS Extension Support
[    0.326779] CPU features: detected: Data cache clean to the PoU not required for I/D cohee
[    0.326783] CPU features: detected: CRC32 instructions
[    0.326786] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
[    0.360151] CPU: All CPU(s) started at EL2
[    0.362362] devtmpfs: initialized
[    0.377642] Registered cp15_barrier emulation handler
[    0.377654] Registered setend emulation handler
[    0.377659] KASLR enabled
[    0.377774] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7s
[    0.377788] futex hash table entries: 4096 (order: 6, 262144 bytes, linear)
[    0.379257] pinctrl core: initialized pinctrl subsystem
[    0.379730] SMBIOS 3.0.0 present.
[    0.379737] DMI:  /, BIOS 1.0-d7fb19b 08/10/2022
[    0.379981] NET: Registered protocol family 16
[    0.380695] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations
[    0.380896] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.381091] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.381116] audit: initializing netlink subsys (disabled)
[    0.381186] audit: type=2000 audit(0.380:1): state=initialized audit_enabled=0 res=1
[    0.381331] thermal_sys: Registered thermal governor 'step_wise'
[    0.381333] thermal_sys: Registered thermal governor 'power_allocator'
[    0.381338] thermal_sys: Registered thermal governor 'pid_thermal_gov'
[    0.383370] cpuidle: using governor menu
[    0.383473] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.383590] ASID allocator initialised with 32768 entries
[    0.383675] Serial: AMBA PL011 UART driver
[    0.385223] tegra234_mc_sid_probe(): tegra234 mc-sid probe successful
[    0.385246] Tegra reboot handler registered.
[    0.402955] mc: mapped MMIO address: 0xffff8000111f0000 -> 0x2c10000
[    0.402969] mc: mapped MMIO address: 0xffff800011210000 -> 0x2c20000
[    0.402977] mc: mapped MMIO address: 0xffff800011230000 -> 0x2c30000
[    0.402984] mc: mapped MMIO address: 0xffff800011250000 -> 0x2c40000
[    0.402992] mc: mapped MMIO address: 0xffff800011270000 -> 0x2c50000
[    0.403001] mc: mapped MMIO address: 0xffff800011290000 -> 0x2b80000
[    0.403009] mc: mapped MMIO address: 0xffff8000112b0000 -> 0x2b90000
[    0.403016] mc: mapped MMIO address: 0xffff8000112d0000 -> 0x2ba0000
[    0.403022] mc: mapped MMIO address: 0xffff8000112f0000 -> 0x2bb0000
[    0.403029] mc: mapped MMIO address: 0xffff800011310000 -> 0x1700000
[    0.403035] mc: mapped MMIO address: 0xffff800011330000 -> 0x1710000
[    0.403042] mc: mapped MMIO address: 0xffff800011350000 -> 0x1720000
[    0.403049] mc: mapped MMIO address: 0xffff800011370000 -> 0x1730000
[    0.403055] mc: mapped MMIO address: 0xffff800011390000 -> 0x1740000
[    0.403061] mc: mapped MMIO address: 0xffff8000113b0000 -> 0x1750000
[    0.403068] mc: mapped MMIO address: 0xffff8000113d0000 -> 0x1760000
[    0.403075] mc: mapped MMIO address: 0xffff8000113f0000 -> 0x1770000
[    0.403085] nv-tegra-mc 2c10000.mc: No mssnvlink node
[    0.403107] mc-err: mcerr ops are set to t23x
[    0.403360] tegra234-cbb 13a00000.cbb-fabric: secure_irq = 21
[    0.403412] tegra234-cbb c600000.aon-fabric: secure_irq = 22
[    0.403453] tegra234-cbb d600000.bpmp-fabric: secure_irq = 23
[    0.403487] tegra234-cbb de00000.dce-fabric: secure_irq = 24
[    0.403520] tegra234-cbb be00000.rce-fabric: secure_irq = 25
[    0.403555] tegra234-cbb b600000.sce-fabric: secure_irq = 26
[    0.403757] 31d0000.serial: ttyAMA0 at MMIO 0x31d0000 (irq = 66, base_baud = 0) is a SBSA
[    1.704208] printk: console [ttyAMA0] enabled
[    1.712522] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    1.719233] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    1.725934] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    1.732632] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    1.741020] ACPI: Interpreter disabled.
[    1.746004] vdd-av1v1-hub: supplied by vdd-5v-sys
[    1.750881] vbusA-cvb: supplied by vdd-5v-sys
[    1.755355] vbusB-cvb: supplied by vdd-5v-sys
[    1.759831] vdd-usbc1-vbus: supplied by vdd-5v-sys
[    1.764729] vdd-usbc2-vbus: supplied by vdd-5v-sys
[    1.769788] iommu: Default domain type: Translated 
[    1.774701] eventlib_kernel: keventlib is initialized, test id: 0
[    1.780954] SCSI subsystem initialized
[    1.784759] usbcore: registered new interface driver usbfs
[    1.790256] usbcore: registered new interface driver hub
[    1.795580] usbcore: registered new device driver usb
[    1.800804] mc: Linux media interface: v0.10
[    1.805083] videodev: Linux video capture interface: v2.00
[    1.810704] pps_core: LinuxPPS API ver. 1 registered
[    1.815668] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giomett>
[    1.824805] PTP clock support registered
[    1.828910] tegra_wdt_t18x 2190000.watchdog: Tegra WDT init timeout = 120 sec
[    1.836073] tegra_wdt_t18x 2190000.watchdog: Registered successfully
[    1.843751] Registered efivars operations
[    1.848192] FPGA manager framework
[    1.851625] Advanced Linux Sound Architecture Driver Initialized.
[    1.857952] Bluetooth: Core ver 2.22
[    1.861534] NET: Registered protocol family 31
[    1.865978] Bluetooth: HCI device and connection manager initialized
[    1.872336] Bluetooth: HCI socket layer initialized
[    1.877214] Bluetooth: L2CAP socket layer initialized
[    1.882270] Bluetooth: SCO socket layer initialized
[    1.887610] camchar: rtcpu character device driver loaded
[    1.893223] clocksource: Switched to clocksource arch_sys_counter
[    2.050285] VFS: Disk quotas dquot_6.6.0
[    2.054247] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.061254] pnp: PnP ACPI: disabled
[    2.066595] la/ptsa driver initialized.
[    2.070478] NET: Registered protocol family 2
[    2.075030] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    2.084734] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, l)
[    2.093477] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    2.101766] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    2.109661] TCP: Hash tables configured (established 262144 bind 65536)
[    2.116351] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    2.123282] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    2.130714] NET: Registered protocol family 1
[    2.135310] RPC: Registered named UNIX socket transport module.
[    2.141244] RPC: Registered udp transport module.
[    2.145947] RPC: Registered tcp transport module.
[    2.150647] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    2.157094] PCI: CLS 0 bytes, default 64
[    2.161133] Trying to unpack rootfs image as initramfs...
[    2.275613] Freeing initrd memory: 8196K
[    2.281676] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[    2.289668] kvm [1]: IPA Size Limit: 48 bits
[    2.294098] kvm [1]: VHE mode initialized successfully
[    2.299937] Initialise system trusted keyrings
[    2.304473] workingset: timestamp_bits=46 max_order=23 bucket_order=0
[    2.312977] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.319048] NFS: Registering the id_resolver key type
[    2.324111] Key type id_resolver registered
[    2.328293] Key type id_legacy registered
[    2.332331] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    2.339039] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    2.346443] ntfs: driver 2.1.32 [Flags: R/W].
[    2.362378] NET: Registered protocol family 38
[    2.366827] Key type asymmetric registered
[    2.370922] Asymmetric key parser 'x509' registered
[    2.375816] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 240)
[    2.383268] io scheduler mq-deadline registered
[    2.387800] io scheduler kyber registered
[    2.395652] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[    2.401257] tegra-pwm 32a0000.pwm: PWM clk can sleep in ops
[    2.406852] tegra-pwm 32c0000.pwm: PWM clk can sleep in ops
[    2.412442] tegra-pwm 32f0000.pwm: PWM clk can sleep in ops
[    2.420382] tegra_dc_assign_hw_data: no matching compatible node
[    2.426397] tegradccommon module_init failed
[    2.430667] tegradc module_init failed
[    2.434657] EINJ: ACPI disabled.
[    2.439345] tegra-pmc c360000.pmc: scratch reg offset dts data not present
[    2.446226] tegra-pmc: ### PMC reset source: SYS_RESET_N
[    2.451538] tegra-pmc: ### PMC reset level: L0
[    2.455979] tegra-pmc: ### PMC reset status reg: 0x0
[    2.462280] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.475584] random: fast init done
[    2.475941] printk: console [ttyTCU0] enabled
[    4.162908] arm-smmu 12000000.iommu: probing hardware configuration...
[    4.170865] arm-smmu 12000000.iommu: SMMUv2 with:
[    4.175691] arm-smmu 12000000.iommu:         stage 1 translation
[    4.181231] arm-smmu 12000000.iommu:         stage 2 translation
[    4.186758] arm-smmu 12000000.iommu:         nested translation
[    4.192213] arm-smmu 12000000.iommu:         stream matching with 128 register groups
[    4.199629] arm-smmu 12000000.iommu:         128 context banks (0 stage-2 only)
[    4.208340] arm-smmu 12000000.iommu:         Supported page sizes: 0x00001000
[    4.215034] arm-smmu 12000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    4.221729] arm-smmu 12000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    4.229208] arm-smmu 10000000.iommu: probing hardware configuration...
[    4.235905] arm-smmu 10000000.iommu: SMMUv2 with:
[    4.240730] arm-smmu 10000000.iommu:         stage 1 translation
[    4.246259] arm-smmu 10000000.iommu:         stage 2 translation
[    4.251800] arm-smmu 10000000.iommu:         nested translation
[    4.257256] arm-smmu 10000000.iommu:         stream matching with 128 register groups
[    4.264660] arm-smmu 10000000.iommu:         128 context banks (0 stage-2 only)
[    4.273356] arm-smmu 10000000.iommu:         Supported page sizes: 0x00001000
[    4.280045] arm-smmu 10000000.iommu:         Stage-1: 48-bit VA -> 48-bit IPA
[    4.286745] arm-smmu 10000000.iommu:         Stage-2: 48-bit IPA -> 48-bit PA
[    4.293857] arm-smmu 8000000.iommu: probing hardware configuration...
[    4.300472] arm-smmu 8000000.iommu: SMMUv2 with:
[    4.305206] arm-smmu 8000000.iommu:  stage 1 translation
[    4.310648] arm-smmu 8000000.iommu:  stage 2 translation
[    4.316089] arm-smmu 8000000.iommu:  nested translation
[    4.321449] arm-smmu 8000000.iommu:  stream matching with 128 register groups
[    4.328776] arm-smmu 8000000.iommu:  128 context banks (0 stage-2 only)
[    4.337412] arm-smmu 8000000.iommu:  Supported page sizes: 0x00001000
[    4.344015] arm-smmu 8000000.iommu:  Stage-1: 48-bit VA -> 48-bit IPA
[    4.350627] arm-smmu 8000000.iommu:  Stage-2: 48-bit IPA -> 48-bit PA
[    4.357876] arm-smmu-suspend 12000000.smmu_suspend: arm_smmu_suspend probe successful
[    4.368439] tegra_profiler: version: 1.147, samples/io: 49/28
[    4.374409] tegra_profiler: auth: init
[    4.378536] nvsciipc: loaded module
[    4.383042] tegra23x-qspi 3270000.spi: Adding to iommu group 0
[    4.389216] tegra23x-qspi 3270000.spi: Node prod has invalid entries
[    4.395736] tegra23x-qspi 3270000.spi: Node prod: Child has not proper setting
[    4.403140] tegra23x-qspi 3270000.spi: Node spi: Faild to read the Prod Setting.
[    4.410729] tegra23x-qspi 3270000.spi: Prod settings list not found
[    4.417274] tegra23x-qspi 3270000.spi: Failed to get reset control: -517
[    4.424714] tun: Universal TUN/TAP device driver, 1.6
[    4.430311] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    4.437729] hns3: Copyright (c) 2017 Huawei Corporation.
[    4.443205] e1000e: Intel(R) PRO/1000 Network Driver
[    4.448287] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    4.454378] igb: Intel(R) Gigabit Ethernet Network Driver
[    4.459915] igb: Copyright (c) 2007-2014 Intel Corporation.
[    4.465770] PPP generic driver version 2.4.2
[    4.470185] PPP BSD Compression module registered
[    4.475017] PPP Deflate Compression module registered
[    4.480216] PPP MPPE Compression module registered
[    4.485156] usbcore: registered new interface driver r8152
[    4.490796] usbcore: registered new interface driver asix
[    4.496347] usbcore: registered new interface driver ax88179_178a
[    4.502609] usbcore: registered new interface driver cdc_ether
[    4.508591] usbcore: registered new interface driver net1080
[    4.514413] usbcore: registered new interface driver cdc_subset
[    4.520489] usbcore: registered new interface driver zaurus
[    4.526232] usbcore: registered new interface driver cdc_ncm
[    4.532036] usbcore: registered new interface driver aqc111
[    4.537864] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    4.544212] ohci-pci: OHCI PCI platform driver
[    4.548785] ohci-platform: OHCI generic platform driver
[    4.554417] tegra-xusb 3610000.xhci: Adding to iommu group 1
[    4.560703] usbcore: registered new interface driver uas
[    4.566178] usbcore: registered new interface driver usb-storage
[    4.572471] tegra-xudc 3550000.xudc: Adding to iommu group 2
[    4.578578] mousedev: PS/2 mouse device common for all mice
[    4.584318] usbcore: registered new interface driver xpad
[    4.591707] tegra_rtc c2a0000.rtc: registered as rtc1
[    4.597180] tegra_rtc c2a0000.rtc: setting system clock to 1970-01-01T00:00:28 UTC (28)
[    4.605401] tegra_rtc c2a0000.rtc: Tegra internal Real Time Clock
[    4.611784] i2c /dev entries driver
[    4.615463] tegra-i2c 3160000.i2c: Adding to iommu group 3
[    4.621271] tegra-i2c c240000.i2c: Adding to iommu group 3
[    4.626937] tegra-i2c 3180000.i2c: Adding to iommu group 3
[    4.632596] tegra-i2c 3190000.i2c: Adding to iommu group 3
[    4.638288] tegra-i2c 31b0000.i2c: Adding to iommu group 3
[    4.643956] tegra-i2c 31c0000.i2c: Adding to iommu group 3
[    4.649635] tegra-i2c c250000.i2c: Adding to iommu group 3
[    4.655304] tegra-i2c 31e0000.i2c: Adding to iommu group 3
[    4.663015] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoin.
[    4.671582] tegra-camrtc-capture-vi tegra-capture-vi: ep of_device is not enabled endpoin.
[    4.680844] device-mapper: uevent: version 1.0.3
[    4.685656] device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@redhat.m
[    4.695413] sdhci: Secure Digital Host Controller Interface driver
[    4.701766] sdhci: Copyright(c) Pierre Ossman
[    4.706239] Synopsys Designware Multimedia Card Interface Driver
[    4.712475] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.718422] sdhci-tegra 3460000.sdhci: Adding to iommu group 4
[    4.718629] sdhci-tegra 3400000.sdhci: Adding to iommu group 5
[    4.719126] SMCCC: SOC_ID: ID = jep106:036b:0023 Revision = 0x00000401
[    4.720006] tegra-bpmp bpmp: firmware: 888a114dda6bea50b6c6-eac380526d6
[    4.721874] nvvrs_pseq 4-003c: NVVRS Vendor ID: 0x9 
[    4.722447] nvvrs_pseq 4-003c: NVVRS Model Rev: 0x81
[    4.726631] nvvrs-pseq-rtc nvvrs-pseq-rtc: registered as rtc0
[    4.760013] nvvrs_pseq 4-003c: NVVRS PSEQ probe successful
��debugfs initialized
��[    5.989432] hid: raw HID events driver (C) Jiri Kosina
[    5.995153] usbcore: registered new interface driver usbhid
[    6.000896] usbhid: USB HID core driver
[    6.005600] tegra-dce d800000.dce: Adding to iommu group 6
[    6.012534] dce: dce_ipc_channel_init:314  Invalid Channel State [0x0] for ch_type [2]
[    6.021725] tegra234-aon c000000.aon: Adding to iommu group 7
[    6.027927] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB :]
[    6.030378]  c000000.aon:hsp: probed
[    6.036912] dce: dce_admin_ipc_handle_signal:89   Spurious signal on channel: [0]. Ignore.
[    6.036924] dce: dce_admin_ipc_handle_signal:89   Spurious signal on channel: [0]. Ignore.
[    6.049434] tegra234-aon c000000.aon: init done
[    6.049834] tegra23x-psc e860000.psc: Adding to iommu group 8
[    6.058238] dce: dce_admin_send_cmd_ver:413  version : [0x2] err : [0x0]
[    6.063246] tegra23x-psc e860000.psc: ext_cfg base:(____ptrval____)
[    6.069419] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB :]
[    6.075748] tegra23x-psc e860000.psc: init done
[    6.075897] tegra186-cam-rtcpu bc00000.rtcpu: Adding to iommu group 9
[    6.082222] dce: dce_admin_setup_clients_ipc:542  Channel Reset Complete for Type [1] ...
[    6.082224] dce: dce_admin_ipc_handle_signal:89   Spurious signal on channel: [1]. Ignore.
[    6.082236] dce: dce_admin_ipc_handle_signal:89   Spurious signal on channel: [1]. Ignore.
[    6.091887] tegra186-cam-rtcpu bc00000.rtcpu: Trace buffer configured at IOVA=0xbff00000
[    6.095897] dce: dce_admin_setup_clients_ipc:518  Get queue info failed for [2]
[    6.096279] dce: dce_mailbox_set_full_interrupt:165  Intr bit set multiple times for MB :]
[    6.106122] tegra-ivc-bus bc00000.rtcpu:ivc-bus: region 0: iova=0xbfec0000-0xbfee01ff siz4
[    6.110989] dce: dce_admin_setup_clients_ipc:542  Channel Reset Complete for Type [3] ...
[    6.110991] dce: dce_admin_ipc_handle_signal:89   Spurious signal on channel: [3]. Ignore.
[    6.111001] dce: dce_admin_ipc_handle_signal:89   Spurious signal on channel: [3]. Ignore.
[    6.119819] tegra-ivc-bus bc00000.rtcpu:ivc-bus:echo@0: echo: ver=0 grp=1 RX[16x64]=0x1000
[    6.128757] dce: dce_start_boot_flow:100  DCE_BOOT_DONE
[    6.137181] tegra-ivc-bus bc00000.rtcpu:ivc-bus:dbg@1: dbg: ver=0 grp=1 RX[1x448]=0x1900-0
[    6.217397] tegra-ivc-bus bc00000.rtcpu:ivc-bus:dbg@2: dbg: ver=0 grp=1 RX[1x8192]=0x1d800
[    6.229169] tegra-ivc-bus bc00000.rtcpu:ivc-bus:ivccontrol@3: ivccontrol: ver=0 grp=1 RX[0
[    6.241780] tegra-ivc-bus bc00000.rtcpu:ivc-bus:ivccapture@4: ivccapture: ver=0 grp=1 RX[0
[    6.254645] tegra-ivc-bus bc00000.rtcpu:ivc-bus:diag@5: diag: ver=0 grp=1 RX[1x64]=0x20080
[    6.267017] tegra186-cam-rtcpu bc00000.rtcpu: using cam RTCPU IRQ (117)
[    6.273851] tegra186-cam-rtcpu bc00000.rtcpu: tegra_camrtc_mon_create is successful
[    6.285599] tegra186-cam-rtcpu bc00000.rtcpu: firmware version cpu=rce cmd=6 sha1=7a10e460
[    6.299523] t19x_cache tegra-cache: probed
[    6.304024] scf-pmu-drv scf-pmu: Registered T23x SCF Uncore PMU
[    6.323162] optee: probing for conduit method.
[    6.327762] optee: revision 3.16 (65cc2218)
[    6.328539] optee: dynamic shared memory is enabled
[    6.337974] optee: initialized driver
[    6.353766] nvpmodel: initialized successfully
[    6.358429] tegra_hv: get_hvd: not initialized yet
[    6.363380] user_ivc_mempool: hypervisor not present
[    6.369522] usbcore: registered new interface driver snd-usb-audio
[    6.376664] u32 classifier
[    6.379457]     input device check on
[    6.383230]     Actions configured
[    6.386855] Initializing XFRM netlink socket
[    6.391642] NET: Registered protocol family 10
[    6.397296] Segment Routing with IPv6
[    6.401105] NET: Registered protocol family 17
[    6.405700] NET: Registered protocol family 15
[    6.410335] bridge: filtering via arp/ip/ip6tables is no longer available by default. Upd.
[    6.424291] Bluetooth: RFCOMM socket layer initialized
[    6.429597] Bluetooth: RFCOMM ver 1.11
[    6.433467] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[    6.439573] Bluetooth: HIDP socket layer initialized
[    6.445401] 9pnet: Installing 9P2000 support
[    6.449852] Key type dns_resolver registered
[    6.454584] registered taskstats version 1
[    6.458824] Loading compiled-in X.509 certificates
[    6.467265] Loaded X.509 cert 'Build time autogenerated kernel key: 90ac610178c49616e6b7a'
[    6.477961] tegra194-pcie 14100000.pcie: Adding to iommu group 10
[    6.489848] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[    6.496907] tegra194-pcie 14160000.pcie: Adding to iommu group 11
[    6.508358] tegra194-pcie 14160000.pcie: Using GICv2m MSI allocator
[    6.515139] tegra194-pcie 141a0000.pcie: Adding to iommu group 12
[    6.526401] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[    6.553978] tegra-soc-hwpm f100000.tegra_soc_hwpm: Adding to iommu group 13
[    6.563781] host1x 13e40000.host1x: Adding to iommu group 14
[    7.580111] t23x_init_gating_regs: Timed out waiting for syncpt ram init!
[    7.600759] host1x 13e40000.host1x: initialized
[    7.605754] iommu_context_dev 13e40000.host1x:niso0_ctx0: Adding to iommu group 15
[    7.613888] iommu_context_dev 13e40000.host1x:niso0_ctx0: initialized (streamid=53, iommu)
[    7.625870] iommu_context_dev 13e40000.host1x:niso0_ctx1: Adding to iommu group 16
[    7.633941] iommu_context_dev 13e40000.host1x:niso0_ctx1: initialized (streamid=54, iommu)
[    7.645905] iommu_context_dev 13e40000.host1x:niso0_ctx2: Adding to iommu group 17
[    7.653971] iommu_context_dev 13e40000.host1x:niso0_ctx2: initialized (streamid=55, iommu)
[    7.665947] iommu_context_dev 13e40000.host1x:niso0_ctx3: Adding to iommu group 18
[    7.674014] iommu_context_dev 13e40000.host1x:niso0_ctx3: initialized (streamid=56, iommu)
[    7.686010] iommu_context_dev 13e40000.host1x:niso0_ctx4: Adding to iommu group 19
[    7.694077] iommu_context_dev 13e40000.host1x:niso0_ctx4: initialized (streamid=57, iommu)
[    7.706033] iommu_context_dev 13e40000.host1x:niso0_ctx5: Adding to iommu group 20
[    7.714097] iommu_context_dev 13e40000.host1x:niso0_ctx5: initialized (streamid=58, iommu)
[    7.726062] iommu_context_dev 13e40000.host1x:niso0_ctx6: Adding to iommu group 21
[    7.734133] iommu_context_dev 13e40000.host1x:niso0_ctx6: initialized (streamid=59, iommu)
[    7.746091] iommu_context_dev 13e40000.host1x:niso0_ctx7: Adding to iommu group 22
[    7.754145] iommu_context_dev 13e40000.host1x:niso0_ctx7: initialized (streamid=60, iommu)
[    7.766098] iommu_context_dev 13e40000.host1x:niso1_ctx0: Adding to iommu group 23
[    7.774190] iommu_context_dev 13e40000.host1x:niso1_ctx0: initialized (streamid=53, iommu)
[    7.786153] iommu_context_dev 13e40000.host1x:niso1_ctx1: Adding to iommu group 24
[    7.794207] iommu_context_dev 13e40000.host1x:niso1_ctx1: initialized (streamid=54, iommu)
[    7.806185] iommu_context_dev 13e40000.host1x:niso1_ctx2: Adding to iommu group 25
[    7.814235] iommu_context_dev 13e40000.host1x:niso1_ctx2: initialized (streamid=55, iommu)
[    7.826213] iommu_context_dev 13e40000.host1x:niso1_ctx3: Adding to iommu group 26
[    7.834265] iommu_context_dev 13e40000.host1x:niso1_ctx3: initialized (streamid=56, iommu)
[    7.846226] iommu_context_dev 13e40000.host1x:niso1_ctx4: Adding to iommu group 27
[    7.854287] iommu_context_dev 13e40000.host1x:niso1_ctx4: initialized (streamid=57, iommu)
[    7.866246] iommu_context_dev 13e40000.host1x:niso1_ctx5: Adding to iommu group 28
[    7.874309] iommu_context_dev 13e40000.host1x:niso1_ctx5: initialized (streamid=58, iommu)
[    7.886273] iommu_context_dev 13e40000.host1x:niso1_ctx6: Adding to iommu group 29
[    7.894314] iommu_context_dev 13e40000.host1x:niso1_ctx6: initialized (streamid=59, iommu)
[    7.906275] iommu_context_dev 13e40000.host1x:niso1_ctx7: Adding to iommu group 30
[    7.914364] iommu_context_dev 13e40000.host1x:niso1_ctx7: initialized (streamid=60, iommu)
[    7.926340] falcon 15340000.vic: Adding to iommu group 31
[    7.939981] falcon 15340000.vic: initialized
[    7.944633] falcon 15380000.nvjpg: Adding to iommu group 32
[    7.955146] falcon 15380000.nvjpg: initialized
[    7.959842] falcon 15540000.nvjpg1: Adding to iommu group 33
[    7.970670] falcon 15540000.nvjpg1: initialized
[    7.975460] falcon 154c0000.nvenc: Adding to iommu group 34
[    7.989169] falcon 154c0000.nvenc: initialized
[    7.993866] falcon 15a50000.ofa: Adding to iommu group 35
[    8.004119] falcon 15a50000.ofa: initialized
[    8.008640] nvdec 15480000.nvdec: Adding to iommu group 36
[    8.025128] nvdec 15480000.nvdec: initialized
[    8.029730] tsec 15500000.tsec: Adding to iommu group 37
[    8.043716] tsec 15500000.tsec: initialized
[    8.051964] gic 2a41000.agic-controller: GIC IRQ controller registered
[    8.067979] tegra-aconnect aconnect@2a41000: Tegra ACONNECT bus registered
[    8.082801] gpio-394 (camera-control-output-low): hogged as output/low
[    8.089582] gpio-397 (camera-control-output-low): hogged as output/low
[    8.096370] gpio-487 (camera-control-output-low): hogged as output/low
[    8.103166] gpio-486 (camera-control-output-low): hogged as output/low
[    8.110827] gpiochip0: registered GPIOs 348 to 511 on tegra234-gpio
[    8.118009] gpiochip1: registered GPIOs 316 to 347 on tegra234-gpio-aon
[    8.125040] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[    8.131452] tegra-pwm 32a0000.pwm: PWM clk can sleep in ops
[    8.137888] tegra-pwm 32c0000.pwm: PWM clk can sleep in ops
[    8.144102] tegra-pwm 32f0000.pwm: PWM clk can sleep in ops
[    8.152076] misc tegra_camera_ctrl: tegra_camera_isomgr_register tpg_max_iso = 3916800KBs
[    8.160643] tegra194-isp5 14800000.isp: Adding to iommu group 38
[    8.169472] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: context isolation disabled due to U
[    8.179035] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: initialized
[    8.185594] tegra194-vi5 13e40000.host1x:vi0@15c00000: Adding to iommu group 39
[    8.193720] tegra194-vi5 13e40000.host1x:vi1@14c00000: Adding to iommu group 40
[    8.201777] scare-pigeon 13e40000.host1x:vi0-thi@15f00000: context isolation disabled dueU
[    8.211669] scare-pigeon 13e40000.host1x:vi0-thi@15f00000: initialized
[    8.218495] scare-pigeon 13e40000.host1x:vi1-thi@14f00000: context isolation disabled dueU
[    8.228389] scare-pigeon 13e40000.host1x:vi1-thi@14f00000: initialized
[    8.235194] scare-pigeon 13e40000.host1x:isp-thi@14b00000: context isolation disabled dueU
[    8.245084] scare-pigeon 13e40000.host1x:isp-thi@14b00000: initialized
[    8.251921] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx0: Adding to iommu group 41
[    8.260669] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx0: initialized (streamid=18)
[    8.272067] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx1: Adding to iommu group 42
[    8.280794] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx1: initialized (streamid=19)
[    8.292205] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx2: Adding to iommu group 43
[    8.300911] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx2: initialized (streamid=20)
[    8.312328] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx3: Adding to iommu group 44
[    8.321053] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx3: initialized (streamid=21)
[    8.332444] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx4: Adding to iommu group 45
[    8.341192] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx4: initialized (streamid=22)
[    8.352620] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx5: Adding to iommu group 46
[    8.361347] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx5: initialized (streamid=23)
[    8.372733] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx6: Adding to iommu group 47
[    8.372790] random: crng init done
[    8.381438] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx6: initialized (streamid=24)
[    8.396055] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx7: Adding to iommu group 48
[    8.404782] pva_iommu_context_dev 16000000.pva0:pva0_niso1_ctx7: initialized (streamid=25)
[    8.416172] pva 16000000.pva0: Adding to iommu group 49
[    8.440905] pva 16000000.pva0: initialized
[    8.451179] nvdla 15880000.nvdla0: Adding to iommu group 50
[    8.465193] nvdla 15880000.nvdla0: initialized
[    8.471374] nvdla 158c0000.nvdla1: Adding to iommu group 51
[    8.491146] nvdla 158c0000.nvdla1: initialized
[    8.497337] tegra-gpcdma 2600000.gpcdma: Adding to iommu group 3
[    8.504508] tegra-gpcdma 2600000.gpcdma: GPC DMA driver register 31 channels
[    8.511982] tegra-fuse-burn efuse-burn: shutdown limit check disabled
[    8.518617] tegra-fuse-burn efuse-burn: Fuse burn driver initialized
[    8.525350] serial-tegra 3100000.serial: Adding to iommu group 3
[    8.532281] 3100000.serial: ttyTHS0 at MMIO 0x3100000 (irq = 13, base_baud = 0) is a TEGRT
[    8.541946] serial-tegra 3130000.serial: Adding to iommu group 3
[    8.548711] 3130000.serial: ttyTHS3 at MMIO 0x3130000 (irq = 65, base_baud = 0) is a TEGRT
[    8.558294] tegra23x-qspi 3270000.spi: Node prod has invalid entries
[    8.564870] tegra23x-qspi 3270000.spi: Node prod: Child has not proper setting
[    8.572331] tegra23x-qspi 3270000.spi: Node spi: Faild to read the Prod Setting.
[    8.579998] tegra23x-qspi 3270000.spi: Prod settings list not found
[    8.616961] tegra-xusb 3610000.xhci: Firmware timestamp: 2021-12-01 05:27:10 UTC, Versione
[    8.626771] tegra-xusb 3610000.xhci: xHCI Host Controller
[    8.632356] tegra-xusb 3610000.xhci: new USB bus registered, assigned bus number 1
[    8.641005] tegra-xusb 3610000.xhci: hcc params 0x0180ff05 hci version 0x120 quirks 0x0000
[    8.650654] tegra-xusb 3610000.xhci: irq 235, io mem 0x03610000
[    8.657109] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.10
[    8.665664] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.673118] usb usb1: Product: xHCI Host Controller
[    8.678154] usb usb1: Manufacturer: Linux 5.10.104-tegra xhci-hcd
[    8.684456] usb usb1: SerialNumber: 3610000.xhci
[    8.689802] hub 1-0:1.0: USB hub found
[    8.693701] hub 1-0:1.0: 4 ports detected
[    8.698079] tegra-xusb 3610000.xhci: xHCI Host Controller
[    8.703660] tegra-xusb 3610000.xhci: new USB bus registered, assigned bus number 2
[    8.711480] tegra-xusb 3610000.xhci: Host supports USB 3.1 Enhanced SuperSpeed
[    8.719030] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.10
[    8.727584] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.735053] usb usb2: Product: xHCI Host Controller
[    8.740107] usb usb2: Manufacturer: Linux 5.10.104-tegra xhci-hcd
[    8.746412] usb usb2: SerialNumber: 3610000.xhci
[    8.752028] hub 2-0:1.0: USB hub found
[    8.755934] hub 2-0:1.0: 4 ports detected
[    8.794053] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 2007000z
[    8.808110] cpufreq: cpufreq_online: CPU4: Running at unlisted initial frequency: 2069000z
[    8.822251] cpufreq: cpufreq_online: CPU8: Running at unlisted initial frequency: 2011000z
[    8.834858] tegra194-cpufreq e000000.ccplex: probed with ICC
[    8.842715] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[    8.844926] sdhci-tegra 3400000.sdhci: Got CD GPIO
[    8.850051] tegra194-pcie 14100000.pcie: host bridge /pcie@14100000 ranges:
[    8.853461] mmc0: CQHCI version 5.10
[    8.862976] irq: IRQ302: trimming hierarchy from :interrupt-controller@f400000-1
[    8.865020] tegra194-pcie 14100000.pcie:       IO 0x0030100000..0x00301fffff -> 0x00301000
[    8.872691] sdhci-tegra 3400000.sdhci: wakeup init done, cdirq 302
[    8.881355] tegra194-pcie 14100000.pcie:      MEM 0x20a8000000..0x20afffffff -> 0x00400000
[    8.881551] mmc0: SDHCI controller on 3460000.sdhci [3460000.sdhci] using ADMA 64-bit
[    8.904496] tegra194-pcie 14100000.pcie:      MEM 0x2080000000..0x20a7ffffff -> 0x20800000
[    8.961218] usb 1-3: new full-speed USB device number 2 using tegra-xusb
[    8.977016] mmc0: Command Queue Engine enabled
[    8.981598] mmc0: periodic cache flush enabled
[    8.986168] mmc0: new HS400 Enhanced strobe MMC card at address 0001
[    8.993056] mmcblk0: mmc0:0001 G1M15M 59.3 GiB 
[    8.997911] mmcblk0rpmb: mmc0:0001 G1M15M partition 3 4.00 MiB, chardev (511:0)
[    9.014469]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
[    9.021219] tegra194-pcie 14100000.pcie: Link up
[    9.027825] tegra194-pcie 14100000.pcie: PCI host bridge to bus 0001:00
[    9.034659] pci_bus 0001:00: root bus resource [bus 00-ff]
[    9.040316] pci_bus 0001:00: root bus resource [io  0x0000-0xfffff] (bus address [0x30100)
[    9.050186] pci_bus 0001:00: root bus resource [mem 0x20a8000000-0x20afffffff] (bus addre)
[    9.061050] pci_bus 0001:00: root bus resource [mem 0x2080000000-0x20a7ffffff pref]
[    9.069007] pci 0001:00:00.0: [10de:229e] type 01 class 0x060400
[    9.075368] pci 0001:00:00.0: PME# supported from D0 D3hot
[    9.085394] pci 0001:01:00.0: [10ec:c822] type 00 class 0x028000
[    9.091736] pci 0001:01:00.0: reg 0x10: [io  0x0000-0x00ff]
[    9.097681] pci 0001:01:00.0: reg 0x18: [mem 0x20a8000000-0x20a800ffff 64bit]
[    9.106132] pci 0001:01:00.0: supports D1 D2
[    9.110541] pci 0001:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    9.121511] pci 0001:00:00.0: BAR 14: assigned [mem 0x20a8000000-0x20a80fffff]
[    9.128964] pci 0001:00:00.0: BAR 13: assigned [io  0x1000-0x1fff]
[    9.129386] usb 1-3: New USB device found, idVendor=13d3, idProduct=3549, bcdDevice= 0.00
[    9.135346] pci 0001:01:00.0: BAR 2: assigned [mem 0x20a8000000-0x20a800ffff 64bit]
[    9.143776] usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    9.143777] usb 1-3: Product: Bluetooth Radio
[    9.143778] usb 1-3: Manufacturer: Realtek
[    9.143779] usb 1-3: SerialNumber: 00e04c000001
[    9.172551] pci 0001:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[    9.178877] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    9.184261] pci 0001:00:00.0:   bridge window [io  0x1000-0x1fff]
[    9.190540] pci 0001:00:00.0:   bridge window [mem 0x20a8000000-0x20a80fffff]
[    9.197919] pci 0001:00:00.0: Max Payload Size set to  128/ 256 (was  256), Max Read Rq  2
[    9.206690] pci 0001:01:00.0: Max Payload Size set to  128/ 128 (was  128), Max Read Rq  2
[    9.215630] pcieport 0001:00:00.0: Adding to iommu group 10
[    9.221611] pcieport 0001:00:00.0: PME: Signaling with IRQ 51
[    9.227767] pcieport 0001:00:00.0: AER: enabled with IRQ 51
[    9.234322] tegra194-pcie 14160000.pcie: Using GICv2m MSI allocator
[    9.241799] tegra194-pcie 14160000.pcie: host bridge /pcie@14160000 ranges:
[    9.248987] tegra194-pcie 14160000.pcie:       IO 0x0036100000..0x00361fffff -> 0x00361000
[    9.254123] usb 2-3: new SuperSpeedPlus Gen 2 USB device number 2 using tegra-xusb
[    9.257696] tegra194-pcie 14160000.pcie:      MEM 0x2428000000..0x242fffffff -> 0x00400000
[    9.274190] tegra194-pcie 14160000.pcie:      MEM 0x2140000000..0x2427ffffff -> 0x21400000
[    9.303120] usb 2-3: New USB device found, idVendor=0bda, idProduct=0420, bcdDevice= 1.40
[    9.311552] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    9.318906] usb 2-3: Product: 4-Port USB 3.0 Hub
[    9.323655] usb 2-3: Manufacturer: Generic
[    9.338425] hub 2-3:1.0: USB hub found
[    9.344459] hub 2-3:1.0: 4 ports detected
[    9.389367] tegra194-pcie 14160000.pcie: Link up
[    9.395091] tegra194-pcie 14160000.pcie: PCI host bridge to bus 0004:00
[    9.401919] pci_bus 0004:00: root bus resource [bus 00-ff]
[    9.407574] pci_bus 0004:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x36)
[    9.417714] pci_bus 0004:00: root bus resource [mem 0x2428000000-0x242fffffff] (bus addre)
[    9.421358] usb 1-4: new high-speed USB device number 3 using tegra-xusb
[    9.428571] pci_bus 0004:00: root bus resource [mem 0x2140000000-0x2427ffffff pref]
[    9.443398] pci 0004:00:00.0: [10de:229c] type 01 class 0x060400
[    9.449739] pci 0004:00:00.0: PME# supported from D0 D3hot
[    9.459392] pci 0004:01:00.0: [144d:a809] type 00 class 0x010802
[    9.465763] pci 0004:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
[    9.478004] pci 0004:00:00.0: BAR 14: assigned [mem 0x2428000000-0x24280fffff]
[    9.485459] pci 0004:01:00.0: BAR 0: assigned [mem 0x2428000000-0x2428003fff 64bit]
[    9.493441] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[    9.498817] pci 0004:00:00.0:   bridge window [mem 0x2428000000-0x24280fffff]
[    9.506187] pci 0004:00:00.0: Max Payload Size set to  256/ 256 (was  256), Max Read Rq  2
[    9.514950] pci 0004:01:00.0: Max Payload Size set to  256/ 256 (was  128), Max Read Rq  2
[    9.523896] pcieport 0004:00:00.0: Adding to iommu group 11
[    9.529864] pcieport 0004:00:00.0: PME: Signaling with IRQ 53
[    9.536022] pcieport 0004:00:00.0: AER: enabled with IRQ 53
[    9.542083] nvme 0004:01:00.0: Adding to iommu group 11
[    9.547909] nvme nvme0: pci function 0004:01:00.0
[    9.552839] nvme 0004:01:00.0: enabling device (0000 -> 0002)
[    9.558793] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[    9.564466] nvme nvme0: Shutdown timeout set to 8 seconds
[    9.570918] vdd-3v3-pcie: supplied by vdd-3v3-sys
[    9.575962] avdd-cam-2v8: supplied by vdd-3v3-sys
[    9.580906] vdd-dp: supplied by vdd-3v3-sys
[    9.585365] vdd-3v3-sd: supplied by vdd-3v3-sys
[    9.587036] nvme nvme0: allocated 64 MiB host memory buffer.
[    9.590461] dsi-vdd-1v8-bl-en: supplied by vdd-1v8-sys
[    9.604221] tegra194-isp5 14800000.isp: initialized
[    9.616077] usb 1-4: New USB device found, idVendor=0bda, idProduct=5420, bcdDevice= 1.40
[    9.618730] tegra194-vi5 13e40000.host1x:vi0@15c00000: initialized
[    9.624510] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    9.631012] (NULL device *): fops function table already registered
[    9.638734] usb 1-4: Product: 4-Port USB 2.0 Hub
[    9.638737] usb 1-4: Manufacturer: Generic
[    9.646105] tegra194-vi5 13e40000.host1x:vi1@14c00000: initialized
[    9.652002] hub 1-4:1.0: USB hub found
[    9.653218] nvme nvme0: 12/0/0 default/read/poll queues
[    9.654878] tegra194-pcie 141a0000.pcie: Using GICv2m MSI allocator
[    9.655078] sdhci-tegra 3400000.sdhci: Got CD GPIO
[    9.661129]  nvme0n1: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10
[    9.664617] hub 1-4:1.0: 4 ports detected
[    9.668523] sdhci-tegra 3400000.sdhci: wakeup init done, cdirq 302
[    9.778613] tegra194-pcie 141a0000.pcie: host bridge /pcie@141a0000 ranges:
[    9.785806] tegra194-pcie 141a0000.pcie:       IO 0x003a100000..0x003a1fffff -> 0x003a1000
[    9.794499] tegra194-pcie 141a0000.pcie:      MEM 0x2b28000000..0x2b2fffffff -> 0x00400000
[    9.803190] tegra194-pcie 141a0000.pcie:      MEM 0x2740000000..0x2b27ffffff -> 0x27400000
[   10.389363] usb 1-4.2: new full-speed USB device number 4 using tegra-xusb
[   10.508726] usb 1-4.2: New USB device found, idVendor=046d, idProduct=c52b, bcdDevice=24.1
[   10.517331] usb 1-4.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   10.524856] usb 1-4.2: Product: USB Receiver
[   10.529257] usb 1-4.2: Manufacturer: Logitech
[   10.539783] input: Logitech USB Receiver as /devices/platform/3610000.xhci/usb1/1-4/1-4.20
[   10.609825] hid-generic 0003:046D:C52B.0001: input,hidraw0: USB HID v1.11 Keyboard [Logit0
[   10.626916] input: Logitech USB Receiver Mouse as /devices/platform/3610000.xhci/usb1/1-41
[   10.640204] input: Logitech USB Receiver Consumer Control as /devices/platform/3610000.xh2
[   10.712318] mmc1: SDHCI controller on 3400000.sdhci [3400000.sdhci] using ADMA 64-bit
[   10.713435] input: Logitech USB Receiver System Control as /devices/platform/3610000.xhci3
[   10.734479] hid-generic 0003:046D:C52B.0002: input,hiddev96,hidraw1: USB HID v1.11 Mouse 1
[   10.751742] hid-generic 0003:046D:C52B.0003: hiddev97,hidraw2: USB HID v1.11 Device [Logi2
[   10.918680] tegra194-pcie 141a0000.pcie: Phy link never came up
[   10.924856] tegra194-pcie 141a0000.pcie: PCI host bridge to bus 0005:00
[   10.931665] pci_bus 0005:00: root bus resource [bus 00-ff]
[   10.937308] pci_bus 0005:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x3a)
[   10.947428] pci_bus 0005:00: root bus resource [mem 0x2b28000000-0x2b2fffffff] (bus addre)
[   10.958255] pci_bus 0005:00: root bus resource [mem 0x2740000000-0x2b27ffffff pref]
[   10.966185] pci 0005:00:00.0: [10de:229a] type 01 class 0x060400
[   10.972520] pci 0005:00:00.0: PME# supported from D0 D3hot
[   10.985674] pci 0005:00:00.0: PCI bridge to [bus 01-ff]
[   10.991071] pci 0005:00:00.0: Max Payload Size set to  256/ 256 (was  256), Max Read Rq  2
[   11.000020] pcieport 0005:00:00.0: Adding to iommu group 12
[   11.005983] pcieport 0005:00:00.0: PME: Signaling with IRQ 55
[   11.012262] pcieport 0005:00:00.0: AER: enabled with IRQ 55
[   11.018235] pci_bus 0005:01: busn_res: [bus 01-ff] is released
[   11.024337] pci 0005:00:00.0: Removing from iommu group 12
[   11.030004] pci_bus 0005:00: busn_res: [bus 00-ff] is released
[   11.037820] nvethernet 6810000.ethernet: Adding to iommu group 52
[   11.044479] nvethernet 6810000.ethernet: failed to read skip mac reset flag, default 0
[   11.052658] nvethernet 6810000.ethernet: failed to read MDIO address
[   11.059222] nvethernet 6810000.ethernet: setting to default DMA bit mask
[   11.066153] nvethernet 6810000.ethernet: failed to read UPHY GBE mode- default to 10G
[   11.464060] nvethernet 6810000.ethernet: Ethernet MAC address: 48:b0:2d:82:7a:46
[   11.472354] nvethernet 6810000.ethernet: Macsec not enabled
[   11.478100] nvethernet 6810000.ethernet: Macsec: Reduced MTU: 1466 Max: 16383
[   11.489277] nvethernet 6810000.ethernet: eth0 (HW ver: 31) created with 10 DMA channels
[   11.497959] irq: IRQ317: trimming hierarchy from :pmc@c360000
[   11.503951] irq: IRQ318: trimming hierarchy from :interrupt-controller@f400000-1
[   11.511652] irq: IRQ319: trimming hierarchy from :pmc@c360000
[   11.517614] input: gpio-keys as /devices/platform/gpio-keys/input/input5
[   11.525562] tegra-se-nvhost 15810000.se: Adding to iommu group 53
[   11.532326] tegra-se-nvhost 15810000.se: initialized
[   11.548095] tegra-se-nvhost 15810000.se: tegra_se_probe: complete
[   11.554474] tegra-se-nvhost 15820000.se: Adding to iommu group 54
[   11.561124] tegra-se-nvhost 15820000.se: initialized
[   11.568347] tegra-se-nvhost 15820000.se: tegra_se_probe: complete
[   11.574723] tegra-se-nvhost 15840000.se: Adding to iommu group 55
[   11.581287] tegra-se-nvhost 15840000.se: initialized
[   11.588904] tegra-se-nvhost 15840000.se: tegra_se_probe: complete
[   11.595720] tegra_actmon d230000.actmon: in actmon_register()...
[   11.602154] tegra_actmon d230000.actmon: bwmgr_disable = 1
[   11.608593] tegra_actmon d230000.actmon: initialization Completed for the device mc_all
[   11.617022] hvc_sysfs: hypervisor is not present
��WARNING: clock_disable: clk_power_ungate on gated domain 35 for gpusysclk
WARNING: clock_disable: clk_power_ungate on gated domain 35 for gpc1clk
WARNING: clock_disable: clk_power_ungate on gated domain 35 for gpc0clk
��[   11.753034] ALSA device list:
[   11.756093]   No soundcards found.
[   11.760154] Freeing unused kernel memory: 3968K
[   11.764889] Run /init as init process
[   11.780171] Root device found: nvme0n1p1
[   11.784690] Found dev node: /dev/nvme0n1p1
[   11.881990] EXT4-fs (nvme0n1p1): 3 orphan inodes deleted
[   11.887465] EXT4-fs (nvme0n1p1): recovery complete
[   11.895637] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null)
[   11.905235] Rootfs mounted over nvme0n1p1
[   11.917599] Switching from initrd to actual rootfs
[   11.969712] systemd[1]: System time before build time, advancing clock.
[   11.999097] systemd[1]: systemd 245.4-4ubuntu3.19 running in system mode. (+PAM +AUDIT +S)
[   12.022824] systemd[1]: Detected architecture arm64.
[   12.050387] systemd[1]: Set hostname to <surveily-orin-01>.
[   12.179809] systemd[1]: system-modprobe.slice: unit configures an IP firewall, but the lo.
[   12.192896] systemd[1]: (This warning is only shown for the first unit using IP firewalli)
[   12.202602] systemd[1]: Created slice system-modprobe.slice.
[   12.208917] systemd[1]: Created slice system-serial\x2dgetty.slice.
[   12.215770] systemd[1]: Created slice User and Session Slice.
[   12.221817] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[   12.229902] systemd[1]: Set up automount Arbitrary Executable File Formats File System Au.
[   12.239681] systemd[1]: Reached target User and Group Name Lookups.
[   12.246196] systemd[1]: Reached target Slices.
[   12.250860] systemd[1]: Reached target Mounting snaps.
[   12.256214] systemd[1]: Reached target Swap.
[   12.260870] systemd[1]: Listening on RPCbind Server Activation Socket.
[   12.267747] systemd[1]: Listening on Syslog Socket.
[   12.272907] systemd[1]: Listening on fsck to fsckd communication Socket.
[   12.279896] systemd[1]: Listening on initctl Compatibility Named Pipe.
[   12.286854] systemd[1]: Listening on Journal Audit Socket.
[   12.292634] systemd[1]: Listening on Journal Socket (/dev/log).
[   12.298886] systemd[1]: Listening on Journal Socket.
[   12.304163] systemd[1]: Listening on udev Control Socket.
[   12.309832] systemd[1]: Listening on udev Kernel Socket.
[   12.316444] systemd[1]: Mounting Huge Pages File System...
[   12.323186] systemd[1]: Mounting POSIX Message Queue File System...
[   12.330662] systemd[1]: Mounting NFSD configuration filesystem...
[   12.337972] systemd[1]: Mounting RPC Pipe File System...
[   12.344635] systemd[1]: Mounting Kernel Debug File System...
[   12.344892] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[   12.358084] systemd[1]: Mounting Kernel Trace File System...
[   12.365367] systemd[1]: Starting Journal Service...
[   12.370552] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS b.
[   12.381274] systemd[1]: Starting Set the console keyboard layout...
[   12.388878] systemd[1]: Starting Create list of static device nodes for the current kerne.
[   12.398928] systemd[1]: Starting Load Kernel Module chromeos_pstore...
[   12.405840] systemd[1]: Condition check resulted in Load Kernel Module drm being skipped.
[   12.415269] systemd[1]: Starting Load Kernel Module efi_pstore...
[   12.422655] systemd[1]: Starting Load Kernel Module pstore_blk...
[   12.430006] systemd[1]: Starting Load Kernel Module pstore_zone...
[   12.437397] systemd[1]: Starting Load Kernel Module ramoops...
[   12.444635] systemd[1]: Starting NVIDIA specific first-boot udev script...
[   12.452096] pstore: Using crash dump compression: deflate
[   12.453090] systemd[1]: Started Nameserver information manager.
[   12.464947] printk: console [ramoops-1] enabled
[   12.469647] pstore: Registered ramoops as persistent store backend
[   12.469650] ramoops: using 0x200000@0x82c200000, ecc: 0
[   12.481591] systemd[1]: Reached target Network (Pre).
[   12.487826] systemd[1]: Condition check resulted in Set Up Additional Binary Formats bein.
[   12.497178] systemd[1]: Condition check resulted in File System Check on Root Device bein.
[   12.508038] systemd[1]: Starting Load Kernel Modules...
[   12.514443] systemd[1]: Starting Remount Root and Kernel File Systems...
[   12.521687] EXT4-fs (nvme0n1p1): re-mounted. Opts: (null)
[   12.522320] nvmap_heap_init: nvmap_heap_init: created heap block cache
[   12.522791] systemd[1]: Starting udev Coldplug all Devices...
[   12.539994] tegra-carveouts tegra-carveouts: fsi :dma coherent mem declare 0x000000082f006
[   12.542008] systemd[1]: Started Journal Service.
[   12.549593] tegra-carveouts tegra-carveouts: assigned reserved memory node fsi-carveout
[   12.549599] tegra-carveouts tegra-carveouts: vpr :dma coherent mem declare 0x0000000849802
[   12.549600] tegra-carveouts tegra-carveouts: assigned reserved memory node vpr-carveout
[   12.549604] nvmap_page_pool_init: Total RAM pages: 7817180
[   12.549605] nvmap_page_pool_init: nvmap page pool size: 977147 pages (3816 MB)
[   12.549652] nvmap_background_zero_thread: PP zeroing thread starting.
[   12.549738] misc nvmap: created heap vpr base 0x0000000849800000 size (892928KiB)
[   12.552695] misc nvmap: created heap fsi base 0x000000082f000000 size (16384KiB)
[   12.597891] nvgpu: 17000000.ga10b          nvgpu_nvhost_syncpt_init:135  [INFO]  syncpt_u0
[   12.597891] 
[   12.649705] systemd-journald[313]: Received client request to flush runtime journal.
[   12.901198] nvidia: loading out-of-tree module taints kernel.
[   13.106790] 1 fixed-partitions partitions found on MTD device spi6.0
[   13.113844] Creating 1 MTD partitions on "spi6.0":
[   13.121920] 0x000000000000-0x000004000000 : "Whole_flash0"
[   13.146680] OF: graph: no port node found in /i2c@c240000/ucsi_ccg@8/connector@0
[   13.159083] gpio gpiochip0: (tegra234-gpio): gpiochip_lock_as_irq: tried to flag a GPIO sQ
[   13.169848] gpio gpiochip0: (tegra234-gpio): unable to lock HW IRQ 126 for IRQ
[   13.177413] genirq: Failed to request resources for 1-0008 (irq 323) on irqchip gpio
[   13.185550] ucsi_ccg 1-0008: request_threaded_irq failed - -5
[   13.197445] ucsi_ccg: probe of 1-0008 failed with error -5
[   13.740689] using random self ethernet address
[   13.745395] using random host ethernet address
[   13.923090] using random self ethernet address
[   13.927700] using random host ethernet address
[   14.942264] Bridge firewalling registered
[   15.824093] audit: type=1326 audit(1675669663.897:2): auid=4294967295 uid=0 gid=0 ses=4290
[   16.504524] audit: type=1326 audit(1675669664.577:3): auid=4294967295 uid=0 gid=0 ses=4290
[   16.530659] audit: type=1326 audit(1675669664.605:4): auid=4294967295 uid=0 gid=0 ses=4290

Ubuntu 20.04.5 LTS surveily-orin-01 ttyTCU0

surveily-orin-01 login: [   99.458167] BUG: scheduling while atomic: swapper/0/0/0x00000102
[   99.464906] WARNING: CPU: 0 PID: 0 at kernel/rcu/tree_plugin.h:297 rcu_note_context_switc0
[   99.474909] ---[ end trace 77006b1ab80bd032 ]---
[   99.480037] Unable to handle kernel paging request at virtual address ffffd293459f08c0
[   99.488296] Mem abort info:
[   99.491560]   ESR = 0x96000005
[   99.494849]   EC = 0x25: DABT (current EL), IL = 32 bits
[   99.500439]   SET = 0, FnV = 0
[   99.503622]   EA = 0, S1PTW = 0
[   99.506871] Data abort info:
[   99.509864]   ISV = 0, ISS = 0x00000005
[   99.513852]   CM = 0, WnR = 0
[   99.516926] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000763fd2000
[   99.523849] [ffffd293459f08c0] pgd=000000082c7ff003, p4d=000000082c7ff003, pud=00000000000
[   99.532849] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[   99.538593] Modules linked in: nvidia_modeset(O) fuse esp4 xt_conntrack xt_MASQUERADE nf_e
[   99.538716]  userspace_alert tegra_bpmp_thermal snd_hda_codec_hdmi snd_hda_tegra snd_hda_s
[   99.650410] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W  O      5.10.104-tegra #1
[   99.658658] Hardware name:  /, BIOS 1.0-d7fb19b 08/10/2022
[   99.664310] pstate: 00400009 (nzcv daif +PAN -UAO -TCO BTYPE=--)
[   99.670501] pc : cpu_logical_map+0x28/0x40
[   99.674727] lr : cpu_logical_map+0x1c/0x40
[   99.678948] sp : ffff800010003880
[   99.682365] x29: ffff800010003850 x28: ffffd292c5c67d40 
[   99.687842] x27: ffff1927fb602c00 x26: ffff192851797c62 
[   99.693321] x25: ffff1927fb602ad4 x24: ffff1927fb602c28 
[   99.698796] x23: ffff8000100038c0 x22: ffffd292c432ba7c 
[   99.704267] x21: ffff800010003870 x20: ffffd292c430f9f0 
[   99.709746] x19: ffff800010003860 x18: 0000000000000060 
[   99.715217] x17: 0000000000018021 x16: ffffd292c3cf0eac 
[   99.720696] x15: ffffd292c59d2bf0 x14: 0000000000000001 
[   99.726175] x13: ffff8000100eb9c8 x12: 0000000000000002 
[   99.731645] x11: 0000000000000040 x10: ffffd292c59e2250 
[   99.737114] x9 : ffffd292c59e2248 x8 : 0000038de8d75934 
[   99.742582] x7 : 000d4f53d47019c4 x6 : 00000000000002e6 
[   99.748057] x5 : 0000000000000017 x4 : 0000000000000500 
[   99.753525] x3 : 00000000000b38ce x2 : ffffd292c3e7e090 
[   99.758999] x1 : ffffd292c59d45c0 x0 : 0000000000000000 
[   99.764474] Call trace:
[   99.766989]  cpu_logical_map+0x28/0x40
[   99.770859] Code: 940050f1 9000e7c1 9116a021 91006021 (f873d820) 
[   99.777139] ---[ end trace 77006b1ab80bd033 ]---
[   99.795396] Kernel panic - not syncing: Oops: Fatal exception
[   99.801323] SMP: stopping secondary CPUs
[   99.805533] Kernel Offset: 0x5292b3cc0000 from 0xffff800010000000
[   99.811820] PHYS_OFFSET: 0xffffe6d940000000
[   99.816131] CPU features: 0x0040006,4a80aa38
[   99.820533] Memory Limit: none
[   99.837214] ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---

The above crash happens the second I run an rsync command to pull a large file over VPN. Sometimes it happens a few minutes into the download.

Could you provide us the detailed steps of how to reproduce your issue?

And we are sure this issue is related to VPN now, right? When VPN connection is absent, will you still hit erorr?

It only crashes when the VPN traffic picks up. LAN transfers work fine. I am using the apt vanilla l2tp.

Another log, different error:


Ubuntu 20.04.5 LTS surveily-orin-01 ttyTCU0

surveily-orin-01 login: [   94.654170] BUG: scheduling while atomic: k3s-server/2375/0x000001
[   94.660938] WARNING: CPU: 0 PID: 2375 at kernel/rcu/tree_plugin.h:297 rcu_note_context_sw0
[   94.670842] ---[ end trace 16cbff5af3e3fa49 ]---
[   94.675691] WARNING: CPU: 0 PID: 0 at kernel/rcu/tree.c:624 rcu_eqs_enter.isra.0+0x128/0x0
[   94.675781] Kernel panic - not syncing: stack-protector: Kernel stack is corrupted in: __0
[   94.684381] Modules linked in: nvidia_modeset(O) fuse esp4
[   94.695458] CPU: 8 PID: 2375 Comm: k3s-server Tainted: G        W  O      5.10.104-tegra 1
[   94.695463]  xt_conntrack
[   94.701131] Hardware name:  /, BIOS 1.0-d7fb19b 08/10/2022
[   94.709727]  xt_MASQUERADE nf_conntrack_netlink
[   94.712439] Call trace:
[   94.718078]  nfnetlink xt_addrtype
[   94.722767]  dump_backtrace+0x0/0x1d0
[   94.725275]  iptable_filter iptable_nat
[   94.728797] SMP: stopping secondary CPUs
[   94.732564]  nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c br_netfilter lzon
[   94.740653]  nvgpu nvmap ramoops reed_solomon nfsd ip_tables x_tables
[   94.839588] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W  O      5.10.104-tegra #1
[   94.847831] Hardware name:  /, BIOS 1.0-d7fb19b 08/10/2022
[   94.853471] pstate: 20c00089 (nzCv daIf +PAN +UAO -TCO BTYPE=--)
[   94.859652] pc : rcu_eqs_enter.isra.0+0x128/0x130
[   94.864494] lr : rcu_idle_enter+0x18/0x20
[   94.868629] sp : ffffa7a7d9b63df0
[   94.872044] x29: ffffa7a7d9b63df0 x28: 0000000514264340 
[   94.877515] x27: 0000000000000000 x26: ffff22b8bef65880 
[   94.882984] x25: 0000000000000000 x24: 000000160b1c4d80 
[   94.888457] x23: 0000000000000000 x22: ffff22b8bef65880 
[   94.893925] x21: ffff22b8ec4baa40 x20: ffff22b8bef65898 
[   94.899398] x19: ffffa7a7d988ca40 x18: 0000000000000060 
[   94.904867] x17: 0000000000000000 x16: ffffa7a7d7e90eac 
[   94.910333] x15: 0000000000000228 x14: 000000000000029b 
[   94.915807] x13: 0000000000000001 x12: 0000000000000500 
[   94.921275] x11: 0000000000000000 x10: 0000000000000a80 
[   94.926744] x9 : ffffa7a7d9b63e00 x8 : ffffa7a7d9b73160 
[   94.932213] x7 : 0000000000000004 x6 : 000000002b438f7f 
[   94.937687] x5 : 00ffffffffffffff x4 : 0000000000000015 
[   94.943156] x3 : 00000000000ec420 x2 : 4000000000000002 
[   94.948624] x1 : 4000000000000000 x0 : ffff22b8ec4bca40 
[   94.954098] Call trace:
[   94.956615]  rcu_eqs_enter.isra.0+0x128/0x130
[   94.961102]  rcu_idle_enter+0x18/0x20
[   94.964872]  cpuidle_enter_state+0x330/0x410
[   94.969272]  cpuidle_enter+0x40/0x60
[   94.972955]  call_cpuidle+0x44/0x80
[   94.976550]  do_idle+0x208/0x270
[   94.979878]  cpu_startup_entry+0x2c/0x70
[   94.983913]  rest_init+0xdc/0xe8
[   94.987245]  arch_call_rest_init+0x18/0x20
[   94.991462]  start_kernel+0x514/0x54c
[   94.995230] ---[ end trace 16cbff5af3e3fa4a ]---
[   95.000017] ------------[ cut here ]------------
[   95.004777] WARNING: CPU: 0 PID: 0 at kernel/sched/core.c:1999 set_task_cpu+0x234/0x240
[   95.013020] Modules linked in: nvidia_modeset(O) fuse esp4 xt_conntrack xt_MASQUERADE nf_t
[   95.013102]  tegra_bpmp_thermal tegra210_adma nvadsp qspi_mtd snd_hda_tegra snd_hda_codecs
[   95.124811] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G        W  O      5.10.104-tegra #1
[   95.133055] Hardware name:  /, BIOS 1.0-d7fb19b 08/10/2022
[   95.138696] pstate: 20400089 (nzCv daIf +PAN -UAO -TCO BTYPE=--)
[   95.144871] pc : set_task_cpu+0x234/0x240
[   95.149003] lr : set_task_cpu+0x24/0x240
[   95.149011] Internal error: SP/PC alignment exception: 8a000000 [#1] PREEMPT SMP
[   95.153040] sp : ffff800010003d50
[   95.160673] Modules linked in:
[   95.164086] x29: ffff800010003d50 x28: ffff22b8ec4b0200 
[   95.167240]  nvidia_modeset(O) fuse
[   95.172711] x27: ffff80002b9bbc80 x26: ffffa7a7d9b693f8 
[   95.176314]  esp4
[   95.176317] 
[   95.181797]  xt_conntrack
[   95.183773] x25: ffffa7a7d988bd40 x24: ffff22b25fc60814 
[   95.185309]  xt_MASQUERADE
[   95.187997] 
[   95.188000] x23: 0000000000000080 
[   95.193481]  nf_conntrack_netlink
[   95.196262] x22: 0000000000000004 
[   95.197798]  nfnetlink
[   95.201298] x21: ffffa7a7d9b68998 x20: 0000000000000003 
[   95.204720]  xt_addrtype
[   95.208220] 
[   95.208224] x19: ffff22b25fc60000 
[   95.210645]  iptable_filter
[   95.216113] x18: 0000000000000060 
[   95.218724]  iptable_nat
[   95.220252] x17: 0000000000000000 x16: ffffa7a7d7e90eac 
[   95.223761]  nf_nat
[   95.226631] 
[   95.226634] x15: 0000000000000228 
[   95.230140]  nf_conntrack
[   95.232741] x14: ffffffffa9a7929f 
[   95.238221]  nf_defrag_ipv6
[   95.240371] x13: 00000000ffff36ff x12: 00000000000046c1 
[   95.241907]  nf_defrag_ipv4
[   95.245408] 
[   95.245411] x11: ffffa7a7d9b66000 
[   95.248106]  libcrc32c
[   95.251607] x10: 00001e8e90bb2c58 
[   95.254494]  br_netfilter
[   95.259968] x9 : 000000000001e848 x8 : 0000000000000004 
[   95.262854]  lzo_rle
[   95.264382] 
[   95.264385] x7 : 0000000000000000 
[   95.267891]  lzo_compress
[   95.270309] x6 : 000000002b438f7f 
[   95.273817]  zram
[   95.276506] x5 : 00ffffffffffffff x4 : 0000000000000015 
[   95.281981]  overlay
[   95.284225] 
[   95.284228] x3 : 00000000000ec420 
[   95.285760]  bnep
[   95.289261] x2 : ffffa7a7d801e090 
[   95.291959]  hid_logitech_hidpp
[   95.295459] x1 : 0000000000000003 x0 : 0000000000000020 
[   95.297446]  input_leds
[   95.302912] 
[   95.302915] Call trace:
[   95.305168]  snd_soc_tegra186_dspk
[   95.306699]  set_task_cpu+0x234/0x240
[   95.310203]  snd_soc_tegra210_ope
[   95.312180]  try_to_wake_up+0x1f0/0x720
[   95.312185]  wake_up_process+0x2c/0x40
[   95.315689]  snd_soc_tegra186_asrc
[   95.318925]  hrtimer_wakeup+0x2c/0x40
[   95.324398]  snd_soc_tegra186_arad
[   95.326914]  __hrtimer_run_queues+0x148/0x380
[   95.326918]  hrtimer_interrupt+0xf0/0x250
[   95.328450]  snd_soc_tegra210_iqc
[   95.330968]  arch_timer_handler_phys+0x40/0x50
[   95.330973]  handle_percpu_devid_irq+0x90/0x280
[   95.334476]  snd_soc_tegra210_mvc
[   95.338245]  generic_handle_irq+0x40/0x60
[   95.338250]  __handle_domain_irq+0x70/0xd0
[   95.341666]  snd_soc_tegra210_admaif
[   95.345619]  gic_handle_irq+0x68/0x134
[   95.345623]  el1_irq+0xd0/0x180
[   95.349493]  snd_soc_tegra210_afc
[   95.352997]  cpuidle_enter_state+0xb8/0x410
[   95.353001]  cpuidle_enter+0x40/0x60
[   95.356775]  snd_soc_tegra_pcm
[   95.360277]  call_cpuidle+0x44/0x80
[   95.360281]  do_idle+0x208/0x270
[   95.364774]  snd_soc_tegra210_dmic
[   95.368907]  cpu_startup_entry+0x2c/0x70
[   95.368911]  rest_init+0xdc/0xe8
[   95.372329]  snd_soc_tegra210_adx
[   95.376904]  arch_call_rest_init+0x18/0x20
[   95.376909]  start_kernel+0x514/0x54c
[   95.381579]  snd_soc_tegra210_amx
[   95.384994] ---[ end trace 16cbff5af3e3fa4b ]---
[   95.456076]  snd_soc_tegra210_i2s snd_soc_tegra210_sfc snd_soc_tegra210_mixer hid_logitecs
[   95.515828] CPU: 8 PID: 2375 Comm: k3s-server Tainted: G        W  O      5.10.104-tegra 1
[   95.524429] Hardware name:  /, BIOS 1.0-d7fb19b 08/10/2022
[   95.530082] pstate: ffffa7a7d9120ea8 (NZcV DaIf -PAN -UAO -TCO BTYPE=j-)
[   95.536990] pc : 0xffff8000100034c0
[   95.540583] lr : 0xffff800090003594
[   95.544183] sp : ffff800010003595
[   95.547602] x29: ffffa7a7d84c2b64 x28: ffff800010003390 
[   95.553087] x27: ffff22b1e623c180 x26: ffff22b1e623c228 
[   95.558572] x25: 0000000000098689 x24: ffffa7a7d9b68000 
[   95.564042] x23: ffff22b201474880 x22: ffffa7a7d84c2884 
[   95.569527] x21: ffff8000100033a0 x20: 0000000000000020 
[   95.575010] x19: ffffa7a7d94e704a x18: 0000000000000010 
[   95.580493] x17: 0000000000000000 x16: ffffa7a7d7e83210 
[   95.585968] x15: ffff22b201474df0 x14: ffffffffffffffff 
[   95.591439] x13: ffff800090003077 x12: ffff80001000307f 
[   95.596911] x11: 0101010101010101 x10: ffffffffffffff7f 
[   95.602391] x9 : ffff800010003390 x8 : 207265746c69665f 
[   95.607877] x7 : 656c626174706920 x6 : 0000000000000003 
[   95.613359] x5 : 0000000000000000 x4 : 0000000000000000 
[   95.618843] x3 : 0000000000000002 x2 : ffffa7a7d9f6c1f0 
[   95.624313] x1 : ffff800010003350 x0 : 0000000000000020 
[   95.629797] Call trace:
[   95.632323] Code: ffffffe0 00000000 7fffffff 00000000 (d94e0d2b) 
[   95.638615] ---[ end trace 16cbff5af3e3fa4c ]---
[   95.658119] Kernel panic - not syncing: SP/PC alignment exception: Fatal exception
[   95.665935] Kernel Offset: 0x27a7c7e60000 from 0xffff800010000000
[   95.672227] PHYS_OFFSET: 0xffffdd4f40000000
[   95.676540] CPU features: 0x0040006,4a80aa38
[   95.680947] Memory Limit: none
[   95.698864] ---[ end Kernel panic - not syncing: SP/PC alignment exception: Fatal excepti-
��

I also tried disabling k3s, I get crashes anyway.

I just reproduced the issue 3 times and it crashed the moment I run the rsync over VPN. LAN is fine.

We are now experimenting with OpenVPN. It is not ideal, as our network hardware prefers Wireguard or L2TP.

Prefers meaning its build-in.

BTW, are we testing this on rel-35.2.1 or rel-35.1?

r35.1