L4T 32.4.2 - GPU error on boot

wangzhaofeng,

How about using “modprobe -r” to remove it?

modprobe -r nvgpu
modprobe: FATAL: Module nvgpu is in use.

the same error.

rmmod -f to force remove it?

1 Like

@WayneWWW Yes - the error above is the one that occurs on boot. I actually don’t have the dmesg output of the error that occurs during GPU usage because it is an instant shutoff - no logs or records of what is causing it. It’s as if the power is shut off instantly. Could this be an out of memory issue? You would think it would be a softer shutdown if so, with errors etc…

@drcnyc I think you should clarify your usecase or setup.

We only see this issue when running stress gpu loading work. For example, below issue. Or the case from @wangzhaofeng.

It does not make sense this issue just happens right after the system boot up. Could you share the full dmesg with us?

Also, still need your explanation for your setup.

i find some message when i run dmesg as follows

@wangzhaofeng,

We will set up oceanFFT today and see if we can reproduce this.

@WayneWWW below is the full dmesg output. The system is stable after the error, but when I try to run a GPU-intensive app such as detectnet, it occasionally crashes (instant shutoff) while loading. Sometimes though, it loads and runs fine.

My setup / use case is as follows: dev kit, with 32gb SDHC UHS-1 card, running Balena OS (yocto based docker host) with a multi-container app running inside of it. Most of the containers are small / low utilization, running things such as VPN connectivity or custom lightweight GPIO apps. There are two “substantial” containers, one running nodejs / node-red which is core to our app’s functionality, and one running an xorg / chromium display (which is also where I run detectnet and imagenet from. All containers are using L4T base images built off of Ubuntu 18.04 overlaid with L4T 32.4.2 and with CUDA/cuDNN/TensorRT installed (the latter in the case of the display and nodejs containers only; others just have L4T on Ubuntu).

dmesg output:

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.140-l4t-r32.4.2 (oe-user@oe-host) (gcc version 8.3.0 (GCC) ) #1 SMP PREEMPT Fri Jun 12 09:10:09 UTC 2020
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 32,
[    0.000000] OF: fdt: - 80000000 ,  7ee00000
[    0.000000] OF: fdt: - 100000000 ,  7f200000
[    0.000000] Found tegra_fbmem: 00800000@92cb4000
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node iram-carveout, compatible id nvidia,iram-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fac00000
[    0.000000] On node 0 totalpages: 1039872
[    0.000000]   DMA zone: 8192 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 519168 pages, LIFO batch:31
[    0.000000]   Normal zone: 8136 pages used for memmap
[    0.000000]   Normal zone: 520704 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 25 pages/cpu @ffffffc0fefb1000 s61912 r8192 d32296 u102400
[    0.000000] pcpu-alloc: s61912 r8192 d32296 u102400 alloc=25*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1023544
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1125 core_edp_ma=4000 tegra_fbmem=0x800000@0x92cb4000 is_hdmi_initialised=1  console=ttyS0,115200 console=tty0 fbcon=map:0 net.ifnames=0 root=UUID=ba1eadef-32ef-4b0a-93ee-08dca7534519 ro rootwait  sdhci_tegra.en_boot_part_access=1
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 98304 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 29572(90%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Memory: 3561848K/4159488K available (15102K kernel code, 2910K rwdata, 9636K rodata, 11456K init, 605K bss, 122504K reserved, 475136K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f40000   ( 15104 KB)
[    0.000000]     .rodata : 0xffffff8008f40000 - 0xffffff80098b0000   (  9664 KB)
[    0.000000]       .init : 0xffffff80098b0000 - 0xffffff800a3e0000   ( 11456 KB)
[    0.000000]       .data : 0xffffff800a3e0000 - 0xffffff800a6b7808   (  2911 KB)
[    0.000000]        .bss : 0xffffff800a6b7808 - 0xffffff800a74ee24   (   606 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf03fc8000   (    63 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0ff200000   (  4082 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 64.
[    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller
[    0.000000] t210 clock and reset probe
[    0.000000] tegra-pmc: get_secure_pmc_setting: done secure_pmc=1
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000005] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.001562] Console: colour dummy device 80x25
[    0.002432] console [tty0] enabled
[    0.002473] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=76800)
[    0.002502] pid_max: default: 32768 minimum: 301
[    0.002960] Security Framework initialized
[    0.003195] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.003218] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.004478] ftrace: allocating 46832 entries in 183 pages
[    0.177240] sched-energy: CPU device node has no sched-energy-costs
[    0.177273] ASID allocator initialised with 65536 entries
[    0.215961] tegra-id: chipid=22117.
[    0.216007] tegra-id: opt_subrevision=0.
[    0.216035] Tegra Speedo/IDDQ fuse revision 4
[    0.216051] Tegra: CPU Speedo ID 9, SoC Speedo ID 0, GPU Speedo ID 2
[    0.216068] Tegra: CPU Process ID 0, SoC Process ID 0, GPU Process ID 0
[    0.216084] Tegra: CPU Speedo Value 2034, SoC Speedo Value 1909, GPU Speedo Value 2017
[    0.216106] Tegra: CPU IDDQ Value 1732, SoC IDDQ Value 1860, GPU IDDQ Value 2185
[    0.216143] Tegra Revision: A02 SKU: 0x8f CPU Process: 0 SoC Process: 0
[    0.216172] DTS File Name: /yocto/resin-board/build/tmp/work-shared/jetson-nano/kernel-source/arch/arm64/boot/dts/../../../../nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-b00.dts
[    0.216250] DTB Build time: Jun 12 2020 09:15:26
[    0.265078] CPU1: Booted secondary processor [411fd071]
[    0.296989] CPU2: Booted secondary processor [411fd071]
[    0.328946] CPU3: Booted secondary processor [411fd071]
[    0.329039] Brought up 4 CPUs
[    0.329101] SMP: Total of 4 processors activated.
[    0.329119] CPU features: detected feature: 32-bit EL0 Support
[    0.329398] CPU: All CPU(s) started at EL2
[    0.329430] alternatives: patching kernel code
[    0.337765] devtmpfs: initialized
[    0.373432] Initilizing CustomIPI irq domain
[    0.373833] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.373876] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.385355] pinctrl core: initialized pinctrl subsystem
[    0.385968] OS set in device tree is not L4T.
[    0.386298] regulator-dummy: no parameters
[    0.386541] Initializing plugin-manager
[    0.386637] Plugin module not found
[    0.386884] node /plugin-manager/fragement@0 match with board >=3448-0000-100
[    0.387463] node /plugin-manager/fragment@1 match with board >=3448-0000-101
[    0.389355] node /plugin-manager/fragement@10 match with board >=3448-0000-300
[    0.391732] node /plugin-manager/fragement@11 match with board >=3448-0000-300
[    0.395337] NET: Registered protocol family 16
[    0.402851] pstore: using zlib compression
[    0.403551] console [pstore-1] enabled
[    0.403571] pstore: Registered ramoops as persistent store backend
[    0.403592] ramoops: attached 0x200000@0xb0000000, ecc: 0/0
[    0.413088] cpuidle: using governor menu
[    0.417009] tegra_smmu 70019000.iommu: Loaded Tegra IOMMU driver
[    0.418472] vdso: 2 pages (1 code @ ffffff8008f47000, 1 data @ ffffff800a3e4000)
[    0.418519] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.421627] atomic_pool_init():622: DMA: preallocated 1024 KiB pool for atomic allocations
[    0.423695] Serial: AMBA PL011 UART driver
[    0.424185] tegra_powergate_init: DONE
[    0.424219] DTS File Name: /yocto/resin-board/build/tmp/work-shared/jetson-nano/kernel-source/arch/arm64/boot/dts/../../../../nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-b00.dts
[    0.424257] DTB Build time: Jun 12 2020 09:15:26
[    0.425846] Tegra reboot handler registered.
[    0.430132] iommu: Adding device tegra-carveouts to group 0
[    0.430316] platform tegra-carveouts: domain=ffffffc0f9bda3d8 allocates as[0]=ffffffc0f9bd00a8
[    0.430659] iommu: Adding device smmu_test to group 1
[    0.430724] platform smmu_test: domain=ffffffc0f9bda618 allocates as[0]=ffffffc0f9bd0110
[    0.431492] mc: mapped MMIO address: 0xffffff800802e000 -> 0x70019000
[    0.431571] mc: mapped MMIO address: 0xffffff8008065000 -> 0x7001c000
[    0.431639] mc: mapped MMIO address: 0xffffff8008079000 -> 0x7001d000
[    0.431663] nv-tegra-mc 70019000.mc: No mssnvlink node
[    0.431698] mc-err: mcerr ops are set to t21x
[    0.441469] iommu: Adding device 70090000.xusb to group 2
[    0.441777] iommu: Adding device 70006000.serial to group 3
[    0.441858] platform 70006000.serial: domain=ffffffc0f9be3cd8 allocates as[0]=ffffffc0f9bd0178
[    0.442313] iommu: Adding device 70006040.serial to group 4
[    0.442566] iommu: Adding device 70006200.serial to group 5
[    0.442753] iommu: Adding device sound to group 6
[    0.442817] platform sound: domain=ffffffc0f944d258 allocates as[0]=ffffffc0f9bd01e0
[    0.443330] iommu: Adding device 7000d400.spi to group 7
[    0.443595] iommu: Adding device 7000d600.spi to group 8
[    0.443886] iommu: Adding device 70410000.spi to group 9
[    0.444183] iommu: Adding device 50000000.host1x to group 10
[    0.444457] iommu: Adding device 54080000.vi to group 11
[    0.444845] iommu: Adding device 54600000.isp to group 12
[    0.445139] iommu: Adding device 54680000.isp to group 13
[    0.445407] iommu: Adding device tegradc.0 to group 14
[    0.445471] platform tegradc.0: domain=ffffffc0f9459198 allocates as[0]=ffffffc0f9bd0248
[    0.449669] platform tegradc.0: IOVA linear map 0x0000000092cb4000(800000)
[    0.450081] platform tegradc.0: IOVA linear map 0x00000000d7000000(19000000)
[    0.450390] iommu: Adding device tegradc.1 to group 15
[    0.450454] platform tegradc.1: domain=ffffffc0f94593d8 allocates as[0]=ffffffc0f9bd02b0
[    0.454627] platform tegradc.1: IOVA linear map 0x0000000092cb4000(800000)
[    0.455037] platform tegradc.1: IOVA linear map 0x00000000d7000000(19000000)
[    0.455303] iommu: Adding device 54340000.vic to group 16
[    0.455529] iommu: Adding device 544c0000.nvenc to group 17
[    0.455723] iommu: Adding device 54500000.tsec to group 18
[    0.455920] iommu: Adding device 54100000.tsecb to group 19
[    0.456118] iommu: Adding device 54480000.nvdec to group 20
[    0.456310] iommu: Adding device 54380000.nvjpg to group 21
[    0.457338] iommu: Adding device 546c0000.i2c to group 22
[    0.457657] iommu: Adding device 57000000.gpu to group 23
[    0.457727] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[0]=ffffffc0f9bd0318
[    0.457808] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[1]=ffffffc0f9bd0380
[    0.457868] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[2]=ffffffc0f9bd03e8
[    0.457921] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[3]=ffffffc0f9bd0450
[    0.458737] tegra-pmc 7000e400.pmc: i2c-thermtrip node not found, emergency thermal reset disabled.
[    0.458777] tegra-pmc 7000e400.pmc: scratch reg offset dts data not present
[    0.458804] tegra-pmc: ### PMC reset source: TEGRA_SOFTWARE_RESET
[    0.458824] tegra-pmc: ### PMC reset level: TEGRA_RESET_LEVEL_WARM
[    0.458841] tegra-pmc: ### PMC reset status reg: 0x3
[    0.509250] padctrl padctrl.0: Pad control driver tegra-pmc-padctrl registered
[    0.509286] tegra-pmc: Clear bootloader IO dpd settings
[    0.509319] tegra-pmc 7000e400.pmc: IO padctrl driver initialized
[    0.509403] tegra-pmc 7000e400.pmc: PMC: Successfully configure bootrom reset commands
[    0.509702] iommu: Adding device 70012000.se to group 24
[    0.510409] iommu: Adding device 7000c000.i2c to group 25
[    0.510673] iommu: Adding device 7000c400.i2c to group 26
[    0.510933] iommu: Adding device 7000c500.i2c to group 27
[    0.511186] iommu: Adding device 7000c700.i2c to group 28
[    0.511438] iommu: Adding device 7000d000.i2c to group 29
[    0.511685] iommu: Adding device 7000d100.i2c to group 30
[    0.511968] iommu: Adding device sdhci-tegra.0 to group 31
[    0.512041] platform sdhci-tegra.0: domain=ffffffc0f94ed558 allocates as[0]=ffffffc0f9bd04b8
[    0.512828] iommu: Adding device 700d0000.xudc to group 32
[    0.515770] vdd-ac-bat: 5000 mV
[    0.516108] vdd-5v0-sys: 5000 mV
[    0.516935] vdd-5v0-hdmi: supplied by vdd-5v0-sys
[    0.516992] vdd-5v0-hdmi: 5000 mV
[    0.517394] vdd-1v8-sys: 1800 mV
[    0.517809] vdd-fan: supplied by vdd-5v0-sys
[    0.517858] vdd-fan: 5000 mV
[    0.518225] vdd-usb-vbus: supplied by vdd-5v0-sys
[    0.518275] vdd-usb-vbus: 5000 mV
[    0.518740] vdd-usb-vbus2: 5000 mV
[    0.541775] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.544248] gpio gpiochip0: gpio-line-names specifies 240 line names but there are 256 lines on the chip
[    0.545435] GPIO line 151 (camera-control-output-low) hogged as output/low
[    0.545473] GPIO line 152 (camera-control-output-low) hogged as output/low
[    0.545525] GPIO line 6 (system-suspend-gpio) hogged as output/high
[    0.545866] gpiochip_setup_dev: registered GPIOs 0 to 255 on device: gpiochip0 (tegra-gpio)
[    0.553535] eventlib_kernel: keventlib is initialized, test id: 0
[    0.553849] SCSI subsystem initialized
[    0.554103] libata version 3.00 loaded.
[    0.554297] usbcore: registered new interface driver usbfs
[    0.554362] usbcore: registered new interface driver hub
[    0.554441] usbcore: registered new device driver usb
[    0.558138] max77620 4-003c: PMIC Version OTP:0x35 and ES:0x8
[    0.564314] vdd-core: 600 <--> 1162 mV at 1125 mV
[    0.565218] random: fast init done
[    0.565252] vdd-ddr-1v1: Bringing 1125000uV into 1150000-1150000uV
[    0.568213] vdd-ddr-1v1: 1150 mV
[    0.572214] vdd-pre-reg-1v35: 1350 mV
[    0.576211] vdd-1v8: 1800 mV
[    0.580212] avdd-sys-1v2: 1200 mV
[    0.584210] vdd-pex-1v0: 1050 mV
[    0.584879] vddio-sdmmc-ap: 1800 <--> 3300 mV at 3300 mV
[    0.585435] max77620-ldo3: at 3100 mV
[    0.588215] vdd-rtc: 850 <--> 1100 mV at 1000 mV
[    0.588745] max77620-ldo5: at 3100 mV
[    0.592213] vddio-sdmmc3-ap: 1800 <--> 3300 mV at 2800 mV
[    0.596215] avdd-1v05-pll: 1050 mV
[    0.600212] avdd-io-hdmi-dp: 1050 mV
[    0.602585] GPIO line 505 (spmic-default-output-high) hogged as output/high
[    0.602913] gpiochip_setup_dev: registered GPIOs 504 to 511 on device: gpiochip1 (max77620-gpio)
[    0.603201] max77620 4-003c: max77620 probe successful
[    0.605663] media: Linux media interface: v0.10
[    0.605727] Linux video capture interface: v2.00
[    0.606536] pps_core: LinuxPPS API ver. 1 registered
[    0.606561] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.606595] PTP clock support registered
[    0.609603] tegra210-emc 7001b000.external-memory-controller: validated EMC DFS table
[    0.610312] Advanced Linux Sound Architecture Driver Initialized.
[    0.610930] Bluetooth: Core ver 2.22
[    0.610986] NET: Registered protocol family 31
[    0.611003] Bluetooth: HCI device and connection manager initialized
[    0.611030] Bluetooth: HCI socket layer initialized
[    0.611051] Bluetooth: L2CAP socket layer initialized
[    0.611090] Bluetooth: SCO socket layer initialized
[    0.612913] tegra210_dvfs: no clock found for sdmmc2_ddr
[    0.612960] tegra210_dvfs: no clock found for sdmmc4_ddr
[    0.613026] tegra210_dvfs: no clock found for sdmmc1_ddr
[    0.613056] tegra210_dvfs: no clock found for sdmmc3_ddr
[    0.616512] tegra_dvfs: Unable to get vdd-cpu rail for step info, defering probe
[    0.617779] vdd-3v3-sys: supplied by vdd-5v0-sys
[    0.617906] vdd-3v3-sys: 3300 mV
[    0.618168] vdd-1v8-sys: supplied by vdd-3v3-sys
[    0.620283] vdd-usb-vbus2: supplied by vdd-3v3-sys
[    0.620748] vdd-3v3-sd: supplied by vdd-3v3-sys
[    0.620799] vdd-3v3-sd: 3300 mV
[    0.621647] avdd-io-edp-1v05: supplied by avdd-1v05-pll
[    0.621712] avdd-io-edp-1v05: 1050 mV
[    0.622113] vdd-usb-hub-en: supplied by vdd-1v8-sys
[    0.622167] vdd-usb-hub-en: 5000 mV
[    0.623355] camchar: rtcpu character device driver loaded
[    0.623978] extcon-gpio-states extcon:extcon@1: Cable state:0, cable id:0
[    0.625162] clocksource: Switched to clocksource arch_sys_counter
[    0.665547] VFS: Disk quotas dquot_6.6.0
[    0.665732] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.666217] nvmap_heap_init: nvmap_heap_init: created heap block cache
[    0.666819] dma_declare_coherent_resizable_cma_memory:324: resizable heap=vpr, base=0x00000000d7000000, size=0x19000000
[    0.666870]  dma-vpr: heap size is not multiple of cma_chunk_size heap_info->num_chunks (13) rem_chunk_size(0x1000000)
[    0.667158] cma: enabled page replacement for spfn=d7000, epfn=f0000
[    0.667189] dma_declare_coherent_resizable_cma_memory:373: resizable cma heap=vpr create successful
[    0.667219] tegra-carveouts tegra-carveouts: assigned reserved memory node vpr-carveout
[    0.667271] tegra-carveouts tegra-carveouts: iram :dma coherent mem declare 0x0000000040001000,258048
[    0.667298] tegra-carveouts tegra-carveouts: assigned reserved memory node iram-carveout
[    0.667335] nvmap: nvmap_select_cache_ops() nvmap cache ops set to set/ways
[    0.667360] nvmap_page_pool_init: Total RAM pages: 1009246
[    0.667379] nvmap_page_pool_init: nvmap page pool size: 126155 pages (492 MB)
[    0.667514] nvmap_background_zero_thread: PP zeroing thread starting.
[    0.667799] misc nvmap: created heap iram base 0x0000000040001000 size (252KiB)
[    0.668739] misc nvmap: created heap vpr base 0x00000000d7000000 size (409600KiB)
[    0.676248] thermal thermal_zone0: Registering thermal zone thermal_zone0 for type AO-therm
[    0.676653] thermal thermal_zone1: Registering thermal zone thermal_zone1 for type CPU-therm
[    0.676892] thermal thermal_zone2: Registering thermal zone thermal_zone2 for type GPU-therm
[    0.677098] thermal thermal_zone3: Registering thermal zone thermal_zone3 for type PLL-therm
[    0.677305] thermal thermal_zone4: Registering thermal zone thermal_zone4 for type PMIC-Die
[    0.677584] pre_t19x_iso_plat_init(): iso emc max clk=1600000KHz
[    0.677604] pre_t19x_iso_plat_init(): max_iso_bw=11520000KB
[    0.678042] NET: Registered protocol family 2
[    0.679015] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    0.679250] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    0.679759] TCP: Hash tables configured (established 32768 bind 32768)
[    0.679928] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    0.680024] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    0.680432] NET: Registered protocol family 1
[    0.681226] RPC: Registered named UNIX socket transport module.
[    0.681257] RPC: Registered udp transport module.
[    0.681274] RPC: Registered tcp transport module.
[    0.681292] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.681317] PCI: CLS 0 bytes, default 64
[    0.892322] Trying to unpack rootfs image as initramfs...
[    0.902343] host1x 50000000.host1x: initialized
[    0.904034] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[    0.908046] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.917693] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.919259] 9p: Installing v9fs 9p2000 file system support
[    0.922624] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 240)
[    0.922797] io scheduler noop registered
[    0.923089] io scheduler cfq registered (default)
[    0.925563] gic 702f9000.agic: GIC IRQ controller registered
[    0.928543] iommu: Adding device 702ef000.adsp to group 33
[    0.937241] iommu: Adding device aconnect@702c0000:adsp_audio to group 34
[    0.937402] tegra-aconnect aconnect@702c0000: Tegra ACONNECT bus registered
[    0.937905] tegra-xusb-padctl 7009f000.xusb_padctl: TEGRA_FUSE_SKU_CALIB_0 = 0x7a68411
[    0.937936] tegra-xusb-padctl 7009f000.xusb_padctl: TEGRA_FUSE_USB_CALIB_EXT_0 = 0x4
[    0.939591] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.0, lane = usb2-0, function = xusb
[    0.939716] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.1, lane = usb2-1, function = xusb
[    0.939816] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.2, lane = usb2-2, function = xusb
[    0.940013] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.3, lane = pcie-0, function = pcie-x1
[    0.940114] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.4, lane = pcie-1, function = pcie-x4
[    0.940217] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.5, lane = pcie-2, function = pcie-x4
[    0.940318] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.6, lane = pcie-3, function = pcie-x4
[    0.940428] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.7, lane = pcie-4, function = pcie-x4
[    0.940527] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.8, lane = pcie-5, function = xusb
[    0.940631] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.9, lane = pcie-6, function = xusb
[    0.946595] tegra-pwm 7000a000.pwm: PWM clk cannot sleep in ops
[    0.948219] tegra-dfll-pwm 70110000.pwm: DFLL pwm-rate: 12800000
[    0.950058] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
[    0.951361] tegra-pcie 1003000.pcie: PCIE: Enable power rails
[    0.951731] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
[    0.952854] tegra_camera_platform tegra-camera-platform: tegra_camera_probe:camera_platform_driver probe
[    0.953108] misc tegra_camera_ctrl: tegra_camera_isomgr_register isp_iso_bw=1500000, vi_iso_bw=1500000, max_bw=1500000
[    0.953305] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
[    0.953575] Adding domain tsec-pd to PM domain host1x-pd
[    0.956930] tsec 54500000.tsec: initialized
[    0.958377] tsec 54100000.tsecb: initialized
[    0.959057] Adding domain nvdec-pd to PM domain host1x-pd
[    0.962363] nvdec 54480000.nvdec: initialized
[    0.963365] Adding domain vic03-pd to PM domain host1x-pd
[    0.963570] Adding domain msenc-pd to PM domain host1x-pd
[    0.963752] Adding domain nvjpg-pd to PM domain host1x-pd
[    0.967458] falcon 54340000.vic: initialized
[    0.968732] falcon 544c0000.nvenc: initialized
[    0.970124] falcon 54380000.nvjpg: initialized
[    0.974017] tegra_cec 70015000.tegra_cec: dt=1 start=0x70015000 end=0x70015FFF irq=97
[    0.974082] tegra_cec 70015000.tegra_cec: Enable clock result: 0.
[    0.974132] tegra_cec 70015000.tegra_cec: tegra_cec_init started
[    0.974186] tegradc tegradc.0: disp0 connected to head0->/host1x/sor1
[    0.974326] display board info: id 0x0, fab 0x0
[    0.974400] generic_infoframe_type: 0x87
[    0.974412] tegra_cec 70015000.tegra_cec: cec_add_sysfs ret=0
[    0.974416] tegra_cec 70015000.tegra_cec: probed
[    0.974601] tegradc tegradc.0: DT parsed successfully
[    0.974661] tegradc tegradc.0: Display dc.ffffff800b100000 registered with id=0
[    0.975208] tegradc tegradc.0: Bootloader disp_param detected. Detected mode: 1920x1080 (on 0x0mm) pclk=148350781
[    0.978367] tegradc tegradc.0: probed
[    0.983124] tegra-apbdma 60020000.dma: Tegra20 APB DMA driver register 32 channels
[    0.989847] tegra-adma 702e2000.adma: Tegra210 ADMA driver registered 22 channels
[    0.990716] tegra-fuse-burn 7000f800.efuse:efuse-burn: shutdown limit check disabled
[    0.990753] tegra-fuse-burn 7000f800.efuse:efuse-burn: Fuse burn driver initialized
[    0.991093] kfuse 7000fc00.kfuse: initialized
[    0.992661] tegra-pmc-iopower pmc-iopower: Regulator supply iopower-dbg-supply not available
[    0.995165] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
[    0.995192] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.995211] pci_bus 0000:00: root bus resource [mem 0x13000000-0x1fffffff]
[    0.995230] pci_bus 0000:00: root bus resource [mem 0x20000000-0x3fffffff pref]
[    0.995253] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.995289] pci 0000:00:01.0: [10de:0fae] type 01 class 0x060400
[    0.995358] pci 0000:00:01.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.995529] pci 0000:00:02.0: [10de:0faf] type 01 class 0x060400
[    0.995588] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.995740] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.995770] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.995903] pci 0000:01:00.0: [8086:2723] type 00 class 0x028000
[    0.995934] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
[    0.996078] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    0.996659] tegra-pmc-iopower pmc-iopower: NO_IOPOWER setting 0x0
[    0.997337] tegra-dfll-pwm 70110000.pwm: DFLL_PWM regulator is available now
[    0.997361] vdd-cpu: 708 <--> 1322 mV at 708 mV
[    0.997663] pwm-regulator pwm_regulators:pwm-regulator@0: PWM regulator registration passed
[    0.998759] vdd-gpu: applied init 1000000uV constraint
[    0.998784] vdd-gpu: 708 <--> 1323 mV at 997 mV
[    0.999126] pwm-regulator pwm_regulators:pwm-regulator@1: PWM regulator registration passed
[    1.001816] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.001901] No Device Node present for smmu client: serial8250 !!
[    1.001923] platform serial8250: No iommus property found in DT node, got swgids from fixup(101004000)
[    1.001968] iommu: Adding device serial8250 to group 35
[    1.003357] console [ttyS0] disabled
[    1.003411] 70006000.serial: ttyS0 at MMIO 0x70006000 (irq = 63, base_baud = 25500000) is a Tegra
[    1.005236] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.005363] pci 0000:02:00.0: [10ec:8168] type 00 class 0x020000
[    1.005385] pci 0000:02:00.0: reg 0x10: [io  0x0000-0x00ff]
[    1.005411] pci 0000:02:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[    1.005429] pci 0000:02:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
[    1.005552] pci 0000:02:00.0: supports D1 D2
[    1.005555] pci 0000:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.017235] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[    1.017277] pci 0000:00:01.0: BAR 14: assigned [mem 0x13000000-0x130fffff]
[    1.017282] pci 0000:00:02.0: BAR 14: assigned [mem 0x13100000-0x131fffff]
[    1.017286] pci 0000:00:02.0: BAR 13: assigned [io  0x1000-0x1fff]
[    1.017294] pci 0000:01:00.0: BAR 0: assigned [mem 0x13000000-0x13003fff 64bit]
[    1.017310] pci 0000:00:01.0: PCI bridge to [bus 01]
[    1.017316] pci 0000:00:01.0:   bridge window [mem 0x13000000-0x130fffff]
[    1.017326] pci 0000:02:00.0: BAR 4: assigned [mem 0x13100000-0x13103fff 64bit]
[    1.017341] pci 0000:02:00.0: BAR 2: assigned [mem 0x13104000-0x13104fff 64bit]
[    1.017356] pci 0000:02:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[    1.017363] pci 0000:00:02.0: PCI bridge to [bus 02]
[    1.017367] pci 0000:00:02.0:   bridge window [io  0x1000-0x1fff]
[    1.017372] pci 0000:00:02.0:   bridge window [mem 0x13100000-0x131fffff]
[    1.017690] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
[    1.017693] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    1.017698] pcie_pme 0000:00:01.0:pcie001: service driver pcie_pme loaded
[    1.017793] aer 0000:00:01.0:pcie002: service driver aer loaded
[    1.017963] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt
[    1.017966] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
[    1.017971] pcie_pme 0000:00:02.0:pcie001: service driver pcie_pme loaded
[    1.018052] aer 0000:00:02.0:pcie002: service driver aer loaded
[    2.005174] tegra_cec 70015000.tegra_cec: Can't find physical addresse.
[    2.005178] tegra_cec 70015000.tegra_cec: tegra_cec_init Done.
[    3.430457] console [ttyS0] enabled
[    3.430806] Console: switching to colour frame buffer device 240x67
[    3.431213] 70006040.serial: ttyTHS1 at MMIO 0x70006040 (irq = 64, base_baud = 0) is a TEGRA_UART
[    3.431438] serial-tegra 70006200.serial: RX in PIO mode
[    3.431519] 70006200.serial: ttyTHS2 at MMIO 0x70006200 (irq = 65, base_baud = 0) is a TEGRA_UART
[    3.432242] [drm] Initialized
[    3.467461] tegradc tegradc.0: fb registered
[    3.475936] brd: module loaded
[    3.477709] tegradc tegradc.0: DC initialized, skipping tegra_dc_program_mode.
[    3.477733] tegradc tegradc.0: hdmi: tmds rate:148351K prod-setting:prod_c_hdmi_75m_150m
[    3.478081] tegradc tegradc.0: hdmi: get RGB quant from REG programmed by BL.
[    3.478089] tegradc tegradc.0: hdmi: get YCC quant from REG programmed by BL.
[    3.503840] extcon-disp-state extcon:disp-state: cable 47 state 1
[    3.503842] Extcon AUX1(HDMI) enable
[    3.504388] tegradc tegradc.1: disp1 connected to head1->/host1x/sor
[    3.504441] tegradc tegradc.1: No lt-data, using default setting
[    3.504459] tegradc tegradc.1: No hpd-gpio in DT
[    3.504484] tegradc tegradc.1: DT parsed successfully
[    3.504511] tegradc tegradc.1: Display dc.ffffff800fa80000 registered with id=1
[    3.505366] tegradc tegradc.1: dpd enable lookup fail:-19
[    3.509409] tegradc tegradc.1: probed
[    3.538989] tegradc tegradc.0: nominal-pclk:148351648 parent:148350781 div:1.0 pclk:148350781 146868084~161703244
[    3.566938] tegradc tegradc.1: fb registered
[    3.574212] loop: module loaded
[    3.574393] hpd: state 7 (Takeover from bootloader), hpd 0, pending_hpd_evt 1
[    3.574397] hpd: switching from state 7 (Takeover from bootloader) to state 0 (Reset)
[    3.574404] hpd: state 0 (Reset), hpd 0, pending_hpd_evt 0
[    3.597834] tegradc tegradc.1: blank - powerdown
[    3.597884] tegra_profiler: version: 1.141, samples/io: 48/28
[    3.608254] extcon-disp-state extcon:disp-state: cable 44 state 0 already set.
[    3.608260] tegra_profiler: auth: init
[    3.615899] THERMAL EST: found 2 subdevs
[    3.615903] THERMAL EST num_resources: 0
[    3.615907] [THERMAL EST subdev 0]
[    3.615911] [THERMAL EST subdev 1]
[    3.616204] thermal thermal_zone5: Registering thermal zone thermal_zone5 for type thermal-fan-est
[    3.616206] THERMAL EST: thz register success.
[    3.616301] THERMAL EST: end of probe, return err: 0
[    3.616863] sd: No Scsi addr parsed to reserve index
[    3.616884] hisi_sas: driver version v1.6
[    3.620058] tegra-qspi 70410000.spi: Prod settings list not found
[    3.627150] qspi_mtd spi32766.0: MX25U3235F (4096 Kbytes)
[    3.627157] qspi_mtd spi32766.0: mtd .name = spi32766.0, .size = 0x400000 (4MiB) .erasesize = 0x00001000 (4KiB) .numeraseregions = 0
[    3.643312] libphy: Fixed MDIO Bus: probed
[    3.643764] tun: Universal TUN/TAP device driver, 1.6
[    3.643766] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    3.644280] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    3.644282] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    3.644322] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    3.644324] igb: Copyright (c) 2007-2014 Intel Corporation.
[    3.644359] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    3.644361] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    3.644393] Intel(R) 10GbE PCI Express Linux Network Driver - version 4.6.4
[    3.644395] Copyright(c) 1999 - 2017 Intel Corporation.
[    3.644939] r8168 0000:02:00.0: enabling device (0000 -> 0003)
[    3.644955] r8168 Gigabit Ethernet driver 8.045.08-NAPI loaded
[    3.659088] r8168 0000:02:00.0 (unnamed net_device) (uninitialized): Invalid ethernet address 00:00:00:00:00:00, trying device tree node
[    3.659160] r8168 0000:02:00.0 (unnamed net_device) (uninitialized): Found valid ethernet address 00:04:4b:eb:f1:11 from device tree
[    3.659553] r8168: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[    3.659586] r8168  Copyright (C) 2017  Realtek NIC software team <nicfae@realtek.com>
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>.
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>.
[    3.659929] PPP generic driver version 2.4.2
[    3.660056] PPP BSD Compression module registered
[    3.660058] PPP Deflate Compression module registered
[    3.660072] PPP MPPE Compression module registered
[    3.660076] NET: Registered protocol family 24
[    3.660188] VFIO - User Level meta-driver version: 0.3
[    3.665717] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.665728] ehci-pci: EHCI PCI platform driver
[    3.665757] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    3.665772] ohci-pci: OHCI PCI platform driver
[    3.665802] ohci-platform: OHCI generic platform driver
[    3.667549] tegra-xusb 70090000.xusb: USB2 port 0 has OTG_CAP
[    3.668105] tegra-xusb-padctl 7009f000.xusb_padctl: enabled OTG on UTMI pad 0
[    3.669162] tegra-xusb 70090000.xusb: extcon 0: ffffffc0f9722000 id
[    3.670536] usbcore: registered new interface driver uas
[    3.670571] usbcore: registered new interface driver usb-storage
[    3.671589] tegra-xusb 70090000.xusb: Firmware timestamp: 2019-10-17 15:58:59 UTC, Version: 50.25 release
[    3.671618] tegra-xusb 70090000.xusb: xHCI Host Controller
[    3.671633] tegra-xusb 70090000.xusb: new USB bus registered, assigned bus number 1
[    3.672361] tegra-xusb 70090000.xusb: hcc params 0x0184f525 hci version 0x100 quirks 0x00050010
[    3.672397] tegra-xusb 70090000.xusb: irq 61, io mem 0x70090000
[    3.672541] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    3.672547] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.672551] usb usb1: Product: xHCI Host Controller
[    3.672554] usb usb1: Manufacturer: Linux 4.9.140-l4t-r32.4.2 xhci-hcd
[    3.672558] usb usb1: SerialNumber: 70090000.xusb
[    3.672960] hub 1-0:1.0: USB hub found
[    3.672991] hub 1-0:1.0: 5 ports detected
[    3.673505] tegra-xusb 70090000.xusb: xHCI Host Controller
[    3.673514] tegra-xusb 70090000.xusb: new USB bus registered, assigned bus number 2
[    3.673580] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    3.673665] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    3.673670] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.673674] usb usb2: Product: xHCI Host Controller
[    3.673677] usb usb2: Manufacturer: Linux 4.9.140-l4t-r32.4.2 xhci-hcd
[    3.673681] usb usb2: SerialNumber: 70090000.xusb
[    3.674021] hub 2-0:1.0: USB hub found
[    3.674049] hub 2-0:1.0: 4 ports detected
[    3.675519] tegra-xudc-new 700d0000.xudc: device count: 1
[    3.676416] tegra-xudc-new 700d0000.xudc: USB charger detection disabled
[    3.676425] tegra-xudc-new 700d0000.xudc: vbus state: 0
[    3.676445] tegra-xudc-new 700d0000.xudc: entering ELPG
[    3.676657] tegra-xudc-new 700d0000.xudc: entering ELPG done
[    3.677056] mousedev: PS/2 mouse device common for all mice
[    3.677103] usbcore: registered new interface driver xpad
[    3.773223] tegra-xusb-padctl 7009f000.xusb_padctl: power on UTMI pads 1
[    3.773276] tegra-xusb-padctl 7009f000.xusb_padctl: power on UTMI pads 2
[    3.908351] max77686-rtc max77620-rtc: rtc core: registered max77620-rtc as rtc0
[    3.925690] rtc rtc1: alarm rtc device
[    3.925702] tegra_rtc 7000e000.rtc: rtc core: registered 7000e000.rtc as rtc1
[    3.925719] tegra_rtc 7000e000.rtc: Tegra internal Real Time Clock
[    3.926013] i2c /dev entries driver
[    3.927160] i2c i2c-6: Added multiplexed i2c bus 7
[    3.927429] i2c i2c-6: Added multiplexed i2c bus 8
[    3.927434] i2c-mux-gpio cam_i2cmux: 2 port mux on Tegra I2C adapter adapter
[    3.927915] imx219 7-0010: tegracam sensor driver:imx219_v2.0.6
[    3.990058] imx219 8-0010: tegracam sensor driver:imx219_v2.0.6
[    4.001176] usb 1-2: new high-speed USB device number 2 using tegra-xusb
[    4.017548] tegra-vii2c 546c0000.i2c: no acknowledge from address 0x10
[    4.017624] imx219 8-0010: imx219_board_setup: error during i2c read probe (-121)
[    4.017654] imx219 8-0010: board setup failed
[    4.017714] imx219: probe of 8-0010 failed with error -121
[    4.024830] max77620-power max77620-power: Event recorder REG_NVERC : 0x0
[    4.061538] tegra_soctherm 700e2000.soctherm: pllx_offset tz:0 max:14, min:2
[    4.061543] tegra_soctherm 700e2000.soctherm: pllx_offset tz:2 max:8, min:4
[    4.061833] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.061845] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when cpu reaches 102500 mC
[    4.061851] tegra_soctherm 700e2000.soctherm: throttrip: will throttle when cpu reaches 100500 mC
[    4.061974] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.061983] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when gpu reaches 103000 mC
[    4.061988] tegra_soctherm 700e2000.soctherm: throttrip: will throttle when gpu reaches 101000 mC
[    4.062126] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.062134] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when pll reaches 127000 mC
[    4.062138] tegra_soctherm 700e2000.soctherm: throttrip: pll: missing hot temperature
[    4.062307] tegra_soctherm 700e2000.soctherm: failed to register sensor: -19
[    4.062851] tegra_aotag tegra-aotag: Registering sensor 0
[    4.062894] tegra_aotag tegra-aotag: Invalid temp readout
[    4.062914] tegra_aotag tegra-aotag: Invalid temp readout
[    4.062930] tegra_aotag tegra-aotag: Bound to TZ : ID 0
[    4.062938] tegra_aotag tegra-aotag: Probe done [SUCCESS]:0
[    4.063640] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.063654] tegra_dfll_action dfll-cdev-cap: Tegra DFLL 'cap cooling device' registered
[    4.063796] tegra_aotag tegra-aotag: Invalid temp readout
[    4.063817] tegra_dfll_action dfll-cdev-floor: Tegra DFLL 'floor cooling device' registered
[    4.064139] parse_throttle_dt_data: Num cap clks = 6
[    4.064143] parse_throttle_dt_data: clk=cclk_g type=2
[    4.064149] parse_throttle_dt_data: clk=gpu type=4
[    4.064167] parse_throttle_dt_data: clk=cap.throttle.c2bus type=0
[    4.064183] parse_throttle_dt_data: clk=cap.throttle.c3bus type=0
[    4.064201] parse_throttle_dt_data: clk=cap.throttle.sclk type=0
[    4.064203] parse_throttle_dt_data: clk=emc type=3
[    4.066313] usb 1-2: New USB device found, idVendor=0bda, idProduct=5411
[    4.066319] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    4.066323] usb 1-2: Product: 4-Port USB 2.1 Hub
[    4.066330] usb 1-2: Manufacturer: Generic
[    4.066446] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.066530] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.066639] tegra_throttle_probe: probe successful. #cdevs=4
[    4.067178] FAN dev name: pwm-fan
[    4.067220] FAN:gpio request success.
[    4.067271] pwm_fan_driver pwm-fan: cap state:7, cap pwm:255
[    4.067494] pwm_fan_driver pwm-fan: got pwm for fan. polarity is normal
[    4.067610] pwm_fan_driver pwm-fan: fan tach request irq success
[    4.067616] pwm_fan_driver pwm-fan: tach period: 1000
[    4.067685] pwm_fan_driver pwm-fan: index 0: pwm=0, rpm=0, rru=40, rrd=40, state:2
[    4.067690] pwm_fan_driver pwm-fan: index 1: pwm=80, rpm=1000, rru=2, rrd=2, state:2
[    4.067694] pwm_fan_driver pwm-fan: index 2: pwm=120, rpm=2000, rru=1, rrd=1, state:2
[    4.067698] pwm_fan_driver pwm-fan: index 3: pwm=160, rpm=3000, rru=1, rrd=1, state:2
[    4.067702] pwm_fan_driver pwm-fan: index 4: pwm=255, rpm=4000, rru=1, rrd=1, state:3
[    4.067705] pwm_fan_driver pwm-fan: index 5: pwm=255, rpm=5000, rru=1, rrd=1, state:3
[    4.067709] pwm_fan_driver pwm-fan: index 6: pwm=255, rpm=6000, rru=1, rrd=1, state:3
[    4.067713] pwm_fan_driver pwm-fan: index 7: pwm=255, rpm=7000, rru=1, rrd=1, state:4
[    4.067717] pwm_fan_driver pwm-fan: index 8: pwm=255, rpm=10000, rru=1, rrd=1, state:4
[    4.067721] pwm_fan_driver pwm-fan: index 9: pwm=255, rpm=11000, rru=1, rrd=1, state:4
[    4.068585] device-mapper: uevent: version 1.0.3
[    4.068958] device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: dm-devel@redhat.com
[    4.071500] tegra210-cpufreq cpufreq: probe()...completed
[    4.072336] sdhci: Secure Digital Host Controller Interface driver
[    4.072338] sdhci: Copyright(c) Pierre Ossman
[    4.072340] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.072963] sdhci-tegra sdhci-tegra.0: Got CD GPIO
[    4.073452] sdhci-tegra sdhci-tegra.0: Client registration for eMC Successful
[    4.073596] sdhci-tegra sdhci-tegra.0: wakeup init done, cdirq 300
[    4.077454] hub 1-2:1.0: USB hub found
[    4.079417] tegra-se 70012000.se: tegra_se_probe: complete
[    4.080032] hidraw: raw HID events driver (C) Jiri Kosina
[    4.081270] usbcore: registered new interface driver usbhid
[    4.081272] usbhid: USB HID core driver
[    4.082882] hub 1-2:1.0: 4 ports detected
[    4.102969] tegra21x_actmon 6000c800.actmon: in actmon_register()...
[    4.103339] tegra21x_actmon 6000c800.actmon: initialization Completed for the device mc_all
[    4.105572] nvpmodel: initialized successfully
[    4.106898] usbcore: registered new interface driver snd-usb-audio
[    4.106968] No Device Node present for smmu client: snd-soc-dummy !!
[    4.106974] platform snd-soc-dummy: No iommus property found in DT node, got swgids from fixup(101004000)
[    4.106995] iommu: Adding device snd-soc-dummy to group 36
[    4.121316] mmc0: SDHCI controller on sdhci-tegra.0 [sdhci-tegra.0] using ADMA 64-bit with 64 bit addr
[    4.133363] OPE platform probe
[    4.133475] OPE platform probe successful
[    4.133762] OPE platform probe
[    4.133881] OPE platform probe successful
[    4.167853] tegra-asoc: sound: ADMAIF1 <-> ADMAIF1 mapping ok
[    4.168013] tegra-asoc: sound: ADMAIF2 <-> ADMAIF2 mapping ok
[    4.168162] tegra-asoc: sound: ADMAIF3 <-> ADMAIF3 mapping ok
[    4.168312] tegra-asoc: sound: ADMAIF4 <-> ADMAIF4 mapping ok
[    4.168463] tegra-asoc: sound: ADMAIF5 <-> ADMAIF5 mapping ok
[    4.168611] tegra-asoc: sound: ADMAIF6 <-> ADMAIF6 mapping ok
[    4.168761] tegra-asoc: sound: ADMAIF7 <-> ADMAIF7 mapping ok
[    4.168912] tegra-asoc: sound: ADMAIF8 <-> ADMAIF8 mapping ok
[    4.169058] tegra-asoc: sound: ADMAIF9 <-> ADMAIF9 mapping ok
[    4.169227] tegra-asoc: sound: ADMAIF10 <-> ADMAIF10 mapping ok
[    4.184677] u32 classifier
[    4.184680]     Actions configured
[    4.184928] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    4.219793] ip_tables: (C) 2000-2006 Netfilter Core Team
[    4.228762] Initializing XFRM netlink socket
[    4.229678] NET: Registered protocol family 10
[    4.230660] NET: Registered protocol family 17
[    4.230675] NET: Registered protocol family 15
[    4.230775] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    4.230902] Bluetooth: RFCOMM socket layer initialized
[    4.230913] Bluetooth: RFCOMM ver 1.11
[    4.230919] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[    4.230925] Bluetooth: HIDP socket layer initialized
[    4.230950] 9pnet: Installing 9P2000 support
[    4.230986] Key type dns_resolver registered
[    4.237641] Registered cp15_barrier emulation handler
[    4.237651] Registered setend emulation handler
[    4.239987] registered taskstats version 1
[    4.240696] Adding domain ve2-pd to PM domain host1x-pd
[    4.240948] Adding domain ve-pd to PM domain host1x-pd
[    4.243482] isp 54600000.isp: initialized
[    4.244529] isp 54680000.isp: initialized
[    4.257186] usb 1-3: new full-speed USB device number 3 using tegra-xusb
[    4.257736] last reset is due to software reset
[    4.257739] KERNEL: PMC reset status reg: 0x3
[    4.257829] BL: PMC reset status reg: 0x3
[    4.257832] BL: PMIC poweroff Event Recorder: 0x0
[    4.259543] clk_cbus_recalc_rate: no gbus parent
[    4.259549] clk_cbus_round_rate: no gbus parent
[    4.259551] clk_cbus_round_rate: no gbus parent
[    4.259558] clk_cbus_recalc_rate: no gbus parent
[    4.259673] clk_cbus_recalc_rate: no gbus parent
[    4.259676] clk_cbus_round_rate: no gbus parent
[    4.259678] clk_cbus_round_rate: no gbus parent
[    4.259683] clk_cbus_recalc_rate: no gbus parent
[    4.260360] tegra_dvfs: GPU-cap: registered
[    4.260425] tegra dvfs: vdd-cpu: nominal 1168mV, offset 708000uV, step 19200uV, scaling enabled
[    4.260427] tegra dvfs: vdd-core: nominal 1125mV, offset 600000uV, step 12500uV, scaling enabled
[    4.260429] tegra dvfs: vdd-gpu: nominal 1078mV, offset 708000uV, step 10000uV, scaling enabled
[    4.335555] tegra_dvfs: vdd-gpu-vts: registered
[    4.357261] usb 1-3: New USB device found, idVendor=8087, idProduct=0029
[    4.357266] usb 1-3: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    4.367376] tegra_core_action core_dvfs_cdev_floor: Tegra CORE DVFS 'floor cooling device' registered
[    4.368242] tegra_core_action core_dvfs_cdev_cap: Tegra CORE DVFS 'cap cooling device' registered
[    4.368931] input: gpio-keys as /devices/gpio-keys/input/input0
[    4.417760] tegra_rtc 7000e000.rtc: setting system clock to 2020-08-12 02:39:32 UTC (1597199972)
[    4.418913] vi 54080000.vi: vi_probe: ++
[    4.525234] usb 1-2.1: new full-speed USB device number 4 using tegra-xusb
[    4.556123] vi 54080000.vi: initialized
[    4.587634] vi 54080000.vi: subdev nvcsi--2 bound
[    4.587650] vi 54080000.vi: subdev imx219 7-0010 bound
[    4.588483] vi 54080000.vi: subdev nvcsi--1 bound
[    4.590507] Disable partitions left on by BL
[    4.590510]   disb
[    4.590563] bwmgr: missing cdev-type property
[    4.590671] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.590678] DRAM derating cdev registered.
[    4.591904] vdd-fan: disabling
[    4.591908] vdd-usb-vbus: disabling
[    4.591910] vdd-usb-vbus2: disabling
[    4.591922] vddio-sdmmc3-ap: disabling
[    4.598029] avdd-io-edp-1v05: disabling
[    4.598032] vdd-usb-hub-en: disabling
[    4.604260] ALSA device list:
[    4.604263]   #1: tegra-snd-t210ref-mobile-rt565x
[    4.624621] usb 1-2.1: New USB device found, idVendor=0403, idProduct=6010
[    4.624626] usb 1-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    4.624630] usb 1-2.1: Product: Dual RS232
[    4.624633] usb 1-2.1: Manufacturer: FTDI
[    4.757230] usb 1-2.2: new full-speed USB device number 5 using tegra-xusb
[    4.814164] usb 1-2.2: New USB device found, idVendor=1a86, idProduct=7523
[    4.814168] usb 1-2.2: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[    4.814172] usb 1-2.2: Product: USB Serial
[    4.929221] usb 1-2.4: new full-speed USB device number 6 using tegra-xusb
[    4.979572] usb 1-2.4: New USB device found, idVendor=222a, idProduct=0001
[    4.979576] usb 1-2.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    5.027446] hid-generic 0003:222A:0001.0002: hidraw0: USB HID v1.10 Device [HID 222a:0001] on usb-70090000.xusb-2.4/input1
[    5.066908] mmc0: hw tuning done ...
[    5.066942] mmc0: new ultra high speed SDR104 SDHC card at address 59b4
[    5.067538] mmcblk0: mmc0:59b4 LX32G 29.4 GiB
[    5.143099]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16
[    5.196841] Extcon DP: HPD disabled
[    5.196844] hpd: hpd_switch 0
[    5.196848] hpd: switching from state 0 (Reset) to state 1 (Check Plug)
[    5.203692] hpd: state 1 (Check Plug), hpd 0, pending_hpd_evt 0
[    5.203696] hpd: switching from state 1 (Check Plug) to state 3 (Disabled)
[    5.223635] input: tegra-hda HDMI/DP,pcm=3 as /devices/70030000.hda/sound/card0/input1
[    5.231882] tegradc tegradc.0: blank - powerdown
[    5.235878] Freeing unused kernel memory: 11456K
[    5.280219] extcon-disp-state extcon:disp-state: cable 47 state 0
[    5.280221] Extcon AUX1(HDMI) disable
[    5.308596] tegradc tegradc.0: unblank
[    5.344231] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[    5.344278] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[    5.345316] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[    5.345323] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[    5.379981] extcon-disp-state extcon:disp-state: cable 47 state 1
[    5.379983] Extcon AUX1(HDMI) enable
[    5.380298] tegradc tegradc.0: sync windows ret = 250
[    5.470075] extcon-disp-state extcon:disp-state: cable 51 state 1
[    5.477641] Extcon HDMI: HPD enabled
[    5.482793] tegradc tegradc.0: hdmi: plugged
[    6.777926] random: crng init done
[    7.836138] tegra-xusb 70090000.xusb: Upgrade port 0 to USB3.0
[    7.852375] tegra-xusb 70090000.xusb: Upgrade port 1 to USB3.0
[    8.166004] usb 2-1: new SuperSpeed USB device number 2 using tegra-xusb
[    8.207624] usb 2-1: New USB device found, idVendor=0bda, idProduct=0411
[    8.221228] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.232677] usb 2-1: Product: 4-Port USB 3.1 Hub
[    8.241456] usb 2-1: Manufacturer: Generic
[    8.252885] hub 2-1:1.0: USB hub found
[    8.261619] hub 2-1:1.0: 4 ports detected
[    8.425296] usb 2-1: usb_suspend_both: status 0
[    8.434140] usb usb2: usb_suspend_both: status 0
[    9.218412] EXT4-fs (mmcblk0p13): couldn't mount as ext3 due to feature incompatibilities
[    9.230061] EXT4-fs (mmcblk0p13): couldn't mount as ext2 due to feature incompatibilities
[    9.244745] EXT4-fs (mmcblk0p13): mounted filesystem with ordered data mode. Opts: (null)
[    9.282246] EXT4-fs (mmcblk0p15): mounted filesystem with ordered data mode. Opts: (null)
[    9.596267] EXT4-fs (mmcblk0p13): re-mounted. Opts: (null)
[    9.795996] cgroup: cgroup2: unknown option "nsdelegate"
[    9.968221] systemd[1]: File /lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    9.992431] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[   10.093432] systemd[1]: /lib/systemd/system/chronyd.service:9: PIDFile= references path below legacy directory /var/run/, updating /var/run/chrony/chronyd.pid \xe2\x86\x92 /run/chrony/chronyd.pid; please update the unit file accordingly.
[   10.389988] nvgpu: 57000000.gpu           gm20b_init_clk_setup_sw:1268 [INFO]  GPCPLL initial settings: NA mode, M=1, N=34, P=3 (id = 1)
[   12.556103] EXT4-fs (mmcblk0p16): mounted filesystem with ordered data mode. Opts: (null)
[   13.986708] compat: loading out-of-tree module taints kernel.
[   13.996722] Loading modules backported from iwlwifi
[   14.003340] iwlwifi-stack-public:release/core52:8470:f4c7b3b5
[   14.070130] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[   14.084137] cfg80211: Problem loading in-kernel X.509 certificate (-129)
[   14.099080] cfg80211: loaded regulatory.db is malformed or signature is missing/invalid
[   14.138997] usbcore: registered new interface driver usbserial
[   14.139037] Intel(R) Wireless WiFi driver for Linux
[   14.139039] Copyright(c) 2003- 2015 Intel Corporation
[   14.139251] iwlwifi 0000:01:00.0: enabling device (0000 -> 0002)
[   14.145453] iwlwifi 0000:01:00.0: Direct firmware load for iwl-dbg-cfg.ini failed with error -2
[   14.145457] iwlwifi 0000:01:00.0: Falling back to user helper
[   14.152357] usbcore: registered new interface driver btusb
[   14.202809] usbcore: registered new interface driver ch341
[   14.211171] usbserial: USB Serial support registered for ch341-uart
[   14.211525] usbcore: registered new interface driver ftdi_sio
[   14.222929] input: HID 222a:0001 as /devices/70090000.xusb/usb1/1-2/1-2.4/1-2.4:1.0/0003:222A:0001.0001/input/input2
[   14.223189] hid-multitouch 0003:222A:0001.0001: input,hidraw1: USB HID v1.10 Device [HID 222a:0001] on usb-70090000.xusb-2.4/input0
[   14.252322] usbserial: USB Serial support registered for FTDI USB Serial Device
[   14.261021] ftdi_sio 1-2.1:1.0: FTDI USB Serial Device converter detected
[   14.265447] ch341 1-2.2:1.0: ch341-uart converter detected
[   14.276896] usb 1-2.1: Detected FT2232C
[   14.282507] usb 1-2.1: FTDI USB Serial Device converter now attached to ttyUSB1
[   14.291720] usb 1-2.2: ch341-uart converter now attached to ttyUSB0
[   14.299929] ftdi_sio 1-2.1:1.1: FTDI USB Serial Device converter detected
[   14.310240] usb 1-2.1: Detected FT2232C
[   14.317300] usb 1-2.1: FTDI USB Serial Device converter now attached to ttyUSB2
[   74.892920] iwlwifi 0000:01:00.0: api flags index 2 larger than supported by driver
[   74.911487] iwlwifi 0000:01:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 89.3.35.22
[   74.934248] iwlwifi 0000:01:00.0: Found debug destination: EXTERNAL_DRAM
[   74.934272] iwlwifi 0000:01:00.0: Found debug configuration: 0
[   74.942159] iwlwifi 0000:01:00.0: loaded firmware version 55.d9698065.0 cc-a0-55.ucode op_mode iwlmvm
[   74.942437] iwlwifi 0000:01:00.0: Direct firmware load for iwl-debug-yoyo.bin failed with error -2
[   74.942455] iwlwifi 0000:01:00.0: Falling back to user helper
[   77.549619] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   77.559318] eth0: 0xffffff800a91e000, 00:04:4b:eb:f1:11, IRQ 406
[   77.622034] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   78.737959] Bridge firewalling registered
[   79.017032] Netfilter messages via NETLINK v0.30.
[   79.028602] ctnetlink v0.93: registering with nfnetlink.
[   79.217878] IPv6: ADDRCONF(NETDEV_UP): supervisor0: link is not ready
[   79.454222] IPv6: ADDRCONF(NETDEV_UP): balena0: link is not ready
[   79.613839] IPv6: ADDRCONF(NETDEV_UP): br-cf2fb069bbe8: link is not ready
[   81.717455] r8168: eth0: link up
[   81.720742] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   85.855224] br-cf2fb069bbe8: port 1(vethff86f6b) entered blocking state
[   85.865411] br-cf2fb069bbe8: port 1(vethff86f6b) entered disabled state
[   85.872476] device vethff86f6b entered promiscuous mode
[   85.878231] IPv6: ADDRCONF(NETDEV_UP): vethff86f6b: link is not ready
[   85.975377] br-cf2fb069bbe8: port 2(vethd329f34) entered blocking state
[   85.984822] br-cf2fb069bbe8: port 2(vethd329f34) entered disabled state
[   85.992147] device vethd329f34 entered promiscuous mode
[   86.004509] IPv6: ADDRCONF(NETDEV_UP): vethd329f34: link is not ready
[   86.012645] br-cf2fb069bbe8: port 2(vethd329f34) entered blocking state
[   86.019326] br-cf2fb069bbe8: port 2(vethd329f34) entered forwarding state
[   86.029040] br-cf2fb069bbe8: port 2(vethd329f34) entered disabled state
[   86.138550] br-cf2fb069bbe8: port 3(vethfbacd6d) entered blocking state
[   86.145230] br-cf2fb069bbe8: port 3(vethfbacd6d) entered disabled state
[   86.152158] device vethfbacd6d entered promiscuous mode
[   86.157871] IPv6: ADDRCONF(NETDEV_UP): vethfbacd6d: link is not ready
[   86.164421] br-cf2fb069bbe8: port 3(vethfbacd6d) entered blocking state
[   86.171045] br-cf2fb069bbe8: port 3(vethfbacd6d) entered forwarding state
[   86.179106] IPv6: ADDRCONF(NETDEV_CHANGE): br-cf2fb069bbe8: link becomes ready
[   86.186645] br-cf2fb069bbe8: port 3(vethfbacd6d) entered disabled state
[   86.319511] br-cf2fb069bbe8: port 4(veth5f20e73) entered blocking state
[   86.326175] br-cf2fb069bbe8: port 4(veth5f20e73) entered disabled state
[   86.333104] device veth5f20e73 entered promiscuous mode
[   86.338700] IPv6: ADDRCONF(NETDEV_UP): veth5f20e73: link is not ready
[   86.345333] br-cf2fb069bbe8: port 4(veth5f20e73) entered blocking state
[   86.351964] br-cf2fb069bbe8: port 4(veth5f20e73) entered forwarding state
[   86.510305] br-cf2fb069bbe8: port 5(vethb983baa) entered blocking state
[   86.519293] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[   86.526525] device vethb983baa entered promiscuous mode
[   86.536163] IPv6: ADDRCONF(NETDEV_UP): vethb983baa: link is not ready
[   86.549323] br-cf2fb069bbe8: port 5(vethb983baa) entered blocking state
[   86.555985] br-cf2fb069bbe8: port 5(vethb983baa) entered forwarding state
[   86.728929] br-cf2fb069bbe8: port 6(veth1431b77) entered blocking state
[   86.735675] br-cf2fb069bbe8: port 6(veth1431b77) entered disabled state
[   86.766524] device veth1431b77 entered promiscuous mode
[   86.801862] IPv6: ADDRCONF(NETDEV_UP): veth1431b77: link is not ready
[   86.818484] br-cf2fb069bbe8: port 6(veth1431b77) entered blocking state
[   86.825127] br-cf2fb069bbe8: port 6(veth1431b77) entered forwarding state
[   86.869676] br-cf2fb069bbe8: port 4(veth5f20e73) entered disabled state
[   86.876595] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[   86.889630] br-cf2fb069bbe8: port 6(veth1431b77) entered disabled state
[   86.915674] br-cf2fb069bbe8: port 7(vethf0594d1) entered blocking state
[   86.924398] br-cf2fb069bbe8: port 7(vethf0594d1) entered disabled state
[   86.959309] device vethf0594d1 entered promiscuous mode
[   87.013374] IPv6: ADDRCONF(NETDEV_UP): vethf0594d1: link is not ready
[   87.040098] br-cf2fb069bbe8: port 7(vethf0594d1) entered blocking state
[   87.046732] br-cf2fb069bbe8: port 7(vethf0594d1) entered forwarding state
[   87.304688] supervisor0: port 1(veth395f796) entered blocking state
[   87.326832] supervisor0: port 1(veth395f796) entered disabled state
[   87.343564] device veth395f796 entered promiscuous mode
[   87.361764] IPv6: ADDRCONF(NETDEV_UP): veth395f796: link is not ready
[   87.379393] supervisor0: port 1(veth395f796) entered blocking state
[   87.385684] supervisor0: port 1(veth395f796) entered forwarding state
[   87.423330] br-cf2fb069bbe8: port 7(vethf0594d1) entered disabled state
[   87.434594] supervisor0: port 1(veth395f796) entered disabled state
[   88.164692] supervisor0: port 2(veth19e8cca) entered blocking state
[   88.173170] supervisor0: port 2(veth19e8cca) entered disabled state
[   88.183789] device veth19e8cca entered promiscuous mode
[   88.202731] IPv6: ADDRCONF(NETDEV_UP): veth19e8cca: link is not ready
[   88.231631] supervisor0: port 2(veth19e8cca) entered blocking state
[   88.237935] supervisor0: port 2(veth19e8cca) entered forwarding state
[   88.261301] supervisor0: port 2(veth19e8cca) entered disabled state
[   88.643819] eth0: renamed from vethb02cdcb
[   88.678052] IPv6: ADDRCONF(NETDEV_CHANGE): vethff86f6b: link becomes ready
[   88.687210] br-cf2fb069bbe8: port 1(vethff86f6b) entered blocking state
[   88.693858] br-cf2fb069bbe8: port 1(vethff86f6b) entered forwarding state
[   88.976342] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   89.124543] eth0: renamed from vethc2ed264
[   89.148243] IPv6: ADDRCONF(NETDEV_CHANGE): vethd329f34: link becomes ready
[   89.156892] br-cf2fb069bbe8: port 2(vethd329f34) entered blocking state
[   89.163530] br-cf2fb069bbe8: port 2(vethd329f34) entered forwarding state
[   89.242645] eth0: renamed from veth4a8f50e
[   89.301796] IPv6: ADDRCONF(NETDEV_CHANGE): vethfbacd6d: link becomes ready
[   89.309979] br-cf2fb069bbe8: port 3(vethfbacd6d) entered blocking state
[   89.316640] br-cf2fb069bbe8: port 3(vethfbacd6d) entered forwarding state
[   89.601125] eth0: renamed from veth39d888d
[   89.640406] IPv6: ADDRCONF(NETDEV_CHANGE): veth5f20e73: link becomes ready
[   89.647656] br-cf2fb069bbe8: port 4(veth5f20e73) entered blocking state
[   89.654314] br-cf2fb069bbe8: port 4(veth5f20e73) entered forwarding state
[   89.900095] eth0: renamed from veth8d45784
[   89.932145] IPv6: ADDRCONF(NETDEV_CHANGE): vethb983baa: link becomes ready
[   89.947019] br-cf2fb069bbe8: port 5(vethb983baa) entered blocking state
[   89.953663] br-cf2fb069bbe8: port 5(vethb983baa) entered forwarding state
[   90.021196] eth1: renamed from vethb1a72b5
[   90.079181] eth0: renamed from veth37db94c
[   90.102739] IPv6: ADDRCONF(NETDEV_CHANGE): veth395f796: link becomes ready
[   90.111581] supervisor0: port 1(veth395f796) entered blocking state
[   90.117899] supervisor0: port 1(veth395f796) entered forwarding state
[   90.125928] IPv6: ADDRCONF(NETDEV_CHANGE): supervisor0: link becomes ready
[   90.135456] IPv6: ADDRCONF(NETDEV_CHANGE): veth1431b77: link becomes ready
[   90.142806] br-cf2fb069bbe8: port 6(veth1431b77) entered blocking state
[   90.149453] br-cf2fb069bbe8: port 6(veth1431b77) entered forwarding state
[   90.352691] eth0: renamed from veth44e9c28
[   90.392476] IPv6: ADDRCONF(NETDEV_CHANGE): vethf0594d1: link becomes ready
[   90.400214] br-cf2fb069bbe8: port 7(vethf0594d1) entered blocking state
[   90.406855] br-cf2fb069bbe8: port 7(vethf0594d1) entered forwarding state
[   90.462171] eth1: renamed from vethfa3c083
[   90.495427] IPv6: ADDRCONF(NETDEV_CHANGE): veth19e8cca: link becomes ready
[   90.502630] supervisor0: port 2(veth19e8cca) entered blocking state
[   90.508928] supervisor0: port 2(veth19e8cca) entered forwarding state
[   91.185075] systemd-udevd[30]: starting version 237
[   91.185170] systemd-udevd[28]: starting version 237
[   91.315987] systemd-udevd[29]: starting version 237
[   91.416314] systemd-udevd[29]: Specified group 'crypto' unknown
[   91.429089] systemd-udevd[29]: Specified group 'trusty' unknown
[   91.430981] systemd-udevd[31]: Specified group 'crypto' unknown
[   91.433853] systemd-udevd[31]: Specified group 'trusty' unknown
[   91.518150] systemd-udevd[30]: Specified group 'crypto' unknown
[   91.533832] systemd-udevd[30]: Specified group 'trusty' unknown
[   94.131597] systemd-udevd[35]: Process '/sbin/crda' failed with exit code 234.
[  101.074202] systemd-udevd[30]: Specified group 'crypto' unknown
[  101.081689] systemd-udevd[30]: Specified group 'trusty' unknown
[  101.130191] systemd-udevd[29]: Specified group 'crypto' unknown
[  101.132356] systemd-udevd[31]: Specified group 'crypto' unknown
[  101.134060] systemd-udevd[31]: Specified group 'trusty' unknown
[  101.148229] systemd-udevd[29]: Specified group 'trusty' unknown
[  101.310472] v4l2loopback driver version 0.12.5 loaded
[  104.080605] systemd-udevd[30]: Specified group 'crypto' unknown
[  104.088008] systemd-udevd[30]: Specified group 'trusty' unknown
[  104.132223] systemd-udevd[31]: Specified group 'crypto' unknown
[  104.140684] systemd-udevd[31]: Specified group 'trusty' unknown
[  105.036965] systemd-udevd[29]: Specified group 'crypto' unknown
[  105.043005] systemd-udevd[29]: Specified group 'trusty' unknown
[  136.620064] iwlwifi 0000:01:00.0: Detected Intel(R) Wi-Fi 6 AX200 160MHz, REV=0x340
[  136.831550] iwlwifi 0000:01:00.0: base HW address: 14:f6:d8:6b:f7:33
[  136.853711] thermal thermal_zone6: failed to read out thermal zone (-61)
[  136.860473] thermal thermal_zone6: Registering thermal zone thermal_zone6 for type iwlwifi_1
[  136.943585] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  137.251093] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  137.491132] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  137.669770] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  141.077679] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  141.459293] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  141.482396] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[  168.482818] systemd-udevd[89]: Process '/sbin/crda' failed with exit code 234.
[  258.798267] Adding 2095332k swap on /var/lib/swap.img.  Priority:-1 extents:454 across:9828344k SS
[  261.551153] systemd-udevd[86]: Process '/sbin/crda' failed with exit code 234.
[  273.652364] tegradc tegradc.0: blank - powerdown
[  273.699524] extcon-disp-state extcon:disp-state: cable 47 state 0
[  273.699527] Extcon AUX1(HDMI) disable
[  273.724266] tegradc tegradc.0: unblank
[  273.764377] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  273.765444] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  273.766521] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  273.766529] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  273.808321] extcon-disp-state extcon:disp-state: cable 47 state 1
[  273.808324] Extcon AUX1(HDMI) enable
[  273.811175] tegradc tegradc.0: sync windows ret = 250
[  275.985682] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  276.001417] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  276.252612] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  277.459346] nvgpu: 57000000.gpu   nvgpu_set_error_notifier_locked:137  [ERR]  error notifier set to 8 for ch 502
[  277.469549] nvgpu: 57000000.gpu     gk20a_fifo_handle_sched_error:2531 [ERR]  fifo sched ctxsw timeout error: engine=0, tsg=6, ms=3100
[  277.469556] tegradc tegradc.0: blank - powerdown
[  277.469665] ---- mlocks ----

[  277.469674] ---- syncpts ----
[  277.469679] id 1 (disp0_a) min 2 max 2 refs 1 (previous client : )
[  277.469682] id 2 (disp0_b) min 2 max 2 refs 1 (previous client : )
[  277.469684] id 3 (disp0_c) min 2 max 2 refs 1 (previous client : )
[  277.469689] id 7 (gm20b_503) min 54 max 54 refs 1 (previous client : )
[  277.469692] id 8 (gm20b_507) min 27914 max 27914 refs 1 (previous client : )
[  277.469694] id 9 (gm20b_506) min 17876 max 17876 refs 1 (previous client : )
[  277.469697] id 11 (gm20b_505) min 2 max 2 refs 1 (previous client : )
[  277.469700] id 12 (gm20b_504) min 2 max 2 refs 1 (previous client : )
[  277.469702] id 13 (gm20b_502) min 2 max 4 refs 1 (previous client : )
[  277.469711] id 26 (vblank0) min 219 max -4 refs 1 (previous client : )

[  277.469796] ---- channels ----
[  277.469803]
               ---- host general irq ----

[  277.469805] sync_hintmask_ext = 0xc0000000
[  277.469807] sync_hintmask = 0x80000000
[  277.469809] sync_intc0mask = 0x00000001
[  277.469811] sync_intmask = 0x00000011
[  277.469812]
               ---- host syncpt irq mask ----

[  277.469814] syncpt_thresh_int_mask(0) = 0x04000001
[  277.469816] syncpt_thresh_int_mask(1) = 0x00000000
[  277.469818] syncpt_thresh_int_mask(2) = 0x00000000
[  277.469820] syncpt_thresh_int_mask(3) = 0x00000000
[  277.469821] syncpt_thresh_int_mask(4) = 0x00000000
[  277.469823] syncpt_thresh_int_mask(5) = 0x00000000
[  277.469825] syncpt_thresh_int_mask(6) = 0x00000000
[  277.469827] syncpt_thresh_int_mask(7) = 0x00000000
[  277.469828] syncpt_thresh_int_mask(8) = 0x00000000
[  277.469830] syncpt_thresh_int_mask(9) = 0x00000000
[  277.469832] syncpt_thresh_int_mask(10) = 0x00000000
[  277.469834] syncpt_thresh_int_mask(11) = 0x00000000
[  277.469835]
               ---- host syncpt irq status ----

[  277.469837] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[  277.469839] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[  277.469841] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[  277.469842] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[  277.469844] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[  277.469846] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[  277.469847]
               ---- host syncpt thresh ----

[  277.469849] syncpt_int_thresh_thresh_0(0) = 1
[  277.469856] syncpt_int_thresh_thresh_0(13) = 4
[  277.469931] gm20b pbdma 0:
[  277.469931] id: 6 (tsg), next_id: 6 (tsg) chan status: valid
[  277.469942] PBDMA_PUT: 0000001e000081d0 PBDMA_GET: 0000001e00000044 GP_PUT: 00000006 GP_GET: 00000004 FETCH: 00000006 HEADER: 800302ec
               HDR: 800160bb SHADOW0: 00000004 SHADOW1: 0081cc1e

[  277.469946] gm20b eng 0:
[  277.469947] id: 0 (channel), next_id: 6 (tsg), ctx status: load
[  277.469948] busy

[  277.469952] gm20b eng 1:
[  277.469953] id: 6 (tsg), next_id: 6 (tsg), ctx status: valid


[  277.470108] 502-gm20b, pid 11563, refs 4:
[  277.470108] channel status:  in use on_pbdma_and_eng busy
[  277.470113] RAMFC : TOP: 8000001f00004020 PUT: 0000001f00004020 GET: 0000001f00004020 FETCH: 0000001f00004020
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000d01 SEMAPHORE 0000000d fc001000 00000001 01100002

[  277.470119] 503-gm20b, pid 10651, refs 2:
[  277.470119] channel status:  in use idle not busy
[  277.470124] RAMFC : TOP: 8000001f00280288 PUT: 0000001f00280288 GET: 0000001f00280288 FETCH: 0000001f00280288
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000701 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470129] 504-gm20b, pid 10651, refs 2:
[  277.470129] channel status:  in use idle not busy
[  277.470134] RAMFC : TOP: 8000001f00240018 PUT: 0000001f00240018 GET: 0000001f00240018 FETCH: 0000001f00240018
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000c01 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470139] 505-gm20b, pid 10651, refs 2:
[  277.470139] channel status:  in use idle not busy
[  277.470143] RAMFC : TOP: 8000001f00200018 PUT: 0000001f00200018 GET: 0000001f00200018 FETCH: 0000001f00200018
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000b01 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470148] 506-gm20b, pid 10651, refs 2:
[  277.470148] channel status:  in use idle not busy
[  277.470152] RAMFC : TOP: 8000001f001745f0 PUT: 0000001f001745f0 GET: 0000001f001745f0 FETCH: 0000001f001745f0
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000901 SEMAPHORE 00000001 0002fff0 00003683 00000004

[  277.470157] 507-gm20b, pid 10651, refs 2:
[  277.470157] channel status:  in use idle not busy
[  277.470162] RAMFC : TOP: 8000001f00051c88 PUT: 0000001f00051c88 GET: 0000001f00051c88 FETCH: 0000001f00051c88
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000801 SEMAPHORE 00000001 0002ffb0 00000003 00001004

[  277.470167] 508-gm20b, pid 3975, refs 2:
[  277.470167] channel status:  in use idle not busy
[  277.470171] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470176] 509-gm20b, pid 3975, refs 2:
[  277.470176] channel status:  in use idle not busy
[  277.470180] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470185] 510-gm20b, pid 3975, refs 2:
[  277.470185] channel status:  in use idle not busy
[  277.470188] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470193] 511-gm20b, pid 3975, refs 2:
[  277.470193] channel status:  in use idle not busy
[  277.470197] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470439] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1721 [ERR]  fake mmu fault on engine 0, engine subid 0 (gpc), client 2 (pe 0), addr 0x547b119000, type 4 (unbound inst), access_type 0x00000001,inst_ptr 0x20f88000
[  277.470443] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:129  [ERR]  gr_fecs_os_r : 0
[  277.470446] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:131  [ERR]  gr_fecs_cpuctl_r : 0x40
[  277.470448] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:133  [ERR]  gr_fecs_idlestate_r : 0x1
[  277.470451] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:135  [ERR]  gr_fecs_mailbox0_r : 0x0
[  277.470454] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:137  [ERR]  gr_fecs_mailbox1_r : 0x0
[  277.470456] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:139  [ERR]  gr_fecs_irqstat_r : 0x0
[  277.470459] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:141  [ERR]  gr_fecs_irqmode_r : 0x4
[  277.470462] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:143  [ERR]  gr_fecs_irqmask_r : 0x8704
[  277.470464] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:145  [ERR]  gr_fecs_irqdest_r : 0x0
[  277.470467] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:147  [ERR]  gr_fecs_debug1_r : 0x40
[  277.470469] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:149  [ERR]  gr_fecs_debuginfo_r : 0x0
[  277.470472] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:151  [ERR]  gr_fecs_ctxsw_status_1_r : 0x304
[  277.470475] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(0) : 0x1
[  277.470478] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(1) : 0x0
[  277.470480] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(2) : 0x90009
[  277.470483] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(3) : 0x14000
[  277.470486] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0
[  277.470488] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(5) : 0x0
[  277.470491] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(6) : 0x0
[  277.470493] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(7) : 0x0
[  277.470496] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(8) : 0x0
[  277.470499] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(9) : 0x0
[  277.470501] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(10) : 0x0
[  277.470504] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(11) : 0x0
[  277.470506] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(12) : 0x0
[  277.470509] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(13) : 0x0
[  277.470511] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(14) : 0x0
[  277.470514] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(15) : 0x0
[  277.470516] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:159  [ERR]  gr_fecs_engctl_r : 0x0
[  277.470519] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:161  [ERR]  gr_fecs_curctx_r : 0x0
[  277.470521] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:163  [ERR]  gr_fecs_nxtctx_r : 0x0
[  277.470524] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:169  [ERR]  FECS_FALCON_REG_IMB : 0xbadfbadf
[  277.470526] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:175  [ERR]  FECS_FALCON_REG_DMB : 0xbadfbadf
[  277.470529] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:181  [ERR]  FECS_FALCON_REG_CSW : 0xbadfbadf
[  277.470532] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:187  [ERR]  FECS_FALCON_REG_CTX : 0xbadfbadf
[  277.470534] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:193  [ERR]  FECS_FALCON_REG_EXCI : 0xbadfbadf
[  277.470537] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470540] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470542] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470545] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470547] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470550] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470553] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470555] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470558] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1726 [ERR]  gr_status_r : 0x400081
[  277.471416] nvgpu: 57000000.gpu                    fifo_error_isr:2605 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100
[  277.522573] extcon-disp-state extcon:disp-state: cable 47 state 0
[  277.522574] Extcon AUX1(HDMI) disable
[  277.543010] tegradc tegradc.0: unblank
[  277.582375] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  277.582510] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  277.584208] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  277.584216] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  277.625664] extcon-disp-state extcon:disp-state: cable 47 state 1
[  277.625667] Extcon AUX1(HDMI) enable
[  277.630816] tegradc tegradc.0: sync windows ret = 250
[  289.172493] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[  289.179255] veth8d45784: renamed from eth0
[  289.462075] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[  289.473780] device vethb983baa left promiscuous mode
[  289.478850] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[  289.628544] supervisor0: port 1(veth395f796) entered disabled state
[  289.635114] vethb1a72b5: renamed from eth1
[  289.695436] supervisor0: port 1(veth395f796) entered disabled state
[  289.711138] device veth395f796 left promiscuous mode
[  289.716216] supervisor0: port 1(veth395f796) entered disabled state
[  289.947375] br-cf2fb069bbe8: port 5(veth4151ce7) entered blocking state
[  289.954067] br-cf2fb069bbe8: port 5(veth4151ce7) entered disabled state
[  289.981862] device veth4151ce7 entered promiscuous mode
[  290.001918] IPv6: ADDRCONF(NETDEV_UP): veth4151ce7: link is not ready
[  290.026065] br-cf2fb069bbe8: port 5(veth4151ce7) entered blocking state
[  290.032768] br-cf2fb069bbe8: port 5(veth4151ce7) entered forwarding state
[  290.174098] br-cf2fb069bbe8: port 5(veth4151ce7) entered disabled state
[  290.290341] supervisor0: port 1(veth7024076) entered blocking state
[  290.299915] supervisor0: port 1(veth7024076) entered disabled state
[  290.307830] device veth7024076 entered promiscuous mode
[  290.314161] IPv6: ADDRCONF(NETDEV_UP): veth7024076: link is not ready
[  290.320632] supervisor0: port 1(veth7024076) entered blocking state
[  290.326918] supervisor0: port 1(veth7024076) entered forwarding state
[  291.198129] supervisor0: port 1(veth7024076) entered disabled state
[  291.455052] eth0: renamed from veth8842ec1
[  291.474833] IPv6: ADDRCONF(NETDEV_CHANGE): veth4151ce7: link becomes ready
[  291.481824] br-cf2fb069bbe8: port 5(veth4151ce7) entered blocking state
[  291.488455] br-cf2fb069bbe8: port 5(veth4151ce7) entered forwarding state
[  291.523724] eth1: renamed from veth5b7966f
[  291.554948] IPv6: ADDRCONF(NETDEV_CHANGE): veth7024076: link becomes ready
[  291.562005] supervisor0: port 1(veth7024076) entered blocking state
[  291.568293] supervisor0: port 1(veth7024076) entered forwarding state
[  292.104218] systemd-udevd[29]: starting version 237
[  292.148244] systemd-udevd[30]: Specified group 'crypto' unknown
[  292.157779] systemd-udevd[30]: Specified group 'trusty' unknown
[  294.425102] systemd-udevd[276]: Process '/sbin/crda' failed with exit code 234.
[  294.860568] tegradc tegradc.0: blank - powerdown
[  294.910588] extcon-disp-state extcon:disp-state: cable 47 state 0
[  294.910591] Extcon AUX1(HDMI) disable
[  294.931247] tegradc tegradc.0: unblank
[  294.994500] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  294.994616] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  294.996313] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  294.996320] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  295.038345] extcon-disp-state extcon:disp-state: cable 47 state 1
[  295.038347] Extcon AUX1(HDMI) enable
[  295.038707] tegradc tegradc.0: sync windows ret = 250
[  295.488873] tegradc tegradc.0: blank - powerdown
[  295.547498] extcon-disp-state extcon:disp-state: cable 47 state 0
[  295.547500] Extcon AUX1(HDMI) disable
[  295.586185] tegradc tegradc.0: unblank
[  295.601256] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  295.601438] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  295.604312] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  295.604324] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  295.642099] extcon-disp-state extcon:disp-state: cable 47 state 1
[  295.642102] Extcon AUX1(HDMI) enable
[  295.697041] tegradc tegradc.0: unblank
[  295.700990] tegradc tegradc.1: blank - powerdown
[  299.849040] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[  299.857758] Bluetooth: BNEP socket layer initialized
[  623.736096] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  939.659178] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready

ok, you should run several oceanFFTs over and make it sure by “ps -e” then unplugged the hdmi monitor ,then may be several hour or days ,you check the displays by connecting the hdmi monitor leadding to black screen sometime.

@drcnyc,

I think your case is special. We cannot guarantee the case of docker or other OS.

For example, I cannot just give lots of docker setup steps to gpu team and ask them to debug.
When we are trying to debug, we only have pure jetpack and devkit. Thus, I will try with @wangzhaofeng usecase first.

Anyway, thanks for sharing the log and usecase.

@WayneWWW Well that’s disappointing. I think you will find that many users who are developing IoT devices use docker-based setups - especially when deploying large fleets. Unfortunately this will likely put our plans with the Jetson Nano on hold.

I think if we resolve the case from @wangzhaofeng, this would resolve your case too. So please wait for our reply.

Thanks.

Hi @wangzhaofeng,

Do you run 15 oceanFFT programs simultaneously?

@WayneWWW, in case its helpful, I found errors in my Xorg log which correspond to the errors in the dmesg log. I wasn’t even using the device at the time. Below is the outputs from both logs with timestamps. The GPU was not being used at the time.

Xorg:

[   102.205] (**) HID 222a:0001: (accel) keeping acceleration scheme 1
[   102.205] (**) HID 222a:0001: (accel) acceleration profile 0
[   102.205] (**) HID 222a:0001: (accel) acceleration factor: 2.000
[   102.205] (**) HID 222a:0001: (accel) acceleration threshold: 4
[   102.206] (II) config/udev: Adding input device HID 222a:0001 (/dev/input/mouse0)
[   102.206] (II) No input driver specified, ignoring this device.
[   102.206] (II) This device may have been added with another device file.
[   102.207] (II) config/udev: Adding input device gpio-keys (/dev/input/event1)
[   102.207] (**) gpio-keys: Applying InputClass "evdev keyboard catchall"
[   102.207] (II) Using input driver 'evdev' for 'gpio-keys'
[   102.207] (**) gpio-keys: always reports core events
[   102.207] (**) evdev: gpio-keys: Device: "/dev/input/event1"
[   102.207] (--) evdev: gpio-keys: Vendor 0x1 Product 0x1
[   102.207] (--) evdev: gpio-keys: Found keys
[   102.207] (II) evdev: gpio-keys: Configuring as keyboard
[   102.207] (**) Option "config_info" "udev:/sys/devices/gpio-keys/input/input1/event1"
[   102.207] (II) XINPUT: Adding extended input device "gpio-keys" (type: KEYBOARD, id 7)
[   102.207] (**) Option "xkb_rules" "evdev"
[   102.208] (**) Option "xkb_model" "pc105"
[   102.208] (**) Option "xkb_layout" "us"
[   102.404] (--) NVIDIA(GPU-0): HCD PJ402D-2 (DFP-0): connected
[   102.404] (--) NVIDIA(GPU-0): HCD PJ402D-2 (DFP-0): External TMDS
[  5274.596] (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
[  5274.596] (EE) NVIDIA(0):     recover...
[  5275.552] (II) NVIDIA(0): Error recovery was successful.
[  5631.459] (WW) NVIDIA(0): WAIT (2-S, 17, 0xc19c, 0x0000a3f8, 0x0000b640)
[  5631.685] (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
[  5631.685] (EE) NVIDIA(0):     recover...
[  5632.653] (II) NVIDIA(0): Error recovery was successful.

Dmesg:

[ 3278.502293] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 3594.476767] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 3910.504475] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 4226.434943] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 4542.455919] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 4858.461871] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 5174.451461] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 5274.732318] nvgpu: 57000000.gpu   nvgpu_set_error_notifier_locked:137  [ERR]  error notifier set to 8 for ch 507
[ 5274.742494] nvgpu: 57000000.gpu     gk20a_fifo_handle_sched_error:2531 [ERR]  fifo sched ctxsw timeout error: engine=0, tsg=4, ms=3100
[ 5274.754691] ---- mlocks ----

[ 5274.759087] ---- syncpts ----
[ 5274.762072] id 1 (disp0_a) min 4 max 4 refs 1 (previous client : )
[ 5274.768262] id 2 (disp0_b) min 3 max 3 refs 1 (previous client : )
[ 5274.774453] id 3 (disp0_c) min 3 max 3 refs 1 (previous client : )
[ 5274.780647] id 7 (gm20b_507) min 46254 max 46260 refs 1 (previous client : )
[ 5274.787702] id 8 (gm20b_506) min 4 max 4 refs 1 (previous client : )
[ 5274.794074] id 26 (vblank0) min 309888 max -6 refs 1 (previous client : )

[ 5274.802455] ---- channels ----
[ 5274.805505]
               ---- host general irq ----

[ 5274.812220] sync_hintmask_ext = 0xc0000000
[ 5274.816325] sync_hintmask = 0x80000000
[ 5274.820081] sync_intc0mask = 0x00000001
[ 5274.823936] sync_intmask = 0x00000011
[ 5274.827607]
               ---- host syncpt irq mask ----

[ 5274.834663] syncpt_thresh_int_mask(0) = 0x00004001
[ 5274.839459] syncpt_thresh_int_mask(1) = 0x00000000
[ 5274.844256] syncpt_thresh_int_mask(2) = 0x00000000
[ 5274.849056] syncpt_thresh_int_mask(3) = 0x00000000
[ 5274.853852] syncpt_thresh_int_mask(4) = 0x00000000
[ 5274.858648] syncpt_thresh_int_mask(5) = 0x00000000
[ 5274.863442] syncpt_thresh_int_mask(6) = 0x00000000
[ 5274.868238] syncpt_thresh_int_mask(7) = 0x00000000
[ 5274.873034] syncpt_thresh_int_mask(8) = 0x00000000
[ 5274.877829] syncpt_thresh_int_mask(9) = 0x00000000
[ 5274.882625] syncpt_thresh_int_mask(10) = 0x00000000
[ 5274.887505] syncpt_thresh_int_mask(11) = 0x00000000
[ 5274.892386]
               ---- host syncpt irq status ----

[ 5274.899610] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[ 5274.905014] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[ 5274.910416] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[ 5274.915820] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[ 5274.921221] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[ 5274.926625] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[ 5274.932030]
               ---- host syncpt thresh ----

[ 5274.938911] syncpt_int_thresh_thresh_0(0) = 1
[ 5274.943278] syncpt_int_thresh_thresh_0(7) = 46256
[ 5274.948065] gm20b pbdma 0:
[ 5274.950694] id: 4 (tsg), next_id: 4 (tsg) chan status: valid
[ 5274.956357] PBDMA_PUT: 0000001e0000ec20 PBDMA_GET: 0000001e0000ebac GP_PUT: 00000708 GP_GET: 00000706 FETCH: 00000708 HEADER: 60030100
               HDR: 60116040 SHADOW0: 0000eb8c SHADOW1: 0000941e

[ 5274.974176] gm20b eng 0:
[ 5274.976614] id: 0 (channel), next_id: 4 (tsg), ctx status: load
[ 5274.982622] busy

[ 5274.984551] gm20b eng 1:
[ 5274.987007] id: 0 (channel), next_id: 0 (channel), ctx status: invalid


[ 5274.996726] 506-gm20b, pid 10688, refs 2:
[ 5275.000660] channel status:  in use idle not busy
[ 5275.005368] RAMFC : TOP: 8000001f0000c030 PUT: 0000001f0000c030 GET: 0000001f0000c030 FETCH: 0000001f0000c030
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000801 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.028817] 507-gm20b, pid 10688, refs 8:
[ 5275.032752] channel status:  in use on_pbdma_and_eng busy
[ 5275.038152] RAMFC : TOP: 8000001f00004ae0 PUT: 0000001f00004ae0 GET: 0000001f00004ae0 FETCH: 0000001f00004ae0
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000701 SEMAPHORE 0000000d fc001000 01583ae3 01100002

[ 5275.061600] 508-gm20b, pid 3975, refs 2:
[ 5275.065444] channel status:  in use idle not busy
[ 5275.070153] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.093611] 509-gm20b, pid 3975, refs 2:
[ 5275.097452] channel status:  in use idle not busy
[ 5275.102155] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.125603] 510-gm20b, pid 3975, refs 2:
[ 5275.129443] channel status:  in use idle not busy
[ 5275.134150] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.157596] 511-gm20b, pid 3975, refs 2:
[ 5275.161437] channel status:  in use idle not busy
[ 5275.166141] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.190261] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1721 [ERR]  fake mmu fault on engine 0, engine subid 0 (gpc), client 0 (l1 0), addr 0x547b019000, type 4 (unbound inst), access_type 0x00000001,inst_ptr 0x20f88000
[ 5275.210676] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:129  [ERR]  gr_fecs_os_r : 0
[ 5275.219291] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:131  [ERR]  gr_fecs_cpuctl_r : 0x40
[ 5275.228509] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:133  [ERR]  gr_fecs_idlestate_r : 0x1
[ 5275.237899] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:135  [ERR]  gr_fecs_mailbox0_r : 0x0
[ 5275.247202] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:137  [ERR]  gr_fecs_mailbox1_r : 0x0
[ 5275.256503] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:139  [ERR]  gr_fecs_irqstat_r : 0x0
[ 5275.265720] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:141  [ERR]  gr_fecs_irqmode_r : 0x4
[ 5275.274935] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:143  [ERR]  gr_fecs_irqmask_r : 0x8704
[ 5275.284412] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:145  [ERR]  gr_fecs_irqdest_r : 0x0
[ 5275.293632] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:147  [ERR]  gr_fecs_debug1_r : 0x40
[ 5275.302853] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:149  [ERR]  gr_fecs_debuginfo_r : 0x0
[ 5275.312247] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:151  [ERR]  gr_fecs_ctxsw_status_1_r : 0x304
[ 5275.322248] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(0) : 0x1
[ 5275.332244] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(1) : 0x0
[ 5275.342241] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(2) : 0x90009
[ 5275.352584] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(3) : 0x14000
[ 5275.362928] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0
[ 5275.373357] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(5) : 0x0
[ 5275.383353] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(6) : 0x0
[ 5275.393348] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(7) : 0x0
[ 5275.403347] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(8) : 0x0
[ 5275.413345] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(9) : 0x0
[ 5275.423346] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(10) : 0x0
[ 5275.433436] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(11) : 0x0
[ 5275.443526] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(12) : 0x0
[ 5275.453609] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(13) : 0x0
[ 5275.463694] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(14) : 0x0
[ 5275.473790] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(15) : 0x0
[ 5275.483881] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:159  [ERR]  gr_fecs_engctl_r : 0x0
[ 5275.493011] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:161  [ERR]  gr_fecs_curctx_r : 0x0
[ 5275.502144] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:163  [ERR]  gr_fecs_nxtctx_r : 0x0
[ 5275.511275] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:169  [ERR]  FECS_FALCON_REG_IMB : 0xbadfbadf
[ 5275.521271] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:175  [ERR]  FECS_FALCON_REG_DMB : 0xbadfbadf
[ 5275.531268] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:181  [ERR]  FECS_FALCON_REG_CSW : 0xbadfbadf
[ 5275.541272] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:187  [ERR]  FECS_FALCON_REG_CTX : 0xbadfbadf
[ 5275.551271] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:193  [ERR]  FECS_FALCON_REG_EXCI : 0xbadfbadf
[ 5275.561356] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.571269] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.581178] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.591088] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.600998] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.610907] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.620817] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.630730] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.640643] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1726 [ERR]  gr_status_r : 0x81
[ 5275.650394] nvgpu: 57000000.gpu                    fifo_error_isr:2605 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100
[ 5490.414530] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 5631.818801] nvgpu: 57000000.gpu   nvgpu_set_error_notifier_locked:137  [ERR]  error notifier set to 8 for ch 507
[ 5631.828989] nvgpu: 57000000.gpu     gk20a_fifo_handle_sched_error:2531 [ERR]  fifo sched ctxsw timeout error: engine=0, tsg=4, ms=3100
[ 5631.841226] ---- mlocks ----

[ 5631.845619] ---- syncpts ----
[ 5631.848606] id 1 (disp0_a) min 4 max 4 refs 1 (previous client : )
[ 5631.854798] id 2 (disp0_b) min 3 max 3 refs 1 (previous client : )
[ 5631.860988] id 3 (disp0_c) min 3 max 3 refs 1 (previous client : )
[ 5631.867175] id 7 (gm20b_507) min 245148 max 245156 refs 1 (previous client : gm20b_507)
[ 5631.875184] id 8 (gm20b_506) min 4 max 4 refs 1 (previous client : )
[ 5631.881553] id 26 (vblank0) min 331281 max -6 refs 1 (previous client : )

[ 5631.889913] ---- channels ----
[ 5631.892980]
               ---- host general irq ----

[ 5631.899689] sync_hintmask_ext = 0xc0000000
[ 5631.903793] sync_hintmask = 0x80000000
[ 5631.907550] sync_intc0mask = 0x00000001
[ 5631.911392] sync_intmask = 0x00000011
[ 5631.915061]
               ---- host syncpt irq mask ----

[ 5631.922117] syncpt_thresh_int_mask(0) = 0x00004001
[ 5631.926919] syncpt_thresh_int_mask(1) = 0x00000000
[ 5631.931715] syncpt_thresh_int_mask(2) = 0x00000000
[ 5631.936513] syncpt_thresh_int_mask(3) = 0x00000000
[ 5631.941309] syncpt_thresh_int_mask(4) = 0x00000000
[ 5631.946106] syncpt_thresh_int_mask(5) = 0x00000000
[ 5631.950903] syncpt_thresh_int_mask(6) = 0x00000000
[ 5631.955699] syncpt_thresh_int_mask(7) = 0x00000000
[ 5631.960494] syncpt_thresh_int_mask(8) = 0x00000000
[ 5631.965290] syncpt_thresh_int_mask(9) = 0x00000000
[ 5631.970085] syncpt_thresh_int_mask(10) = 0x00000000
[ 5631.974968] syncpt_thresh_int_mask(11) = 0x00000000
[ 5631.979852]
               ---- host syncpt irq status ----

[ 5631.987084] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[ 5631.992487] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[ 5631.997891] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[ 5632.003294] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[ 5632.008709] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[ 5632.014113] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[ 5632.019519]
               ---- host syncpt thresh ----

[ 5632.026400] syncpt_int_thresh_thresh_0(0) = 1
[ 5632.030766] syncpt_int_thresh_thresh_0(7) = 245150
[ 5632.035636] gm20b pbdma 0:
[ 5632.038248] id: 4 (tsg), next_id: 4 (tsg) chan status: valid
[ 5632.043924] PBDMA_PUT: 0000001e0000b5c8 PBDMA_GET: 0000001e0000a410 GP_PUT: 0000055f GP_GET: 0000055d FETCH: 0000055f HEADER: 60030100
               HDR: 60116040 SHADOW0: 0000a3f8 SHADOW1: 0011d01e

[ 5632.061736] gm20b eng 0:
[ 5632.064194] id: 0 (channel), next_id: 4 (tsg), ctx status: load
[ 5632.070196] busy

[ 5632.072141] gm20b eng 1:
[ 5632.074581] id: 0 (channel), next_id: 0 (channel), ctx status: invalid


[ 5632.084385] 506-gm20b, pid 10688, refs 2:
[ 5632.088322] channel status:  in use idle not busy
[ 5632.093032] RAMFC : TOP: 8000001f0000c030 PUT: 0000001f0000c030 GET: 0000001f0000c030 FETCH: 0000001f0000c030
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000801 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.116490] 507-gm20b, pid 10688, refs 10:
[ 5632.120512] channel status:  in use on_pbdma_and_eng busy
[ 5632.125919] RAMFC : TOP: 8000001f00014e80 PUT: 0000001f00014e80 GET: 0000001f00014e80 FETCH: 0000001f00014e80
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000701 SEMAPHORE 0000000d fc000000 01cc28fe 01100002

[ 5632.149381] 508-gm20b, pid 3975, refs 2:
[ 5632.153226] channel status:  in use idle not busy
[ 5632.157932] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.181387] 509-gm20b, pid 3975, refs 2:
[ 5632.185251] channel status:  in use idle not busy
[ 5632.189959] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.213416] 510-gm20b, pid 3975, refs 2:
[ 5632.217263] channel status:  in use idle not busy
[ 5632.221971] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.245416] 511-gm20b, pid 3975, refs 2:
[ 5632.249258] channel status:  in use idle not busy
[ 5632.254001] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.277760] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1721 [ERR]  fake mmu fault on engine 0, engine subid 0 (gpc), client 8 (pe 2), addr 0x547b109000, type 4 (unbound inst), access_type 0x00000001,inst_ptr 0x20f88000
[ 5632.298245] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:129  [ERR]  gr_fecs_os_r : 0
[ 5632.306874] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:131  [ERR]  gr_fecs_cpuctl_r : 0x40
[ 5632.316139] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:133  [ERR]  gr_fecs_idlestate_r : 0x1
[ 5632.325543] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:135  [ERR]  gr_fecs_mailbox0_r : 0x0
[ 5632.334882] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:137  [ERR]  gr_fecs_mailbox1_r : 0x0
[ 5632.344199] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:139  [ERR]  gr_fecs_irqstat_r : 0x0
[ 5632.353448] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:141  [ERR]  gr_fecs_irqmode_r : 0x4
[ 5632.362675] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:143  [ERR]  gr_fecs_irqmask_r : 0x8704
[ 5632.372215] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:145  [ERR]  gr_fecs_irqdest_r : 0x0
[ 5632.381475] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:147  [ERR]  gr_fecs_debug1_r : 0x40
[ 5632.390723] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:149  [ERR]  gr_fecs_debuginfo_r : 0x0
[ 5632.400130] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:151  [ERR]  gr_fecs_ctxsw_status_1_r : 0x304
[ 5632.410186] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(0) : 0x1
[ 5632.420225] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(1) : 0x0
[ 5632.430261] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(2) : 0x90009
[ 5632.440621] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(3) : 0x14000
[ 5632.451009] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0
[ 5632.461457] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(5) : 0x0
[ 5632.471508] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(6) : 0x0
[ 5632.481534] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(7) : 0x0
[ 5632.491605] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(8) : 0x0
[ 5632.501630] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(9) : 0x0
[ 5632.511680] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(10) : 0x0
[ 5632.521793] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(11) : 0x0
[ 5632.531931] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(12) : 0x0
[ 5632.542061] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(13) : 0x0
[ 5632.552194] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(14) : 0x0
[ 5632.562318] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(15) : 0x0
[ 5632.572449] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:159  [ERR]  gr_fecs_engctl_r : 0x0
[ 5632.581606] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:161  [ERR]  gr_fecs_curctx_r : 0x0
[ 5632.590780] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:163  [ERR]  gr_fecs_nxtctx_r : 0x0
[ 5632.599937] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:169  [ERR]  FECS_FALCON_REG_IMB : 0xbadfbadf
[ 5632.609983] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:175  [ERR]  FECS_FALCON_REG_DMB : 0xbadfbadf
[ 5632.620009] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:181  [ERR]  FECS_FALCON_REG_CSW : 0xbadfbadf
[ 5632.630056] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:187  [ERR]  FECS_FALCON_REG_CTX : 0xbadfbadf
[ 5632.640105] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:193  [ERR]  FECS_FALCON_REG_EXCI : 0xbadfbadf
[ 5632.650270] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.660220] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.670192] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.680155] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.690174] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.700117] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.710052] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.720049] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.729981] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1726 [ERR]  gr_status_r : 0x81
[ 5632.739919] nvgpu: 57000000.gpu                    fifo_error_isr:2605 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100
[ 5806.419639] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 6122.382657] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 6438.400266] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready

yes ,

Hi @drcnyc,

Does your system have high gpu usage right after boot up?
Do you still have tegrastats tool or anything else that can check the gpu clock/loading there?

@wangzhaofeng

Hi,

I have run 18 hours for this but still cannot reproduce issue.
I cannot occupy 1 jetson nano for couple of days because other colleagues also need it too.

Then I think we should change to usecase oriented.
Why do you want to run 15 oceanFFT? Is it a stress test? What will be your finalized gpu application?

yeah i run a stress test

What is your criteria to define this test is passed ?