L4T 32.4.2 - GPU error on boot

@drcnyc I think you should clarify your usecase or setup.

We only see this issue when running stress gpu loading work. For example, below issue. Or the case from @wangzhaofeng.

It does not make sense this issue just happens right after the system boot up. Could you share the full dmesg with us?

Also, still need your explanation for your setup.

i find some message when i run dmesg as follows

@wangzhaofeng,

We will set up oceanFFT today and see if we can reproduce this.

@WayneWWW below is the full dmesg output. The system is stable after the error, but when I try to run a GPU-intensive app such as detectnet, it occasionally crashes (instant shutoff) while loading. Sometimes though, it loads and runs fine.

My setup / use case is as follows: dev kit, with 32gb SDHC UHS-1 card, running Balena OS (yocto based docker host) with a multi-container app running inside of it. Most of the containers are small / low utilization, running things such as VPN connectivity or custom lightweight GPIO apps. There are two “substantial” containers, one running nodejs / node-red which is core to our app’s functionality, and one running an xorg / chromium display (which is also where I run detectnet and imagenet from. All containers are using L4T base images built off of Ubuntu 18.04 overlaid with L4T 32.4.2 and with CUDA/cuDNN/TensorRT installed (the latter in the case of the display and nodejs containers only; others just have L4T on Ubuntu).

dmesg output:

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.140-l4t-r32.4.2 (oe-user@oe-host) (gcc version 8.3.0 (GCC) ) #1 SMP PREEMPT Fri Jun 12 09:10:09 UTC 2020
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 32,
[    0.000000] OF: fdt: - 80000000 ,  7ee00000
[    0.000000] OF: fdt: - 100000000 ,  7f200000
[    0.000000] Found tegra_fbmem: 00800000@92cb4000
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node iram-carveout, compatible id nvidia,iram-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fac00000
[    0.000000] On node 0 totalpages: 1039872
[    0.000000]   DMA zone: 8192 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 519168 pages, LIFO batch:31
[    0.000000]   Normal zone: 8136 pages used for memmap
[    0.000000]   Normal zone: 520704 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 25 pages/cpu @ffffffc0fefb1000 s61912 r8192 d32296 u102400
[    0.000000] pcpu-alloc: s61912 r8192 d32296 u102400 alloc=25*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1023544
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1125 core_edp_ma=4000 tegra_fbmem=0x800000@0x92cb4000 is_hdmi_initialised=1  console=ttyS0,115200 console=tty0 fbcon=map:0 net.ifnames=0 root=UUID=ba1eadef-32ef-4b0a-93ee-08dca7534519 ro rootwait  sdhci_tegra.en_boot_part_access=1
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 98304 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 29572(90%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Memory: 3561848K/4159488K available (15102K kernel code, 2910K rwdata, 9636K rodata, 11456K init, 605K bss, 122504K reserved, 475136K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f40000   ( 15104 KB)
[    0.000000]     .rodata : 0xffffff8008f40000 - 0xffffff80098b0000   (  9664 KB)
[    0.000000]       .init : 0xffffff80098b0000 - 0xffffff800a3e0000   ( 11456 KB)
[    0.000000]       .data : 0xffffff800a3e0000 - 0xffffff800a6b7808   (  2911 KB)
[    0.000000]        .bss : 0xffffff800a6b7808 - 0xffffff800a74ee24   (   606 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf03fc8000   (    63 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0ff200000   (  4082 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 64.
[    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller
[    0.000000] t210 clock and reset probe
[    0.000000] tegra-pmc: get_secure_pmc_setting: done secure_pmc=1
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000005] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.001562] Console: colour dummy device 80x25
[    0.002432] console [tty0] enabled
[    0.002473] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=76800)
[    0.002502] pid_max: default: 32768 minimum: 301
[    0.002960] Security Framework initialized
[    0.003195] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.003218] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.004478] ftrace: allocating 46832 entries in 183 pages
[    0.177240] sched-energy: CPU device node has no sched-energy-costs
[    0.177273] ASID allocator initialised with 65536 entries
[    0.215961] tegra-id: chipid=22117.
[    0.216007] tegra-id: opt_subrevision=0.
[    0.216035] Tegra Speedo/IDDQ fuse revision 4
[    0.216051] Tegra: CPU Speedo ID 9, SoC Speedo ID 0, GPU Speedo ID 2
[    0.216068] Tegra: CPU Process ID 0, SoC Process ID 0, GPU Process ID 0
[    0.216084] Tegra: CPU Speedo Value 2034, SoC Speedo Value 1909, GPU Speedo Value 2017
[    0.216106] Tegra: CPU IDDQ Value 1732, SoC IDDQ Value 1860, GPU IDDQ Value 2185
[    0.216143] Tegra Revision: A02 SKU: 0x8f CPU Process: 0 SoC Process: 0
[    0.216172] DTS File Name: /yocto/resin-board/build/tmp/work-shared/jetson-nano/kernel-source/arch/arm64/boot/dts/../../../../nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-b00.dts
[    0.216250] DTB Build time: Jun 12 2020 09:15:26
[    0.265078] CPU1: Booted secondary processor [411fd071]
[    0.296989] CPU2: Booted secondary processor [411fd071]
[    0.328946] CPU3: Booted secondary processor [411fd071]
[    0.329039] Brought up 4 CPUs
[    0.329101] SMP: Total of 4 processors activated.
[    0.329119] CPU features: detected feature: 32-bit EL0 Support
[    0.329398] CPU: All CPU(s) started at EL2
[    0.329430] alternatives: patching kernel code
[    0.337765] devtmpfs: initialized
[    0.373432] Initilizing CustomIPI irq domain
[    0.373833] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.373876] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.385355] pinctrl core: initialized pinctrl subsystem
[    0.385968] OS set in device tree is not L4T.
[    0.386298] regulator-dummy: no parameters
[    0.386541] Initializing plugin-manager
[    0.386637] Plugin module not found
[    0.386884] node /plugin-manager/fragement@0 match with board >=3448-0000-100
[    0.387463] node /plugin-manager/fragment@1 match with board >=3448-0000-101
[    0.389355] node /plugin-manager/fragement@10 match with board >=3448-0000-300
[    0.391732] node /plugin-manager/fragement@11 match with board >=3448-0000-300
[    0.395337] NET: Registered protocol family 16
[    0.402851] pstore: using zlib compression
[    0.403551] console [pstore-1] enabled
[    0.403571] pstore: Registered ramoops as persistent store backend
[    0.403592] ramoops: attached 0x200000@0xb0000000, ecc: 0/0
[    0.413088] cpuidle: using governor menu
[    0.417009] tegra_smmu 70019000.iommu: Loaded Tegra IOMMU driver
[    0.418472] vdso: 2 pages (1 code @ ffffff8008f47000, 1 data @ ffffff800a3e4000)
[    0.418519] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.421627] atomic_pool_init():622: DMA: preallocated 1024 KiB pool for atomic allocations
[    0.423695] Serial: AMBA PL011 UART driver
[    0.424185] tegra_powergate_init: DONE
[    0.424219] DTS File Name: /yocto/resin-board/build/tmp/work-shared/jetson-nano/kernel-source/arch/arm64/boot/dts/../../../../nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-b00.dts
[    0.424257] DTB Build time: Jun 12 2020 09:15:26
[    0.425846] Tegra reboot handler registered.
[    0.430132] iommu: Adding device tegra-carveouts to group 0
[    0.430316] platform tegra-carveouts: domain=ffffffc0f9bda3d8 allocates as[0]=ffffffc0f9bd00a8
[    0.430659] iommu: Adding device smmu_test to group 1
[    0.430724] platform smmu_test: domain=ffffffc0f9bda618 allocates as[0]=ffffffc0f9bd0110
[    0.431492] mc: mapped MMIO address: 0xffffff800802e000 -> 0x70019000
[    0.431571] mc: mapped MMIO address: 0xffffff8008065000 -> 0x7001c000
[    0.431639] mc: mapped MMIO address: 0xffffff8008079000 -> 0x7001d000
[    0.431663] nv-tegra-mc 70019000.mc: No mssnvlink node
[    0.431698] mc-err: mcerr ops are set to t21x
[    0.441469] iommu: Adding device 70090000.xusb to group 2
[    0.441777] iommu: Adding device 70006000.serial to group 3
[    0.441858] platform 70006000.serial: domain=ffffffc0f9be3cd8 allocates as[0]=ffffffc0f9bd0178
[    0.442313] iommu: Adding device 70006040.serial to group 4
[    0.442566] iommu: Adding device 70006200.serial to group 5
[    0.442753] iommu: Adding device sound to group 6
[    0.442817] platform sound: domain=ffffffc0f944d258 allocates as[0]=ffffffc0f9bd01e0
[    0.443330] iommu: Adding device 7000d400.spi to group 7
[    0.443595] iommu: Adding device 7000d600.spi to group 8
[    0.443886] iommu: Adding device 70410000.spi to group 9
[    0.444183] iommu: Adding device 50000000.host1x to group 10
[    0.444457] iommu: Adding device 54080000.vi to group 11
[    0.444845] iommu: Adding device 54600000.isp to group 12
[    0.445139] iommu: Adding device 54680000.isp to group 13
[    0.445407] iommu: Adding device tegradc.0 to group 14
[    0.445471] platform tegradc.0: domain=ffffffc0f9459198 allocates as[0]=ffffffc0f9bd0248
[    0.449669] platform tegradc.0: IOVA linear map 0x0000000092cb4000(800000)
[    0.450081] platform tegradc.0: IOVA linear map 0x00000000d7000000(19000000)
[    0.450390] iommu: Adding device tegradc.1 to group 15
[    0.450454] platform tegradc.1: domain=ffffffc0f94593d8 allocates as[0]=ffffffc0f9bd02b0
[    0.454627] platform tegradc.1: IOVA linear map 0x0000000092cb4000(800000)
[    0.455037] platform tegradc.1: IOVA linear map 0x00000000d7000000(19000000)
[    0.455303] iommu: Adding device 54340000.vic to group 16
[    0.455529] iommu: Adding device 544c0000.nvenc to group 17
[    0.455723] iommu: Adding device 54500000.tsec to group 18
[    0.455920] iommu: Adding device 54100000.tsecb to group 19
[    0.456118] iommu: Adding device 54480000.nvdec to group 20
[    0.456310] iommu: Adding device 54380000.nvjpg to group 21
[    0.457338] iommu: Adding device 546c0000.i2c to group 22
[    0.457657] iommu: Adding device 57000000.gpu to group 23
[    0.457727] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[0]=ffffffc0f9bd0318
[    0.457808] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[1]=ffffffc0f9bd0380
[    0.457868] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[2]=ffffffc0f9bd03e8
[    0.457921] platform 57000000.gpu: domain=ffffffc0f94dd3d8 allocates as[3]=ffffffc0f9bd0450
[    0.458737] tegra-pmc 7000e400.pmc: i2c-thermtrip node not found, emergency thermal reset disabled.
[    0.458777] tegra-pmc 7000e400.pmc: scratch reg offset dts data not present
[    0.458804] tegra-pmc: ### PMC reset source: TEGRA_SOFTWARE_RESET
[    0.458824] tegra-pmc: ### PMC reset level: TEGRA_RESET_LEVEL_WARM
[    0.458841] tegra-pmc: ### PMC reset status reg: 0x3
[    0.509250] padctrl padctrl.0: Pad control driver tegra-pmc-padctrl registered
[    0.509286] tegra-pmc: Clear bootloader IO dpd settings
[    0.509319] tegra-pmc 7000e400.pmc: IO padctrl driver initialized
[    0.509403] tegra-pmc 7000e400.pmc: PMC: Successfully configure bootrom reset commands
[    0.509702] iommu: Adding device 70012000.se to group 24
[    0.510409] iommu: Adding device 7000c000.i2c to group 25
[    0.510673] iommu: Adding device 7000c400.i2c to group 26
[    0.510933] iommu: Adding device 7000c500.i2c to group 27
[    0.511186] iommu: Adding device 7000c700.i2c to group 28
[    0.511438] iommu: Adding device 7000d000.i2c to group 29
[    0.511685] iommu: Adding device 7000d100.i2c to group 30
[    0.511968] iommu: Adding device sdhci-tegra.0 to group 31
[    0.512041] platform sdhci-tegra.0: domain=ffffffc0f94ed558 allocates as[0]=ffffffc0f9bd04b8
[    0.512828] iommu: Adding device 700d0000.xudc to group 32
[    0.515770] vdd-ac-bat: 5000 mV
[    0.516108] vdd-5v0-sys: 5000 mV
[    0.516935] vdd-5v0-hdmi: supplied by vdd-5v0-sys
[    0.516992] vdd-5v0-hdmi: 5000 mV
[    0.517394] vdd-1v8-sys: 1800 mV
[    0.517809] vdd-fan: supplied by vdd-5v0-sys
[    0.517858] vdd-fan: 5000 mV
[    0.518225] vdd-usb-vbus: supplied by vdd-5v0-sys
[    0.518275] vdd-usb-vbus: 5000 mV
[    0.518740] vdd-usb-vbus2: 5000 mV
[    0.541775] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.544248] gpio gpiochip0: gpio-line-names specifies 240 line names but there are 256 lines on the chip
[    0.545435] GPIO line 151 (camera-control-output-low) hogged as output/low
[    0.545473] GPIO line 152 (camera-control-output-low) hogged as output/low
[    0.545525] GPIO line 6 (system-suspend-gpio) hogged as output/high
[    0.545866] gpiochip_setup_dev: registered GPIOs 0 to 255 on device: gpiochip0 (tegra-gpio)
[    0.553535] eventlib_kernel: keventlib is initialized, test id: 0
[    0.553849] SCSI subsystem initialized
[    0.554103] libata version 3.00 loaded.
[    0.554297] usbcore: registered new interface driver usbfs
[    0.554362] usbcore: registered new interface driver hub
[    0.554441] usbcore: registered new device driver usb
[    0.558138] max77620 4-003c: PMIC Version OTP:0x35 and ES:0x8
[    0.564314] vdd-core: 600 <--> 1162 mV at 1125 mV
[    0.565218] random: fast init done
[    0.565252] vdd-ddr-1v1: Bringing 1125000uV into 1150000-1150000uV
[    0.568213] vdd-ddr-1v1: 1150 mV
[    0.572214] vdd-pre-reg-1v35: 1350 mV
[    0.576211] vdd-1v8: 1800 mV
[    0.580212] avdd-sys-1v2: 1200 mV
[    0.584210] vdd-pex-1v0: 1050 mV
[    0.584879] vddio-sdmmc-ap: 1800 <--> 3300 mV at 3300 mV
[    0.585435] max77620-ldo3: at 3100 mV
[    0.588215] vdd-rtc: 850 <--> 1100 mV at 1000 mV
[    0.588745] max77620-ldo5: at 3100 mV
[    0.592213] vddio-sdmmc3-ap: 1800 <--> 3300 mV at 2800 mV
[    0.596215] avdd-1v05-pll: 1050 mV
[    0.600212] avdd-io-hdmi-dp: 1050 mV
[    0.602585] GPIO line 505 (spmic-default-output-high) hogged as output/high
[    0.602913] gpiochip_setup_dev: registered GPIOs 504 to 511 on device: gpiochip1 (max77620-gpio)
[    0.603201] max77620 4-003c: max77620 probe successful
[    0.605663] media: Linux media interface: v0.10
[    0.605727] Linux video capture interface: v2.00
[    0.606536] pps_core: LinuxPPS API ver. 1 registered
[    0.606561] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.606595] PTP clock support registered
[    0.609603] tegra210-emc 7001b000.external-memory-controller: validated EMC DFS table
[    0.610312] Advanced Linux Sound Architecture Driver Initialized.
[    0.610930] Bluetooth: Core ver 2.22
[    0.610986] NET: Registered protocol family 31
[    0.611003] Bluetooth: HCI device and connection manager initialized
[    0.611030] Bluetooth: HCI socket layer initialized
[    0.611051] Bluetooth: L2CAP socket layer initialized
[    0.611090] Bluetooth: SCO socket layer initialized
[    0.612913] tegra210_dvfs: no clock found for sdmmc2_ddr
[    0.612960] tegra210_dvfs: no clock found for sdmmc4_ddr
[    0.613026] tegra210_dvfs: no clock found for sdmmc1_ddr
[    0.613056] tegra210_dvfs: no clock found for sdmmc3_ddr
[    0.616512] tegra_dvfs: Unable to get vdd-cpu rail for step info, defering probe
[    0.617779] vdd-3v3-sys: supplied by vdd-5v0-sys
[    0.617906] vdd-3v3-sys: 3300 mV
[    0.618168] vdd-1v8-sys: supplied by vdd-3v3-sys
[    0.620283] vdd-usb-vbus2: supplied by vdd-3v3-sys
[    0.620748] vdd-3v3-sd: supplied by vdd-3v3-sys
[    0.620799] vdd-3v3-sd: 3300 mV
[    0.621647] avdd-io-edp-1v05: supplied by avdd-1v05-pll
[    0.621712] avdd-io-edp-1v05: 1050 mV
[    0.622113] vdd-usb-hub-en: supplied by vdd-1v8-sys
[    0.622167] vdd-usb-hub-en: 5000 mV
[    0.623355] camchar: rtcpu character device driver loaded
[    0.623978] extcon-gpio-states extcon:extcon@1: Cable state:0, cable id:0
[    0.625162] clocksource: Switched to clocksource arch_sys_counter
[    0.665547] VFS: Disk quotas dquot_6.6.0
[    0.665732] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.666217] nvmap_heap_init: nvmap_heap_init: created heap block cache
[    0.666819] dma_declare_coherent_resizable_cma_memory:324: resizable heap=vpr, base=0x00000000d7000000, size=0x19000000
[    0.666870]  dma-vpr: heap size is not multiple of cma_chunk_size heap_info->num_chunks (13) rem_chunk_size(0x1000000)
[    0.667158] cma: enabled page replacement for spfn=d7000, epfn=f0000
[    0.667189] dma_declare_coherent_resizable_cma_memory:373: resizable cma heap=vpr create successful
[    0.667219] tegra-carveouts tegra-carveouts: assigned reserved memory node vpr-carveout
[    0.667271] tegra-carveouts tegra-carveouts: iram :dma coherent mem declare 0x0000000040001000,258048
[    0.667298] tegra-carveouts tegra-carveouts: assigned reserved memory node iram-carveout
[    0.667335] nvmap: nvmap_select_cache_ops() nvmap cache ops set to set/ways
[    0.667360] nvmap_page_pool_init: Total RAM pages: 1009246
[    0.667379] nvmap_page_pool_init: nvmap page pool size: 126155 pages (492 MB)
[    0.667514] nvmap_background_zero_thread: PP zeroing thread starting.
[    0.667799] misc nvmap: created heap iram base 0x0000000040001000 size (252KiB)
[    0.668739] misc nvmap: created heap vpr base 0x00000000d7000000 size (409600KiB)
[    0.676248] thermal thermal_zone0: Registering thermal zone thermal_zone0 for type AO-therm
[    0.676653] thermal thermal_zone1: Registering thermal zone thermal_zone1 for type CPU-therm
[    0.676892] thermal thermal_zone2: Registering thermal zone thermal_zone2 for type GPU-therm
[    0.677098] thermal thermal_zone3: Registering thermal zone thermal_zone3 for type PLL-therm
[    0.677305] thermal thermal_zone4: Registering thermal zone thermal_zone4 for type PMIC-Die
[    0.677584] pre_t19x_iso_plat_init(): iso emc max clk=1600000KHz
[    0.677604] pre_t19x_iso_plat_init(): max_iso_bw=11520000KB
[    0.678042] NET: Registered protocol family 2
[    0.679015] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    0.679250] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    0.679759] TCP: Hash tables configured (established 32768 bind 32768)
[    0.679928] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    0.680024] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    0.680432] NET: Registered protocol family 1
[    0.681226] RPC: Registered named UNIX socket transport module.
[    0.681257] RPC: Registered udp transport module.
[    0.681274] RPC: Registered tcp transport module.
[    0.681292] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.681317] PCI: CLS 0 bytes, default 64
[    0.892322] Trying to unpack rootfs image as initramfs...
[    0.902343] host1x 50000000.host1x: initialized
[    0.904034] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[    0.908046] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.917693] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.919259] 9p: Installing v9fs 9p2000 file system support
[    0.922624] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 240)
[    0.922797] io scheduler noop registered
[    0.923089] io scheduler cfq registered (default)
[    0.925563] gic 702f9000.agic: GIC IRQ controller registered
[    0.928543] iommu: Adding device 702ef000.adsp to group 33
[    0.937241] iommu: Adding device aconnect@702c0000:adsp_audio to group 34
[    0.937402] tegra-aconnect aconnect@702c0000: Tegra ACONNECT bus registered
[    0.937905] tegra-xusb-padctl 7009f000.xusb_padctl: TEGRA_FUSE_SKU_CALIB_0 = 0x7a68411
[    0.937936] tegra-xusb-padctl 7009f000.xusb_padctl: TEGRA_FUSE_USB_CALIB_EXT_0 = 0x4
[    0.939591] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.0, lane = usb2-0, function = xusb
[    0.939716] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.1, lane = usb2-1, function = xusb
[    0.939816] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-usb2.2, lane = usb2-2, function = xusb
[    0.940013] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.3, lane = pcie-0, function = pcie-x1
[    0.940114] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.4, lane = pcie-1, function = pcie-x4
[    0.940217] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.5, lane = pcie-2, function = pcie-x4
[    0.940318] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.6, lane = pcie-3, function = pcie-x4
[    0.940428] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.7, lane = pcie-4, function = pcie-x4
[    0.940527] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.8, lane = pcie-5, function = xusb
[    0.940631] tegra-xusb-padctl 7009f000.xusb_padctl: dev = phy-pcie.9, lane = pcie-6, function = xusb
[    0.946595] tegra-pwm 7000a000.pwm: PWM clk cannot sleep in ops
[    0.948219] tegra-dfll-pwm 70110000.pwm: DFLL pwm-rate: 12800000
[    0.950058] tegra-pcie 1003000.pcie: 4x1, 1x1 configuration
[    0.951361] tegra-pcie 1003000.pcie: PCIE: Enable power rails
[    0.951731] tegra-pcie 1003000.pcie: probing port 0, using 4 lanes
[    0.952854] tegra_camera_platform tegra-camera-platform: tegra_camera_probe:camera_platform_driver probe
[    0.953108] misc tegra_camera_ctrl: tegra_camera_isomgr_register isp_iso_bw=1500000, vi_iso_bw=1500000, max_bw=1500000
[    0.953305] tegra-pcie 1003000.pcie: probing port 1, using 1 lanes
[    0.953575] Adding domain tsec-pd to PM domain host1x-pd
[    0.956930] tsec 54500000.tsec: initialized
[    0.958377] tsec 54100000.tsecb: initialized
[    0.959057] Adding domain nvdec-pd to PM domain host1x-pd
[    0.962363] nvdec 54480000.nvdec: initialized
[    0.963365] Adding domain vic03-pd to PM domain host1x-pd
[    0.963570] Adding domain msenc-pd to PM domain host1x-pd
[    0.963752] Adding domain nvjpg-pd to PM domain host1x-pd
[    0.967458] falcon 54340000.vic: initialized
[    0.968732] falcon 544c0000.nvenc: initialized
[    0.970124] falcon 54380000.nvjpg: initialized
[    0.974017] tegra_cec 70015000.tegra_cec: dt=1 start=0x70015000 end=0x70015FFF irq=97
[    0.974082] tegra_cec 70015000.tegra_cec: Enable clock result: 0.
[    0.974132] tegra_cec 70015000.tegra_cec: tegra_cec_init started
[    0.974186] tegradc tegradc.0: disp0 connected to head0->/host1x/sor1
[    0.974326] display board info: id 0x0, fab 0x0
[    0.974400] generic_infoframe_type: 0x87
[    0.974412] tegra_cec 70015000.tegra_cec: cec_add_sysfs ret=0
[    0.974416] tegra_cec 70015000.tegra_cec: probed
[    0.974601] tegradc tegradc.0: DT parsed successfully
[    0.974661] tegradc tegradc.0: Display dc.ffffff800b100000 registered with id=0
[    0.975208] tegradc tegradc.0: Bootloader disp_param detected. Detected mode: 1920x1080 (on 0x0mm) pclk=148350781
[    0.978367] tegradc tegradc.0: probed
[    0.983124] tegra-apbdma 60020000.dma: Tegra20 APB DMA driver register 32 channels
[    0.989847] tegra-adma 702e2000.adma: Tegra210 ADMA driver registered 22 channels
[    0.990716] tegra-fuse-burn 7000f800.efuse:efuse-burn: shutdown limit check disabled
[    0.990753] tegra-fuse-burn 7000f800.efuse:efuse-burn: Fuse burn driver initialized
[    0.991093] kfuse 7000fc00.kfuse: initialized
[    0.992661] tegra-pmc-iopower pmc-iopower: Regulator supply iopower-dbg-supply not available
[    0.995165] tegra-pcie 1003000.pcie: PCI host bridge to bus 0000:00
[    0.995192] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.995211] pci_bus 0000:00: root bus resource [mem 0x13000000-0x1fffffff]
[    0.995230] pci_bus 0000:00: root bus resource [mem 0x20000000-0x3fffffff pref]
[    0.995253] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.995289] pci 0000:00:01.0: [10de:0fae] type 01 class 0x060400
[    0.995358] pci 0000:00:01.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.995529] pci 0000:00:02.0: [10de:0faf] type 01 class 0x060400
[    0.995588] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.995740] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.995770] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.995903] pci 0000:01:00.0: [8086:2723] type 00 class 0x028000
[    0.995934] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit]
[    0.996078] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    0.996659] tegra-pmc-iopower pmc-iopower: NO_IOPOWER setting 0x0
[    0.997337] tegra-dfll-pwm 70110000.pwm: DFLL_PWM regulator is available now
[    0.997361] vdd-cpu: 708 <--> 1322 mV at 708 mV
[    0.997663] pwm-regulator pwm_regulators:pwm-regulator@0: PWM regulator registration passed
[    0.998759] vdd-gpu: applied init 1000000uV constraint
[    0.998784] vdd-gpu: 708 <--> 1323 mV at 997 mV
[    0.999126] pwm-regulator pwm_regulators:pwm-regulator@1: PWM regulator registration passed
[    1.001816] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.001901] No Device Node present for smmu client: serial8250 !!
[    1.001923] platform serial8250: No iommus property found in DT node, got swgids from fixup(101004000)
[    1.001968] iommu: Adding device serial8250 to group 35
[    1.003357] console [ttyS0] disabled
[    1.003411] 70006000.serial: ttyS0 at MMIO 0x70006000 (irq = 63, base_baud = 25500000) is a Tegra
[    1.005236] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.005363] pci 0000:02:00.0: [10ec:8168] type 00 class 0x020000
[    1.005385] pci 0000:02:00.0: reg 0x10: [io  0x0000-0x00ff]
[    1.005411] pci 0000:02:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit]
[    1.005429] pci 0000:02:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit]
[    1.005552] pci 0000:02:00.0: supports D1 D2
[    1.005555] pci 0000:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    1.017235] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[    1.017277] pci 0000:00:01.0: BAR 14: assigned [mem 0x13000000-0x130fffff]
[    1.017282] pci 0000:00:02.0: BAR 14: assigned [mem 0x13100000-0x131fffff]
[    1.017286] pci 0000:00:02.0: BAR 13: assigned [io  0x1000-0x1fff]
[    1.017294] pci 0000:01:00.0: BAR 0: assigned [mem 0x13000000-0x13003fff 64bit]
[    1.017310] pci 0000:00:01.0: PCI bridge to [bus 01]
[    1.017316] pci 0000:00:01.0:   bridge window [mem 0x13000000-0x130fffff]
[    1.017326] pci 0000:02:00.0: BAR 4: assigned [mem 0x13100000-0x13103fff 64bit]
[    1.017341] pci 0000:02:00.0: BAR 2: assigned [mem 0x13104000-0x13104fff 64bit]
[    1.017356] pci 0000:02:00.0: BAR 0: assigned [io  0x1000-0x10ff]
[    1.017363] pci 0000:00:02.0: PCI bridge to [bus 02]
[    1.017367] pci 0000:00:02.0:   bridge window [io  0x1000-0x1fff]
[    1.017372] pci 0000:00:02.0:   bridge window [mem 0x13100000-0x131fffff]
[    1.017690] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
[    1.017693] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    1.017698] pcie_pme 0000:00:01.0:pcie001: service driver pcie_pme loaded
[    1.017793] aer 0000:00:01.0:pcie002: service driver aer loaded
[    1.017963] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt
[    1.017966] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
[    1.017971] pcie_pme 0000:00:02.0:pcie001: service driver pcie_pme loaded
[    1.018052] aer 0000:00:02.0:pcie002: service driver aer loaded
[    2.005174] tegra_cec 70015000.tegra_cec: Can't find physical addresse.
[    2.005178] tegra_cec 70015000.tegra_cec: tegra_cec_init Done.
[    3.430457] console [ttyS0] enabled
[    3.430806] Console: switching to colour frame buffer device 240x67
[    3.431213] 70006040.serial: ttyTHS1 at MMIO 0x70006040 (irq = 64, base_baud = 0) is a TEGRA_UART
[    3.431438] serial-tegra 70006200.serial: RX in PIO mode
[    3.431519] 70006200.serial: ttyTHS2 at MMIO 0x70006200 (irq = 65, base_baud = 0) is a TEGRA_UART
[    3.432242] [drm] Initialized
[    3.467461] tegradc tegradc.0: fb registered
[    3.475936] brd: module loaded
[    3.477709] tegradc tegradc.0: DC initialized, skipping tegra_dc_program_mode.
[    3.477733] tegradc tegradc.0: hdmi: tmds rate:148351K prod-setting:prod_c_hdmi_75m_150m
[    3.478081] tegradc tegradc.0: hdmi: get RGB quant from REG programmed by BL.
[    3.478089] tegradc tegradc.0: hdmi: get YCC quant from REG programmed by BL.
[    3.503840] extcon-disp-state extcon:disp-state: cable 47 state 1
[    3.503842] Extcon AUX1(HDMI) enable
[    3.504388] tegradc tegradc.1: disp1 connected to head1->/host1x/sor
[    3.504441] tegradc tegradc.1: No lt-data, using default setting
[    3.504459] tegradc tegradc.1: No hpd-gpio in DT
[    3.504484] tegradc tegradc.1: DT parsed successfully
[    3.504511] tegradc tegradc.1: Display dc.ffffff800fa80000 registered with id=1
[    3.505366] tegradc tegradc.1: dpd enable lookup fail:-19
[    3.509409] tegradc tegradc.1: probed
[    3.538989] tegradc tegradc.0: nominal-pclk:148351648 parent:148350781 div:1.0 pclk:148350781 146868084~161703244
[    3.566938] tegradc tegradc.1: fb registered
[    3.574212] loop: module loaded
[    3.574393] hpd: state 7 (Takeover from bootloader), hpd 0, pending_hpd_evt 1
[    3.574397] hpd: switching from state 7 (Takeover from bootloader) to state 0 (Reset)
[    3.574404] hpd: state 0 (Reset), hpd 0, pending_hpd_evt 0
[    3.597834] tegradc tegradc.1: blank - powerdown
[    3.597884] tegra_profiler: version: 1.141, samples/io: 48/28
[    3.608254] extcon-disp-state extcon:disp-state: cable 44 state 0 already set.
[    3.608260] tegra_profiler: auth: init
[    3.615899] THERMAL EST: found 2 subdevs
[    3.615903] THERMAL EST num_resources: 0
[    3.615907] [THERMAL EST subdev 0]
[    3.615911] [THERMAL EST subdev 1]
[    3.616204] thermal thermal_zone5: Registering thermal zone thermal_zone5 for type thermal-fan-est
[    3.616206] THERMAL EST: thz register success.
[    3.616301] THERMAL EST: end of probe, return err: 0
[    3.616863] sd: No Scsi addr parsed to reserve index
[    3.616884] hisi_sas: driver version v1.6
[    3.620058] tegra-qspi 70410000.spi: Prod settings list not found
[    3.627150] qspi_mtd spi32766.0: MX25U3235F (4096 Kbytes)
[    3.627157] qspi_mtd spi32766.0: mtd .name = spi32766.0, .size = 0x400000 (4MiB) .erasesize = 0x00001000 (4KiB) .numeraseregions = 0
[    3.643312] libphy: Fixed MDIO Bus: probed
[    3.643764] tun: Universal TUN/TAP device driver, 1.6
[    3.643766] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    3.644280] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    3.644282] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    3.644322] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    3.644324] igb: Copyright (c) 2007-2014 Intel Corporation.
[    3.644359] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[    3.644361] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    3.644393] Intel(R) 10GbE PCI Express Linux Network Driver - version 4.6.4
[    3.644395] Copyright(c) 1999 - 2017 Intel Corporation.
[    3.644939] r8168 0000:02:00.0: enabling device (0000 -> 0003)
[    3.644955] r8168 Gigabit Ethernet driver 8.045.08-NAPI loaded
[    3.659088] r8168 0000:02:00.0 (unnamed net_device) (uninitialized): Invalid ethernet address 00:00:00:00:00:00, trying device tree node
[    3.659160] r8168 0000:02:00.0 (unnamed net_device) (uninitialized): Found valid ethernet address 00:04:4b:eb:f1:11 from device tree
[    3.659553] r8168: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[    3.659586] r8168  Copyright (C) 2017  Realtek NIC software team <nicfae@realtek.com>
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>.
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>.
[    3.659929] PPP generic driver version 2.4.2
[    3.660056] PPP BSD Compression module registered
[    3.660058] PPP Deflate Compression module registered
[    3.660072] PPP MPPE Compression module registered
[    3.660076] NET: Registered protocol family 24
[    3.660188] VFIO - User Level meta-driver version: 0.3
[    3.665717] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.665728] ehci-pci: EHCI PCI platform driver
[    3.665757] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    3.665772] ohci-pci: OHCI PCI platform driver
[    3.665802] ohci-platform: OHCI generic platform driver
[    3.667549] tegra-xusb 70090000.xusb: USB2 port 0 has OTG_CAP
[    3.668105] tegra-xusb-padctl 7009f000.xusb_padctl: enabled OTG on UTMI pad 0
[    3.669162] tegra-xusb 70090000.xusb: extcon 0: ffffffc0f9722000 id
[    3.670536] usbcore: registered new interface driver uas
[    3.670571] usbcore: registered new interface driver usb-storage
[    3.671589] tegra-xusb 70090000.xusb: Firmware timestamp: 2019-10-17 15:58:59 UTC, Version: 50.25 release
[    3.671618] tegra-xusb 70090000.xusb: xHCI Host Controller
[    3.671633] tegra-xusb 70090000.xusb: new USB bus registered, assigned bus number 1
[    3.672361] tegra-xusb 70090000.xusb: hcc params 0x0184f525 hci version 0x100 quirks 0x00050010
[    3.672397] tegra-xusb 70090000.xusb: irq 61, io mem 0x70090000
[    3.672541] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    3.672547] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.672551] usb usb1: Product: xHCI Host Controller
[    3.672554] usb usb1: Manufacturer: Linux 4.9.140-l4t-r32.4.2 xhci-hcd
[    3.672558] usb usb1: SerialNumber: 70090000.xusb
[    3.672960] hub 1-0:1.0: USB hub found
[    3.672991] hub 1-0:1.0: 5 ports detected
[    3.673505] tegra-xusb 70090000.xusb: xHCI Host Controller
[    3.673514] tegra-xusb 70090000.xusb: new USB bus registered, assigned bus number 2
[    3.673580] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    3.673665] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    3.673670] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    3.673674] usb usb2: Product: xHCI Host Controller
[    3.673677] usb usb2: Manufacturer: Linux 4.9.140-l4t-r32.4.2 xhci-hcd
[    3.673681] usb usb2: SerialNumber: 70090000.xusb
[    3.674021] hub 2-0:1.0: USB hub found
[    3.674049] hub 2-0:1.0: 4 ports detected
[    3.675519] tegra-xudc-new 700d0000.xudc: device count: 1
[    3.676416] tegra-xudc-new 700d0000.xudc: USB charger detection disabled
[    3.676425] tegra-xudc-new 700d0000.xudc: vbus state: 0
[    3.676445] tegra-xudc-new 700d0000.xudc: entering ELPG
[    3.676657] tegra-xudc-new 700d0000.xudc: entering ELPG done
[    3.677056] mousedev: PS/2 mouse device common for all mice
[    3.677103] usbcore: registered new interface driver xpad
[    3.773223] tegra-xusb-padctl 7009f000.xusb_padctl: power on UTMI pads 1
[    3.773276] tegra-xusb-padctl 7009f000.xusb_padctl: power on UTMI pads 2
[    3.908351] max77686-rtc max77620-rtc: rtc core: registered max77620-rtc as rtc0
[    3.925690] rtc rtc1: alarm rtc device
[    3.925702] tegra_rtc 7000e000.rtc: rtc core: registered 7000e000.rtc as rtc1
[    3.925719] tegra_rtc 7000e000.rtc: Tegra internal Real Time Clock
[    3.926013] i2c /dev entries driver
[    3.927160] i2c i2c-6: Added multiplexed i2c bus 7
[    3.927429] i2c i2c-6: Added multiplexed i2c bus 8
[    3.927434] i2c-mux-gpio cam_i2cmux: 2 port mux on Tegra I2C adapter adapter
[    3.927915] imx219 7-0010: tegracam sensor driver:imx219_v2.0.6
[    3.990058] imx219 8-0010: tegracam sensor driver:imx219_v2.0.6
[    4.001176] usb 1-2: new high-speed USB device number 2 using tegra-xusb
[    4.017548] tegra-vii2c 546c0000.i2c: no acknowledge from address 0x10
[    4.017624] imx219 8-0010: imx219_board_setup: error during i2c read probe (-121)
[    4.017654] imx219 8-0010: board setup failed
[    4.017714] imx219: probe of 8-0010 failed with error -121
[    4.024830] max77620-power max77620-power: Event recorder REG_NVERC : 0x0
[    4.061538] tegra_soctherm 700e2000.soctherm: pllx_offset tz:0 max:14, min:2
[    4.061543] tegra_soctherm 700e2000.soctherm: pllx_offset tz:2 max:8, min:4
[    4.061833] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.061845] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when cpu reaches 102500 mC
[    4.061851] tegra_soctherm 700e2000.soctherm: throttrip: will throttle when cpu reaches 100500 mC
[    4.061974] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.061983] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when gpu reaches 103000 mC
[    4.061988] tegra_soctherm 700e2000.soctherm: throttrip: will throttle when gpu reaches 101000 mC
[    4.062126] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.062134] tegra_soctherm 700e2000.soctherm: thermtrip: will shut down when pll reaches 127000 mC
[    4.062138] tegra_soctherm 700e2000.soctherm: throttrip: pll: missing hot temperature
[    4.062307] tegra_soctherm 700e2000.soctherm: failed to register sensor: -19
[    4.062851] tegra_aotag tegra-aotag: Registering sensor 0
[    4.062894] tegra_aotag tegra-aotag: Invalid temp readout
[    4.062914] tegra_aotag tegra-aotag: Invalid temp readout
[    4.062930] tegra_aotag tegra-aotag: Bound to TZ : ID 0
[    4.062938] tegra_aotag tegra-aotag: Probe done [SUCCESS]:0
[    4.063640] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.063654] tegra_dfll_action dfll-cdev-cap: Tegra DFLL 'cap cooling device' registered
[    4.063796] tegra_aotag tegra-aotag: Invalid temp readout
[    4.063817] tegra_dfll_action dfll-cdev-floor: Tegra DFLL 'floor cooling device' registered
[    4.064139] parse_throttle_dt_data: Num cap clks = 6
[    4.064143] parse_throttle_dt_data: clk=cclk_g type=2
[    4.064149] parse_throttle_dt_data: clk=gpu type=4
[    4.064167] parse_throttle_dt_data: clk=cap.throttle.c2bus type=0
[    4.064183] parse_throttle_dt_data: clk=cap.throttle.c3bus type=0
[    4.064201] parse_throttle_dt_data: clk=cap.throttle.sclk type=0
[    4.064203] parse_throttle_dt_data: clk=emc type=3
[    4.066313] usb 1-2: New USB device found, idVendor=0bda, idProduct=5411
[    4.066319] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    4.066323] usb 1-2: Product: 4-Port USB 2.1 Hub
[    4.066330] usb 1-2: Manufacturer: Generic
[    4.066446] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.066530] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.066639] tegra_throttle_probe: probe successful. #cdevs=4
[    4.067178] FAN dev name: pwm-fan
[    4.067220] FAN:gpio request success.
[    4.067271] pwm_fan_driver pwm-fan: cap state:7, cap pwm:255
[    4.067494] pwm_fan_driver pwm-fan: got pwm for fan. polarity is normal
[    4.067610] pwm_fan_driver pwm-fan: fan tach request irq success
[    4.067616] pwm_fan_driver pwm-fan: tach period: 1000
[    4.067685] pwm_fan_driver pwm-fan: index 0: pwm=0, rpm=0, rru=40, rrd=40, state:2
[    4.067690] pwm_fan_driver pwm-fan: index 1: pwm=80, rpm=1000, rru=2, rrd=2, state:2
[    4.067694] pwm_fan_driver pwm-fan: index 2: pwm=120, rpm=2000, rru=1, rrd=1, state:2
[    4.067698] pwm_fan_driver pwm-fan: index 3: pwm=160, rpm=3000, rru=1, rrd=1, state:2
[    4.067702] pwm_fan_driver pwm-fan: index 4: pwm=255, rpm=4000, rru=1, rrd=1, state:3
[    4.067705] pwm_fan_driver pwm-fan: index 5: pwm=255, rpm=5000, rru=1, rrd=1, state:3
[    4.067709] pwm_fan_driver pwm-fan: index 6: pwm=255, rpm=6000, rru=1, rrd=1, state:3
[    4.067713] pwm_fan_driver pwm-fan: index 7: pwm=255, rpm=7000, rru=1, rrd=1, state:4
[    4.067717] pwm_fan_driver pwm-fan: index 8: pwm=255, rpm=10000, rru=1, rrd=1, state:4
[    4.067721] pwm_fan_driver pwm-fan: index 9: pwm=255, rpm=11000, rru=1, rrd=1, state:4
[    4.068585] device-mapper: uevent: version 1.0.3
[    4.068958] device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: dm-devel@redhat.com
[    4.071500] tegra210-cpufreq cpufreq: probe()...completed
[    4.072336] sdhci: Secure Digital Host Controller Interface driver
[    4.072338] sdhci: Copyright(c) Pierre Ossman
[    4.072340] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.072963] sdhci-tegra sdhci-tegra.0: Got CD GPIO
[    4.073452] sdhci-tegra sdhci-tegra.0: Client registration for eMC Successful
[    4.073596] sdhci-tegra sdhci-tegra.0: wakeup init done, cdirq 300
[    4.077454] hub 1-2:1.0: USB hub found
[    4.079417] tegra-se 70012000.se: tegra_se_probe: complete
[    4.080032] hidraw: raw HID events driver (C) Jiri Kosina
[    4.081270] usbcore: registered new interface driver usbhid
[    4.081272] usbhid: USB HID core driver
[    4.082882] hub 1-2:1.0: 4 ports detected
[    4.102969] tegra21x_actmon 6000c800.actmon: in actmon_register()...
[    4.103339] tegra21x_actmon 6000c800.actmon: initialization Completed for the device mc_all
[    4.105572] nvpmodel: initialized successfully
[    4.106898] usbcore: registered new interface driver snd-usb-audio
[    4.106968] No Device Node present for smmu client: snd-soc-dummy !!
[    4.106974] platform snd-soc-dummy: No iommus property found in DT node, got swgids from fixup(101004000)
[    4.106995] iommu: Adding device snd-soc-dummy to group 36
[    4.121316] mmc0: SDHCI controller on sdhci-tegra.0 [sdhci-tegra.0] using ADMA 64-bit with 64 bit addr
[    4.133363] OPE platform probe
[    4.133475] OPE platform probe successful
[    4.133762] OPE platform probe
[    4.133881] OPE platform probe successful
[    4.167853] tegra-asoc: sound: ADMAIF1 <-> ADMAIF1 mapping ok
[    4.168013] tegra-asoc: sound: ADMAIF2 <-> ADMAIF2 mapping ok
[    4.168162] tegra-asoc: sound: ADMAIF3 <-> ADMAIF3 mapping ok
[    4.168312] tegra-asoc: sound: ADMAIF4 <-> ADMAIF4 mapping ok
[    4.168463] tegra-asoc: sound: ADMAIF5 <-> ADMAIF5 mapping ok
[    4.168611] tegra-asoc: sound: ADMAIF6 <-> ADMAIF6 mapping ok
[    4.168761] tegra-asoc: sound: ADMAIF7 <-> ADMAIF7 mapping ok
[    4.168912] tegra-asoc: sound: ADMAIF8 <-> ADMAIF8 mapping ok
[    4.169058] tegra-asoc: sound: ADMAIF9 <-> ADMAIF9 mapping ok
[    4.169227] tegra-asoc: sound: ADMAIF10 <-> ADMAIF10 mapping ok
[    4.184677] u32 classifier
[    4.184680]     Actions configured
[    4.184928] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    4.219793] ip_tables: (C) 2000-2006 Netfilter Core Team
[    4.228762] Initializing XFRM netlink socket
[    4.229678] NET: Registered protocol family 10
[    4.230660] NET: Registered protocol family 17
[    4.230675] NET: Registered protocol family 15
[    4.230775] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    4.230902] Bluetooth: RFCOMM socket layer initialized
[    4.230913] Bluetooth: RFCOMM ver 1.11
[    4.230919] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[    4.230925] Bluetooth: HIDP socket layer initialized
[    4.230950] 9pnet: Installing 9P2000 support
[    4.230986] Key type dns_resolver registered
[    4.237641] Registered cp15_barrier emulation handler
[    4.237651] Registered setend emulation handler
[    4.239987] registered taskstats version 1
[    4.240696] Adding domain ve2-pd to PM domain host1x-pd
[    4.240948] Adding domain ve-pd to PM domain host1x-pd
[    4.243482] isp 54600000.isp: initialized
[    4.244529] isp 54680000.isp: initialized
[    4.257186] usb 1-3: new full-speed USB device number 3 using tegra-xusb
[    4.257736] last reset is due to software reset
[    4.257739] KERNEL: PMC reset status reg: 0x3
[    4.257829] BL: PMC reset status reg: 0x3
[    4.257832] BL: PMIC poweroff Event Recorder: 0x0
[    4.259543] clk_cbus_recalc_rate: no gbus parent
[    4.259549] clk_cbus_round_rate: no gbus parent
[    4.259551] clk_cbus_round_rate: no gbus parent
[    4.259558] clk_cbus_recalc_rate: no gbus parent
[    4.259673] clk_cbus_recalc_rate: no gbus parent
[    4.259676] clk_cbus_round_rate: no gbus parent
[    4.259678] clk_cbus_round_rate: no gbus parent
[    4.259683] clk_cbus_recalc_rate: no gbus parent
[    4.260360] tegra_dvfs: GPU-cap: registered
[    4.260425] tegra dvfs: vdd-cpu: nominal 1168mV, offset 708000uV, step 19200uV, scaling enabled
[    4.260427] tegra dvfs: vdd-core: nominal 1125mV, offset 600000uV, step 12500uV, scaling enabled
[    4.260429] tegra dvfs: vdd-gpu: nominal 1078mV, offset 708000uV, step 10000uV, scaling enabled
[    4.335555] tegra_dvfs: vdd-gpu-vts: registered
[    4.357261] usb 1-3: New USB device found, idVendor=8087, idProduct=0029
[    4.357266] usb 1-3: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    4.367376] tegra_core_action core_dvfs_cdev_floor: Tegra CORE DVFS 'floor cooling device' registered
[    4.368242] tegra_core_action core_dvfs_cdev_cap: Tegra CORE DVFS 'cap cooling device' registered
[    4.368931] input: gpio-keys as /devices/gpio-keys/input/input0
[    4.417760] tegra_rtc 7000e000.rtc: setting system clock to 2020-08-12 02:39:32 UTC (1597199972)
[    4.418913] vi 54080000.vi: vi_probe: ++
[    4.525234] usb 1-2.1: new full-speed USB device number 4 using tegra-xusb
[    4.556123] vi 54080000.vi: initialized
[    4.587634] vi 54080000.vi: subdev nvcsi--2 bound
[    4.587650] vi 54080000.vi: subdev imx219 7-0010 bound
[    4.588483] vi 54080000.vi: subdev nvcsi--1 bound
[    4.590507] Disable partitions left on by BL
[    4.590510]   disb
[    4.590563] bwmgr: missing cdev-type property
[    4.590671] tegra_soctherm 700e2000.soctherm: soctherm: trip temperature -2147483647 forced to -127000
[    4.590678] DRAM derating cdev registered.
[    4.591904] vdd-fan: disabling
[    4.591908] vdd-usb-vbus: disabling
[    4.591910] vdd-usb-vbus2: disabling
[    4.591922] vddio-sdmmc3-ap: disabling
[    4.598029] avdd-io-edp-1v05: disabling
[    4.598032] vdd-usb-hub-en: disabling
[    4.604260] ALSA device list:
[    4.604263]   #1: tegra-snd-t210ref-mobile-rt565x
[    4.624621] usb 1-2.1: New USB device found, idVendor=0403, idProduct=6010
[    4.624626] usb 1-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    4.624630] usb 1-2.1: Product: Dual RS232
[    4.624633] usb 1-2.1: Manufacturer: FTDI
[    4.757230] usb 1-2.2: new full-speed USB device number 5 using tegra-xusb
[    4.814164] usb 1-2.2: New USB device found, idVendor=1a86, idProduct=7523
[    4.814168] usb 1-2.2: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[    4.814172] usb 1-2.2: Product: USB Serial
[    4.929221] usb 1-2.4: new full-speed USB device number 6 using tegra-xusb
[    4.979572] usb 1-2.4: New USB device found, idVendor=222a, idProduct=0001
[    4.979576] usb 1-2.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    5.027446] hid-generic 0003:222A:0001.0002: hidraw0: USB HID v1.10 Device [HID 222a:0001] on usb-70090000.xusb-2.4/input1
[    5.066908] mmc0: hw tuning done ...
[    5.066942] mmc0: new ultra high speed SDR104 SDHC card at address 59b4
[    5.067538] mmcblk0: mmc0:59b4 LX32G 29.4 GiB
[    5.143099]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16
[    5.196841] Extcon DP: HPD disabled
[    5.196844] hpd: hpd_switch 0
[    5.196848] hpd: switching from state 0 (Reset) to state 1 (Check Plug)
[    5.203692] hpd: state 1 (Check Plug), hpd 0, pending_hpd_evt 0
[    5.203696] hpd: switching from state 1 (Check Plug) to state 3 (Disabled)
[    5.223635] input: tegra-hda HDMI/DP,pcm=3 as /devices/70030000.hda/sound/card0/input1
[    5.231882] tegradc tegradc.0: blank - powerdown
[    5.235878] Freeing unused kernel memory: 11456K
[    5.280219] extcon-disp-state extcon:disp-state: cable 47 state 0
[    5.280221] Extcon AUX1(HDMI) disable
[    5.308596] tegradc tegradc.0: unblank
[    5.344231] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[    5.344278] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[    5.345316] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[    5.345323] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[    5.379981] extcon-disp-state extcon:disp-state: cable 47 state 1
[    5.379983] Extcon AUX1(HDMI) enable
[    5.380298] tegradc tegradc.0: sync windows ret = 250
[    5.470075] extcon-disp-state extcon:disp-state: cable 51 state 1
[    5.477641] Extcon HDMI: HPD enabled
[    5.482793] tegradc tegradc.0: hdmi: plugged
[    6.777926] random: crng init done
[    7.836138] tegra-xusb 70090000.xusb: Upgrade port 0 to USB3.0
[    7.852375] tegra-xusb 70090000.xusb: Upgrade port 1 to USB3.0
[    8.166004] usb 2-1: new SuperSpeed USB device number 2 using tegra-xusb
[    8.207624] usb 2-1: New USB device found, idVendor=0bda, idProduct=0411
[    8.221228] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.232677] usb 2-1: Product: 4-Port USB 3.1 Hub
[    8.241456] usb 2-1: Manufacturer: Generic
[    8.252885] hub 2-1:1.0: USB hub found
[    8.261619] hub 2-1:1.0: 4 ports detected
[    8.425296] usb 2-1: usb_suspend_both: status 0
[    8.434140] usb usb2: usb_suspend_both: status 0
[    9.218412] EXT4-fs (mmcblk0p13): couldn't mount as ext3 due to feature incompatibilities
[    9.230061] EXT4-fs (mmcblk0p13): couldn't mount as ext2 due to feature incompatibilities
[    9.244745] EXT4-fs (mmcblk0p13): mounted filesystem with ordered data mode. Opts: (null)
[    9.282246] EXT4-fs (mmcblk0p15): mounted filesystem with ordered data mode. Opts: (null)
[    9.596267] EXT4-fs (mmcblk0p13): re-mounted. Opts: (null)
[    9.795996] cgroup: cgroup2: unknown option "nsdelegate"
[    9.968221] systemd[1]: File /lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    9.992431] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[   10.093432] systemd[1]: /lib/systemd/system/chronyd.service:9: PIDFile= references path below legacy directory /var/run/, updating /var/run/chrony/chronyd.pid \xe2\x86\x92 /run/chrony/chronyd.pid; please update the unit file accordingly.
[   10.389988] nvgpu: 57000000.gpu           gm20b_init_clk_setup_sw:1268 [INFO]  GPCPLL initial settings: NA mode, M=1, N=34, P=3 (id = 1)
[   12.556103] EXT4-fs (mmcblk0p16): mounted filesystem with ordered data mode. Opts: (null)
[   13.986708] compat: loading out-of-tree module taints kernel.
[   13.996722] Loading modules backported from iwlwifi
[   14.003340] iwlwifi-stack-public:release/core52:8470:f4c7b3b5
[   14.070130] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[   14.084137] cfg80211: Problem loading in-kernel X.509 certificate (-129)
[   14.099080] cfg80211: loaded regulatory.db is malformed or signature is missing/invalid
[   14.138997] usbcore: registered new interface driver usbserial
[   14.139037] Intel(R) Wireless WiFi driver for Linux
[   14.139039] Copyright(c) 2003- 2015 Intel Corporation
[   14.139251] iwlwifi 0000:01:00.0: enabling device (0000 -> 0002)
[   14.145453] iwlwifi 0000:01:00.0: Direct firmware load for iwl-dbg-cfg.ini failed with error -2
[   14.145457] iwlwifi 0000:01:00.0: Falling back to user helper
[   14.152357] usbcore: registered new interface driver btusb
[   14.202809] usbcore: registered new interface driver ch341
[   14.211171] usbserial: USB Serial support registered for ch341-uart
[   14.211525] usbcore: registered new interface driver ftdi_sio
[   14.222929] input: HID 222a:0001 as /devices/70090000.xusb/usb1/1-2/1-2.4/1-2.4:1.0/0003:222A:0001.0001/input/input2
[   14.223189] hid-multitouch 0003:222A:0001.0001: input,hidraw1: USB HID v1.10 Device [HID 222a:0001] on usb-70090000.xusb-2.4/input0
[   14.252322] usbserial: USB Serial support registered for FTDI USB Serial Device
[   14.261021] ftdi_sio 1-2.1:1.0: FTDI USB Serial Device converter detected
[   14.265447] ch341 1-2.2:1.0: ch341-uart converter detected
[   14.276896] usb 1-2.1: Detected FT2232C
[   14.282507] usb 1-2.1: FTDI USB Serial Device converter now attached to ttyUSB1
[   14.291720] usb 1-2.2: ch341-uart converter now attached to ttyUSB0
[   14.299929] ftdi_sio 1-2.1:1.1: FTDI USB Serial Device converter detected
[   14.310240] usb 1-2.1: Detected FT2232C
[   14.317300] usb 1-2.1: FTDI USB Serial Device converter now attached to ttyUSB2
[   74.892920] iwlwifi 0000:01:00.0: api flags index 2 larger than supported by driver
[   74.911487] iwlwifi 0000:01:00.0: TLV_FW_FSEQ_VERSION: FSEQ Version: 89.3.35.22
[   74.934248] iwlwifi 0000:01:00.0: Found debug destination: EXTERNAL_DRAM
[   74.934272] iwlwifi 0000:01:00.0: Found debug configuration: 0
[   74.942159] iwlwifi 0000:01:00.0: loaded firmware version 55.d9698065.0 cc-a0-55.ucode op_mode iwlmvm
[   74.942437] iwlwifi 0000:01:00.0: Direct firmware load for iwl-debug-yoyo.bin failed with error -2
[   74.942455] iwlwifi 0000:01:00.0: Falling back to user helper
[   77.549619] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   77.559318] eth0: 0xffffff800a91e000, 00:04:4b:eb:f1:11, IRQ 406
[   77.622034] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   78.737959] Bridge firewalling registered
[   79.017032] Netfilter messages via NETLINK v0.30.
[   79.028602] ctnetlink v0.93: registering with nfnetlink.
[   79.217878] IPv6: ADDRCONF(NETDEV_UP): supervisor0: link is not ready
[   79.454222] IPv6: ADDRCONF(NETDEV_UP): balena0: link is not ready
[   79.613839] IPv6: ADDRCONF(NETDEV_UP): br-cf2fb069bbe8: link is not ready
[   81.717455] r8168: eth0: link up
[   81.720742] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   85.855224] br-cf2fb069bbe8: port 1(vethff86f6b) entered blocking state
[   85.865411] br-cf2fb069bbe8: port 1(vethff86f6b) entered disabled state
[   85.872476] device vethff86f6b entered promiscuous mode
[   85.878231] IPv6: ADDRCONF(NETDEV_UP): vethff86f6b: link is not ready
[   85.975377] br-cf2fb069bbe8: port 2(vethd329f34) entered blocking state
[   85.984822] br-cf2fb069bbe8: port 2(vethd329f34) entered disabled state
[   85.992147] device vethd329f34 entered promiscuous mode
[   86.004509] IPv6: ADDRCONF(NETDEV_UP): vethd329f34: link is not ready
[   86.012645] br-cf2fb069bbe8: port 2(vethd329f34) entered blocking state
[   86.019326] br-cf2fb069bbe8: port 2(vethd329f34) entered forwarding state
[   86.029040] br-cf2fb069bbe8: port 2(vethd329f34) entered disabled state
[   86.138550] br-cf2fb069bbe8: port 3(vethfbacd6d) entered blocking state
[   86.145230] br-cf2fb069bbe8: port 3(vethfbacd6d) entered disabled state
[   86.152158] device vethfbacd6d entered promiscuous mode
[   86.157871] IPv6: ADDRCONF(NETDEV_UP): vethfbacd6d: link is not ready
[   86.164421] br-cf2fb069bbe8: port 3(vethfbacd6d) entered blocking state
[   86.171045] br-cf2fb069bbe8: port 3(vethfbacd6d) entered forwarding state
[   86.179106] IPv6: ADDRCONF(NETDEV_CHANGE): br-cf2fb069bbe8: link becomes ready
[   86.186645] br-cf2fb069bbe8: port 3(vethfbacd6d) entered disabled state
[   86.319511] br-cf2fb069bbe8: port 4(veth5f20e73) entered blocking state
[   86.326175] br-cf2fb069bbe8: port 4(veth5f20e73) entered disabled state
[   86.333104] device veth5f20e73 entered promiscuous mode
[   86.338700] IPv6: ADDRCONF(NETDEV_UP): veth5f20e73: link is not ready
[   86.345333] br-cf2fb069bbe8: port 4(veth5f20e73) entered blocking state
[   86.351964] br-cf2fb069bbe8: port 4(veth5f20e73) entered forwarding state
[   86.510305] br-cf2fb069bbe8: port 5(vethb983baa) entered blocking state
[   86.519293] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[   86.526525] device vethb983baa entered promiscuous mode
[   86.536163] IPv6: ADDRCONF(NETDEV_UP): vethb983baa: link is not ready
[   86.549323] br-cf2fb069bbe8: port 5(vethb983baa) entered blocking state
[   86.555985] br-cf2fb069bbe8: port 5(vethb983baa) entered forwarding state
[   86.728929] br-cf2fb069bbe8: port 6(veth1431b77) entered blocking state
[   86.735675] br-cf2fb069bbe8: port 6(veth1431b77) entered disabled state
[   86.766524] device veth1431b77 entered promiscuous mode
[   86.801862] IPv6: ADDRCONF(NETDEV_UP): veth1431b77: link is not ready
[   86.818484] br-cf2fb069bbe8: port 6(veth1431b77) entered blocking state
[   86.825127] br-cf2fb069bbe8: port 6(veth1431b77) entered forwarding state
[   86.869676] br-cf2fb069bbe8: port 4(veth5f20e73) entered disabled state
[   86.876595] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[   86.889630] br-cf2fb069bbe8: port 6(veth1431b77) entered disabled state
[   86.915674] br-cf2fb069bbe8: port 7(vethf0594d1) entered blocking state
[   86.924398] br-cf2fb069bbe8: port 7(vethf0594d1) entered disabled state
[   86.959309] device vethf0594d1 entered promiscuous mode
[   87.013374] IPv6: ADDRCONF(NETDEV_UP): vethf0594d1: link is not ready
[   87.040098] br-cf2fb069bbe8: port 7(vethf0594d1) entered blocking state
[   87.046732] br-cf2fb069bbe8: port 7(vethf0594d1) entered forwarding state
[   87.304688] supervisor0: port 1(veth395f796) entered blocking state
[   87.326832] supervisor0: port 1(veth395f796) entered disabled state
[   87.343564] device veth395f796 entered promiscuous mode
[   87.361764] IPv6: ADDRCONF(NETDEV_UP): veth395f796: link is not ready
[   87.379393] supervisor0: port 1(veth395f796) entered blocking state
[   87.385684] supervisor0: port 1(veth395f796) entered forwarding state
[   87.423330] br-cf2fb069bbe8: port 7(vethf0594d1) entered disabled state
[   87.434594] supervisor0: port 1(veth395f796) entered disabled state
[   88.164692] supervisor0: port 2(veth19e8cca) entered blocking state
[   88.173170] supervisor0: port 2(veth19e8cca) entered disabled state
[   88.183789] device veth19e8cca entered promiscuous mode
[   88.202731] IPv6: ADDRCONF(NETDEV_UP): veth19e8cca: link is not ready
[   88.231631] supervisor0: port 2(veth19e8cca) entered blocking state
[   88.237935] supervisor0: port 2(veth19e8cca) entered forwarding state
[   88.261301] supervisor0: port 2(veth19e8cca) entered disabled state
[   88.643819] eth0: renamed from vethb02cdcb
[   88.678052] IPv6: ADDRCONF(NETDEV_CHANGE): vethff86f6b: link becomes ready
[   88.687210] br-cf2fb069bbe8: port 1(vethff86f6b) entered blocking state
[   88.693858] br-cf2fb069bbe8: port 1(vethff86f6b) entered forwarding state
[   88.976342] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   89.124543] eth0: renamed from vethc2ed264
[   89.148243] IPv6: ADDRCONF(NETDEV_CHANGE): vethd329f34: link becomes ready
[   89.156892] br-cf2fb069bbe8: port 2(vethd329f34) entered blocking state
[   89.163530] br-cf2fb069bbe8: port 2(vethd329f34) entered forwarding state
[   89.242645] eth0: renamed from veth4a8f50e
[   89.301796] IPv6: ADDRCONF(NETDEV_CHANGE): vethfbacd6d: link becomes ready
[   89.309979] br-cf2fb069bbe8: port 3(vethfbacd6d) entered blocking state
[   89.316640] br-cf2fb069bbe8: port 3(vethfbacd6d) entered forwarding state
[   89.601125] eth0: renamed from veth39d888d
[   89.640406] IPv6: ADDRCONF(NETDEV_CHANGE): veth5f20e73: link becomes ready
[   89.647656] br-cf2fb069bbe8: port 4(veth5f20e73) entered blocking state
[   89.654314] br-cf2fb069bbe8: port 4(veth5f20e73) entered forwarding state
[   89.900095] eth0: renamed from veth8d45784
[   89.932145] IPv6: ADDRCONF(NETDEV_CHANGE): vethb983baa: link becomes ready
[   89.947019] br-cf2fb069bbe8: port 5(vethb983baa) entered blocking state
[   89.953663] br-cf2fb069bbe8: port 5(vethb983baa) entered forwarding state
[   90.021196] eth1: renamed from vethb1a72b5
[   90.079181] eth0: renamed from veth37db94c
[   90.102739] IPv6: ADDRCONF(NETDEV_CHANGE): veth395f796: link becomes ready
[   90.111581] supervisor0: port 1(veth395f796) entered blocking state
[   90.117899] supervisor0: port 1(veth395f796) entered forwarding state
[   90.125928] IPv6: ADDRCONF(NETDEV_CHANGE): supervisor0: link becomes ready
[   90.135456] IPv6: ADDRCONF(NETDEV_CHANGE): veth1431b77: link becomes ready
[   90.142806] br-cf2fb069bbe8: port 6(veth1431b77) entered blocking state
[   90.149453] br-cf2fb069bbe8: port 6(veth1431b77) entered forwarding state
[   90.352691] eth0: renamed from veth44e9c28
[   90.392476] IPv6: ADDRCONF(NETDEV_CHANGE): vethf0594d1: link becomes ready
[   90.400214] br-cf2fb069bbe8: port 7(vethf0594d1) entered blocking state
[   90.406855] br-cf2fb069bbe8: port 7(vethf0594d1) entered forwarding state
[   90.462171] eth1: renamed from vethfa3c083
[   90.495427] IPv6: ADDRCONF(NETDEV_CHANGE): veth19e8cca: link becomes ready
[   90.502630] supervisor0: port 2(veth19e8cca) entered blocking state
[   90.508928] supervisor0: port 2(veth19e8cca) entered forwarding state
[   91.185075] systemd-udevd[30]: starting version 237
[   91.185170] systemd-udevd[28]: starting version 237
[   91.315987] systemd-udevd[29]: starting version 237
[   91.416314] systemd-udevd[29]: Specified group 'crypto' unknown
[   91.429089] systemd-udevd[29]: Specified group 'trusty' unknown
[   91.430981] systemd-udevd[31]: Specified group 'crypto' unknown
[   91.433853] systemd-udevd[31]: Specified group 'trusty' unknown
[   91.518150] systemd-udevd[30]: Specified group 'crypto' unknown
[   91.533832] systemd-udevd[30]: Specified group 'trusty' unknown
[   94.131597] systemd-udevd[35]: Process '/sbin/crda' failed with exit code 234.
[  101.074202] systemd-udevd[30]: Specified group 'crypto' unknown
[  101.081689] systemd-udevd[30]: Specified group 'trusty' unknown
[  101.130191] systemd-udevd[29]: Specified group 'crypto' unknown
[  101.132356] systemd-udevd[31]: Specified group 'crypto' unknown
[  101.134060] systemd-udevd[31]: Specified group 'trusty' unknown
[  101.148229] systemd-udevd[29]: Specified group 'trusty' unknown
[  101.310472] v4l2loopback driver version 0.12.5 loaded
[  104.080605] systemd-udevd[30]: Specified group 'crypto' unknown
[  104.088008] systemd-udevd[30]: Specified group 'trusty' unknown
[  104.132223] systemd-udevd[31]: Specified group 'crypto' unknown
[  104.140684] systemd-udevd[31]: Specified group 'trusty' unknown
[  105.036965] systemd-udevd[29]: Specified group 'crypto' unknown
[  105.043005] systemd-udevd[29]: Specified group 'trusty' unknown
[  136.620064] iwlwifi 0000:01:00.0: Detected Intel(R) Wi-Fi 6 AX200 160MHz, REV=0x340
[  136.831550] iwlwifi 0000:01:00.0: base HW address: 14:f6:d8:6b:f7:33
[  136.853711] thermal thermal_zone6: failed to read out thermal zone (-61)
[  136.860473] thermal thermal_zone6: Registering thermal zone thermal_zone6 for type iwlwifi_1
[  136.943585] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  137.251093] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  137.491132] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  137.669770] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  141.077679] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  141.459293] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  141.482396] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[  168.482818] systemd-udevd[89]: Process '/sbin/crda' failed with exit code 234.
[  258.798267] Adding 2095332k swap on /var/lib/swap.img.  Priority:-1 extents:454 across:9828344k SS
[  261.551153] systemd-udevd[86]: Process '/sbin/crda' failed with exit code 234.
[  273.652364] tegradc tegradc.0: blank - powerdown
[  273.699524] extcon-disp-state extcon:disp-state: cable 47 state 0
[  273.699527] Extcon AUX1(HDMI) disable
[  273.724266] tegradc tegradc.0: unblank
[  273.764377] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  273.765444] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  273.766521] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  273.766529] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  273.808321] extcon-disp-state extcon:disp-state: cable 47 state 1
[  273.808324] Extcon AUX1(HDMI) enable
[  273.811175] tegradc tegradc.0: sync windows ret = 250
[  275.985682] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  276.001417] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  276.252612] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  277.459346] nvgpu: 57000000.gpu   nvgpu_set_error_notifier_locked:137  [ERR]  error notifier set to 8 for ch 502
[  277.469549] nvgpu: 57000000.gpu     gk20a_fifo_handle_sched_error:2531 [ERR]  fifo sched ctxsw timeout error: engine=0, tsg=6, ms=3100
[  277.469556] tegradc tegradc.0: blank - powerdown
[  277.469665] ---- mlocks ----

[  277.469674] ---- syncpts ----
[  277.469679] id 1 (disp0_a) min 2 max 2 refs 1 (previous client : )
[  277.469682] id 2 (disp0_b) min 2 max 2 refs 1 (previous client : )
[  277.469684] id 3 (disp0_c) min 2 max 2 refs 1 (previous client : )
[  277.469689] id 7 (gm20b_503) min 54 max 54 refs 1 (previous client : )
[  277.469692] id 8 (gm20b_507) min 27914 max 27914 refs 1 (previous client : )
[  277.469694] id 9 (gm20b_506) min 17876 max 17876 refs 1 (previous client : )
[  277.469697] id 11 (gm20b_505) min 2 max 2 refs 1 (previous client : )
[  277.469700] id 12 (gm20b_504) min 2 max 2 refs 1 (previous client : )
[  277.469702] id 13 (gm20b_502) min 2 max 4 refs 1 (previous client : )
[  277.469711] id 26 (vblank0) min 219 max -4 refs 1 (previous client : )

[  277.469796] ---- channels ----
[  277.469803]
               ---- host general irq ----

[  277.469805] sync_hintmask_ext = 0xc0000000
[  277.469807] sync_hintmask = 0x80000000
[  277.469809] sync_intc0mask = 0x00000001
[  277.469811] sync_intmask = 0x00000011
[  277.469812]
               ---- host syncpt irq mask ----

[  277.469814] syncpt_thresh_int_mask(0) = 0x04000001
[  277.469816] syncpt_thresh_int_mask(1) = 0x00000000
[  277.469818] syncpt_thresh_int_mask(2) = 0x00000000
[  277.469820] syncpt_thresh_int_mask(3) = 0x00000000
[  277.469821] syncpt_thresh_int_mask(4) = 0x00000000
[  277.469823] syncpt_thresh_int_mask(5) = 0x00000000
[  277.469825] syncpt_thresh_int_mask(6) = 0x00000000
[  277.469827] syncpt_thresh_int_mask(7) = 0x00000000
[  277.469828] syncpt_thresh_int_mask(8) = 0x00000000
[  277.469830] syncpt_thresh_int_mask(9) = 0x00000000
[  277.469832] syncpt_thresh_int_mask(10) = 0x00000000
[  277.469834] syncpt_thresh_int_mask(11) = 0x00000000
[  277.469835]
               ---- host syncpt irq status ----

[  277.469837] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[  277.469839] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[  277.469841] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[  277.469842] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[  277.469844] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[  277.469846] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[  277.469847]
               ---- host syncpt thresh ----

[  277.469849] syncpt_int_thresh_thresh_0(0) = 1
[  277.469856] syncpt_int_thresh_thresh_0(13) = 4
[  277.469931] gm20b pbdma 0:
[  277.469931] id: 6 (tsg), next_id: 6 (tsg) chan status: valid
[  277.469942] PBDMA_PUT: 0000001e000081d0 PBDMA_GET: 0000001e00000044 GP_PUT: 00000006 GP_GET: 00000004 FETCH: 00000006 HEADER: 800302ec
               HDR: 800160bb SHADOW0: 00000004 SHADOW1: 0081cc1e

[  277.469946] gm20b eng 0:
[  277.469947] id: 0 (channel), next_id: 6 (tsg), ctx status: load
[  277.469948] busy

[  277.469952] gm20b eng 1:
[  277.469953] id: 6 (tsg), next_id: 6 (tsg), ctx status: valid


[  277.470108] 502-gm20b, pid 11563, refs 4:
[  277.470108] channel status:  in use on_pbdma_and_eng busy
[  277.470113] RAMFC : TOP: 8000001f00004020 PUT: 0000001f00004020 GET: 0000001f00004020 FETCH: 0000001f00004020
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000d01 SEMAPHORE 0000000d fc001000 00000001 01100002

[  277.470119] 503-gm20b, pid 10651, refs 2:
[  277.470119] channel status:  in use idle not busy
[  277.470124] RAMFC : TOP: 8000001f00280288 PUT: 0000001f00280288 GET: 0000001f00280288 FETCH: 0000001f00280288
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000701 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470129] 504-gm20b, pid 10651, refs 2:
[  277.470129] channel status:  in use idle not busy
[  277.470134] RAMFC : TOP: 8000001f00240018 PUT: 0000001f00240018 GET: 0000001f00240018 FETCH: 0000001f00240018
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000c01 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470139] 505-gm20b, pid 10651, refs 2:
[  277.470139] channel status:  in use idle not busy
[  277.470143] RAMFC : TOP: 8000001f00200018 PUT: 0000001f00200018 GET: 0000001f00200018 FETCH: 0000001f00200018
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000b01 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470148] 506-gm20b, pid 10651, refs 2:
[  277.470148] channel status:  in use idle not busy
[  277.470152] RAMFC : TOP: 8000001f001745f0 PUT: 0000001f001745f0 GET: 0000001f001745f0 FETCH: 0000001f001745f0
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000901 SEMAPHORE 00000001 0002fff0 00003683 00000004

[  277.470157] 507-gm20b, pid 10651, refs 2:
[  277.470157] channel status:  in use idle not busy
[  277.470162] RAMFC : TOP: 8000001f00051c88 PUT: 0000001f00051c88 GET: 0000001f00051c88 FETCH: 0000001f00051c88
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000801 SEMAPHORE 00000001 0002ffb0 00000003 00001004

[  277.470167] 508-gm20b, pid 3975, refs 2:
[  277.470167] channel status:  in use idle not busy
[  277.470171] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470176] 509-gm20b, pid 3975, refs 2:
[  277.470176] channel status:  in use idle not busy
[  277.470180] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470185] 510-gm20b, pid 3975, refs 2:
[  277.470185] channel status:  in use idle not busy
[  277.470188] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470193] 511-gm20b, pid 3975, refs 2:
[  277.470193] channel status:  in use idle not busy
[  277.470197] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[  277.470439] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1721 [ERR]  fake mmu fault on engine 0, engine subid 0 (gpc), client 2 (pe 0), addr 0x547b119000, type 4 (unbound inst), access_type 0x00000001,inst_ptr 0x20f88000
[  277.470443] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:129  [ERR]  gr_fecs_os_r : 0
[  277.470446] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:131  [ERR]  gr_fecs_cpuctl_r : 0x40
[  277.470448] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:133  [ERR]  gr_fecs_idlestate_r : 0x1
[  277.470451] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:135  [ERR]  gr_fecs_mailbox0_r : 0x0
[  277.470454] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:137  [ERR]  gr_fecs_mailbox1_r : 0x0
[  277.470456] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:139  [ERR]  gr_fecs_irqstat_r : 0x0
[  277.470459] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:141  [ERR]  gr_fecs_irqmode_r : 0x4
[  277.470462] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:143  [ERR]  gr_fecs_irqmask_r : 0x8704
[  277.470464] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:145  [ERR]  gr_fecs_irqdest_r : 0x0
[  277.470467] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:147  [ERR]  gr_fecs_debug1_r : 0x40
[  277.470469] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:149  [ERR]  gr_fecs_debuginfo_r : 0x0
[  277.470472] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:151  [ERR]  gr_fecs_ctxsw_status_1_r : 0x304
[  277.470475] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(0) : 0x1
[  277.470478] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(1) : 0x0
[  277.470480] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(2) : 0x90009
[  277.470483] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(3) : 0x14000
[  277.470486] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0
[  277.470488] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(5) : 0x0
[  277.470491] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(6) : 0x0
[  277.470493] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(7) : 0x0
[  277.470496] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(8) : 0x0
[  277.470499] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(9) : 0x0
[  277.470501] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(10) : 0x0
[  277.470504] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(11) : 0x0
[  277.470506] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(12) : 0x0
[  277.470509] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(13) : 0x0
[  277.470511] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(14) : 0x0
[  277.470514] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(15) : 0x0
[  277.470516] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:159  [ERR]  gr_fecs_engctl_r : 0x0
[  277.470519] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:161  [ERR]  gr_fecs_curctx_r : 0x0
[  277.470521] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:163  [ERR]  gr_fecs_nxtctx_r : 0x0
[  277.470524] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:169  [ERR]  FECS_FALCON_REG_IMB : 0xbadfbadf
[  277.470526] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:175  [ERR]  FECS_FALCON_REG_DMB : 0xbadfbadf
[  277.470529] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:181  [ERR]  FECS_FALCON_REG_CSW : 0xbadfbadf
[  277.470532] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:187  [ERR]  FECS_FALCON_REG_CTX : 0xbadfbadf
[  277.470534] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:193  [ERR]  FECS_FALCON_REG_EXCI : 0xbadfbadf
[  277.470537] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470540] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470542] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470545] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470547] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470550] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470553] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[  277.470555] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[  277.470558] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1726 [ERR]  gr_status_r : 0x400081
[  277.471416] nvgpu: 57000000.gpu                    fifo_error_isr:2605 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100
[  277.522573] extcon-disp-state extcon:disp-state: cable 47 state 0
[  277.522574] Extcon AUX1(HDMI) disable
[  277.543010] tegradc tegradc.0: unblank
[  277.582375] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  277.582510] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  277.584208] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  277.584216] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  277.625664] extcon-disp-state extcon:disp-state: cable 47 state 1
[  277.625667] Extcon AUX1(HDMI) enable
[  277.630816] tegradc tegradc.0: sync windows ret = 250
[  289.172493] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[  289.179255] veth8d45784: renamed from eth0
[  289.462075] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[  289.473780] device vethb983baa left promiscuous mode
[  289.478850] br-cf2fb069bbe8: port 5(vethb983baa) entered disabled state
[  289.628544] supervisor0: port 1(veth395f796) entered disabled state
[  289.635114] vethb1a72b5: renamed from eth1
[  289.695436] supervisor0: port 1(veth395f796) entered disabled state
[  289.711138] device veth395f796 left promiscuous mode
[  289.716216] supervisor0: port 1(veth395f796) entered disabled state
[  289.947375] br-cf2fb069bbe8: port 5(veth4151ce7) entered blocking state
[  289.954067] br-cf2fb069bbe8: port 5(veth4151ce7) entered disabled state
[  289.981862] device veth4151ce7 entered promiscuous mode
[  290.001918] IPv6: ADDRCONF(NETDEV_UP): veth4151ce7: link is not ready
[  290.026065] br-cf2fb069bbe8: port 5(veth4151ce7) entered blocking state
[  290.032768] br-cf2fb069bbe8: port 5(veth4151ce7) entered forwarding state
[  290.174098] br-cf2fb069bbe8: port 5(veth4151ce7) entered disabled state
[  290.290341] supervisor0: port 1(veth7024076) entered blocking state
[  290.299915] supervisor0: port 1(veth7024076) entered disabled state
[  290.307830] device veth7024076 entered promiscuous mode
[  290.314161] IPv6: ADDRCONF(NETDEV_UP): veth7024076: link is not ready
[  290.320632] supervisor0: port 1(veth7024076) entered blocking state
[  290.326918] supervisor0: port 1(veth7024076) entered forwarding state
[  291.198129] supervisor0: port 1(veth7024076) entered disabled state
[  291.455052] eth0: renamed from veth8842ec1
[  291.474833] IPv6: ADDRCONF(NETDEV_CHANGE): veth4151ce7: link becomes ready
[  291.481824] br-cf2fb069bbe8: port 5(veth4151ce7) entered blocking state
[  291.488455] br-cf2fb069bbe8: port 5(veth4151ce7) entered forwarding state
[  291.523724] eth1: renamed from veth5b7966f
[  291.554948] IPv6: ADDRCONF(NETDEV_CHANGE): veth7024076: link becomes ready
[  291.562005] supervisor0: port 1(veth7024076) entered blocking state
[  291.568293] supervisor0: port 1(veth7024076) entered forwarding state
[  292.104218] systemd-udevd[29]: starting version 237
[  292.148244] systemd-udevd[30]: Specified group 'crypto' unknown
[  292.157779] systemd-udevd[30]: Specified group 'trusty' unknown
[  294.425102] systemd-udevd[276]: Process '/sbin/crda' failed with exit code 234.
[  294.860568] tegradc tegradc.0: blank - powerdown
[  294.910588] extcon-disp-state extcon:disp-state: cable 47 state 0
[  294.910591] Extcon AUX1(HDMI) disable
[  294.931247] tegradc tegradc.0: unblank
[  294.994500] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  294.994616] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  294.996313] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  294.996320] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  295.038345] extcon-disp-state extcon:disp-state: cable 47 state 1
[  295.038347] Extcon AUX1(HDMI) enable
[  295.038707] tegradc tegradc.0: sync windows ret = 250
[  295.488873] tegradc tegradc.0: blank - powerdown
[  295.547498] extcon-disp-state extcon:disp-state: cable 47 state 0
[  295.547500] Extcon AUX1(HDMI) disable
[  295.586185] tegradc tegradc.0: unblank
[  295.601256] tegradc tegradc.0: nominal-pclk:71000000 parent:71000000 div:1.0 pclk:71000000 70290000~77390000
[  295.601438] tegradc tegradc.0: hdmi: tmds rate:71000K prod-setting:prod_c_hdmi_54m_75m
[  295.604312] tegradc tegradc.0: hdmi: get RGB quant from EDID.
[  295.604324] tegradc tegradc.0: hdmi: get YCC quant from EDID.
[  295.642099] extcon-disp-state extcon:disp-state: cable 47 state 1
[  295.642102] Extcon AUX1(HDMI) enable
[  295.697041] tegradc tegradc.0: unblank
[  295.700990] tegradc tegradc.1: blank - powerdown
[  299.849040] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[  299.857758] Bluetooth: BNEP socket layer initialized
[  623.736096] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[  939.659178] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready

ok, you should run several oceanFFTs over and make it sure by “ps -e” then unplugged the hdmi monitor ,then may be several hour or days ,you check the displays by connecting the hdmi monitor leadding to black screen sometime.

@drcnyc,

I think your case is special. We cannot guarantee the case of docker or other OS.

For example, I cannot just give lots of docker setup steps to gpu team and ask them to debug.
When we are trying to debug, we only have pure jetpack and devkit. Thus, I will try with @wangzhaofeng usecase first.

Anyway, thanks for sharing the log and usecase.

@WayneWWW Well that’s disappointing. I think you will find that many users who are developing IoT devices use docker-based setups - especially when deploying large fleets. Unfortunately this will likely put our plans with the Jetson Nano on hold.

I think if we resolve the case from @wangzhaofeng, this would resolve your case too. So please wait for our reply.

Thanks.

Hi @wangzhaofeng,

Do you run 15 oceanFFT programs simultaneously?

@WayneWWW, in case its helpful, I found errors in my Xorg log which correspond to the errors in the dmesg log. I wasn’t even using the device at the time. Below is the outputs from both logs with timestamps. The GPU was not being used at the time.

Xorg:

[   102.205] (**) HID 222a:0001: (accel) keeping acceleration scheme 1
[   102.205] (**) HID 222a:0001: (accel) acceleration profile 0
[   102.205] (**) HID 222a:0001: (accel) acceleration factor: 2.000
[   102.205] (**) HID 222a:0001: (accel) acceleration threshold: 4
[   102.206] (II) config/udev: Adding input device HID 222a:0001 (/dev/input/mouse0)
[   102.206] (II) No input driver specified, ignoring this device.
[   102.206] (II) This device may have been added with another device file.
[   102.207] (II) config/udev: Adding input device gpio-keys (/dev/input/event1)
[   102.207] (**) gpio-keys: Applying InputClass "evdev keyboard catchall"
[   102.207] (II) Using input driver 'evdev' for 'gpio-keys'
[   102.207] (**) gpio-keys: always reports core events
[   102.207] (**) evdev: gpio-keys: Device: "/dev/input/event1"
[   102.207] (--) evdev: gpio-keys: Vendor 0x1 Product 0x1
[   102.207] (--) evdev: gpio-keys: Found keys
[   102.207] (II) evdev: gpio-keys: Configuring as keyboard
[   102.207] (**) Option "config_info" "udev:/sys/devices/gpio-keys/input/input1/event1"
[   102.207] (II) XINPUT: Adding extended input device "gpio-keys" (type: KEYBOARD, id 7)
[   102.207] (**) Option "xkb_rules" "evdev"
[   102.208] (**) Option "xkb_model" "pc105"
[   102.208] (**) Option "xkb_layout" "us"
[   102.404] (--) NVIDIA(GPU-0): HCD PJ402D-2 (DFP-0): connected
[   102.404] (--) NVIDIA(GPU-0): HCD PJ402D-2 (DFP-0): External TMDS
[  5274.596] (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
[  5274.596] (EE) NVIDIA(0):     recover...
[  5275.552] (II) NVIDIA(0): Error recovery was successful.
[  5631.459] (WW) NVIDIA(0): WAIT (2-S, 17, 0xc19c, 0x0000a3f8, 0x0000b640)
[  5631.685] (EE) NVIDIA(0): The NVIDIA X driver has encountered an error; attempting to
[  5631.685] (EE) NVIDIA(0):     recover...
[  5632.653] (II) NVIDIA(0): Error recovery was successful.

Dmesg:

[ 3278.502293] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 3594.476767] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 3910.504475] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 4226.434943] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 4542.455919] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 4858.461871] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 5174.451461] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 5274.732318] nvgpu: 57000000.gpu   nvgpu_set_error_notifier_locked:137  [ERR]  error notifier set to 8 for ch 507
[ 5274.742494] nvgpu: 57000000.gpu     gk20a_fifo_handle_sched_error:2531 [ERR]  fifo sched ctxsw timeout error: engine=0, tsg=4, ms=3100
[ 5274.754691] ---- mlocks ----

[ 5274.759087] ---- syncpts ----
[ 5274.762072] id 1 (disp0_a) min 4 max 4 refs 1 (previous client : )
[ 5274.768262] id 2 (disp0_b) min 3 max 3 refs 1 (previous client : )
[ 5274.774453] id 3 (disp0_c) min 3 max 3 refs 1 (previous client : )
[ 5274.780647] id 7 (gm20b_507) min 46254 max 46260 refs 1 (previous client : )
[ 5274.787702] id 8 (gm20b_506) min 4 max 4 refs 1 (previous client : )
[ 5274.794074] id 26 (vblank0) min 309888 max -6 refs 1 (previous client : )

[ 5274.802455] ---- channels ----
[ 5274.805505]
               ---- host general irq ----

[ 5274.812220] sync_hintmask_ext = 0xc0000000
[ 5274.816325] sync_hintmask = 0x80000000
[ 5274.820081] sync_intc0mask = 0x00000001
[ 5274.823936] sync_intmask = 0x00000011
[ 5274.827607]
               ---- host syncpt irq mask ----

[ 5274.834663] syncpt_thresh_int_mask(0) = 0x00004001
[ 5274.839459] syncpt_thresh_int_mask(1) = 0x00000000
[ 5274.844256] syncpt_thresh_int_mask(2) = 0x00000000
[ 5274.849056] syncpt_thresh_int_mask(3) = 0x00000000
[ 5274.853852] syncpt_thresh_int_mask(4) = 0x00000000
[ 5274.858648] syncpt_thresh_int_mask(5) = 0x00000000
[ 5274.863442] syncpt_thresh_int_mask(6) = 0x00000000
[ 5274.868238] syncpt_thresh_int_mask(7) = 0x00000000
[ 5274.873034] syncpt_thresh_int_mask(8) = 0x00000000
[ 5274.877829] syncpt_thresh_int_mask(9) = 0x00000000
[ 5274.882625] syncpt_thresh_int_mask(10) = 0x00000000
[ 5274.887505] syncpt_thresh_int_mask(11) = 0x00000000
[ 5274.892386]
               ---- host syncpt irq status ----

[ 5274.899610] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[ 5274.905014] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[ 5274.910416] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[ 5274.915820] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[ 5274.921221] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[ 5274.926625] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[ 5274.932030]
               ---- host syncpt thresh ----

[ 5274.938911] syncpt_int_thresh_thresh_0(0) = 1
[ 5274.943278] syncpt_int_thresh_thresh_0(7) = 46256
[ 5274.948065] gm20b pbdma 0:
[ 5274.950694] id: 4 (tsg), next_id: 4 (tsg) chan status: valid
[ 5274.956357] PBDMA_PUT: 0000001e0000ec20 PBDMA_GET: 0000001e0000ebac GP_PUT: 00000708 GP_GET: 00000706 FETCH: 00000708 HEADER: 60030100
               HDR: 60116040 SHADOW0: 0000eb8c SHADOW1: 0000941e

[ 5274.974176] gm20b eng 0:
[ 5274.976614] id: 0 (channel), next_id: 4 (tsg), ctx status: load
[ 5274.982622] busy

[ 5274.984551] gm20b eng 1:
[ 5274.987007] id: 0 (channel), next_id: 0 (channel), ctx status: invalid


[ 5274.996726] 506-gm20b, pid 10688, refs 2:
[ 5275.000660] channel status:  in use idle not busy
[ 5275.005368] RAMFC : TOP: 8000001f0000c030 PUT: 0000001f0000c030 GET: 0000001f0000c030 FETCH: 0000001f0000c030
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000801 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.028817] 507-gm20b, pid 10688, refs 8:
[ 5275.032752] channel status:  in use on_pbdma_and_eng busy
[ 5275.038152] RAMFC : TOP: 8000001f00004ae0 PUT: 0000001f00004ae0 GET: 0000001f00004ae0 FETCH: 0000001f00004ae0
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000701 SEMAPHORE 0000000d fc001000 01583ae3 01100002

[ 5275.061600] 508-gm20b, pid 3975, refs 2:
[ 5275.065444] channel status:  in use idle not busy
[ 5275.070153] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.093611] 509-gm20b, pid 3975, refs 2:
[ 5275.097452] channel status:  in use idle not busy
[ 5275.102155] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.125603] 510-gm20b, pid 3975, refs 2:
[ 5275.129443] channel status:  in use idle not busy
[ 5275.134150] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.157596] 511-gm20b, pid 3975, refs 2:
[ 5275.161437] channel status:  in use idle not busy
[ 5275.166141] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5275.190261] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1721 [ERR]  fake mmu fault on engine 0, engine subid 0 (gpc), client 0 (l1 0), addr 0x547b019000, type 4 (unbound inst), access_type 0x00000001,inst_ptr 0x20f88000
[ 5275.210676] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:129  [ERR]  gr_fecs_os_r : 0
[ 5275.219291] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:131  [ERR]  gr_fecs_cpuctl_r : 0x40
[ 5275.228509] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:133  [ERR]  gr_fecs_idlestate_r : 0x1
[ 5275.237899] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:135  [ERR]  gr_fecs_mailbox0_r : 0x0
[ 5275.247202] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:137  [ERR]  gr_fecs_mailbox1_r : 0x0
[ 5275.256503] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:139  [ERR]  gr_fecs_irqstat_r : 0x0
[ 5275.265720] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:141  [ERR]  gr_fecs_irqmode_r : 0x4
[ 5275.274935] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:143  [ERR]  gr_fecs_irqmask_r : 0x8704
[ 5275.284412] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:145  [ERR]  gr_fecs_irqdest_r : 0x0
[ 5275.293632] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:147  [ERR]  gr_fecs_debug1_r : 0x40
[ 5275.302853] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:149  [ERR]  gr_fecs_debuginfo_r : 0x0
[ 5275.312247] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:151  [ERR]  gr_fecs_ctxsw_status_1_r : 0x304
[ 5275.322248] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(0) : 0x1
[ 5275.332244] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(1) : 0x0
[ 5275.342241] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(2) : 0x90009
[ 5275.352584] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(3) : 0x14000
[ 5275.362928] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0
[ 5275.373357] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(5) : 0x0
[ 5275.383353] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(6) : 0x0
[ 5275.393348] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(7) : 0x0
[ 5275.403347] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(8) : 0x0
[ 5275.413345] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(9) : 0x0
[ 5275.423346] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(10) : 0x0
[ 5275.433436] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(11) : 0x0
[ 5275.443526] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(12) : 0x0
[ 5275.453609] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(13) : 0x0
[ 5275.463694] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(14) : 0x0
[ 5275.473790] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(15) : 0x0
[ 5275.483881] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:159  [ERR]  gr_fecs_engctl_r : 0x0
[ 5275.493011] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:161  [ERR]  gr_fecs_curctx_r : 0x0
[ 5275.502144] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:163  [ERR]  gr_fecs_nxtctx_r : 0x0
[ 5275.511275] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:169  [ERR]  FECS_FALCON_REG_IMB : 0xbadfbadf
[ 5275.521271] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:175  [ERR]  FECS_FALCON_REG_DMB : 0xbadfbadf
[ 5275.531268] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:181  [ERR]  FECS_FALCON_REG_CSW : 0xbadfbadf
[ 5275.541272] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:187  [ERR]  FECS_FALCON_REG_CTX : 0xbadfbadf
[ 5275.551271] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:193  [ERR]  FECS_FALCON_REG_EXCI : 0xbadfbadf
[ 5275.561356] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.571269] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.581178] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.591088] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.600998] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.610907] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.620817] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5275.630730] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5275.640643] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1726 [ERR]  gr_status_r : 0x81
[ 5275.650394] nvgpu: 57000000.gpu                    fifo_error_isr:2605 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100
[ 5490.414530] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 5631.818801] nvgpu: 57000000.gpu   nvgpu_set_error_notifier_locked:137  [ERR]  error notifier set to 8 for ch 507
[ 5631.828989] nvgpu: 57000000.gpu     gk20a_fifo_handle_sched_error:2531 [ERR]  fifo sched ctxsw timeout error: engine=0, tsg=4, ms=3100
[ 5631.841226] ---- mlocks ----

[ 5631.845619] ---- syncpts ----
[ 5631.848606] id 1 (disp0_a) min 4 max 4 refs 1 (previous client : )
[ 5631.854798] id 2 (disp0_b) min 3 max 3 refs 1 (previous client : )
[ 5631.860988] id 3 (disp0_c) min 3 max 3 refs 1 (previous client : )
[ 5631.867175] id 7 (gm20b_507) min 245148 max 245156 refs 1 (previous client : gm20b_507)
[ 5631.875184] id 8 (gm20b_506) min 4 max 4 refs 1 (previous client : )
[ 5631.881553] id 26 (vblank0) min 331281 max -6 refs 1 (previous client : )

[ 5631.889913] ---- channels ----
[ 5631.892980]
               ---- host general irq ----

[ 5631.899689] sync_hintmask_ext = 0xc0000000
[ 5631.903793] sync_hintmask = 0x80000000
[ 5631.907550] sync_intc0mask = 0x00000001
[ 5631.911392] sync_intmask = 0x00000011
[ 5631.915061]
               ---- host syncpt irq mask ----

[ 5631.922117] syncpt_thresh_int_mask(0) = 0x00004001
[ 5631.926919] syncpt_thresh_int_mask(1) = 0x00000000
[ 5631.931715] syncpt_thresh_int_mask(2) = 0x00000000
[ 5631.936513] syncpt_thresh_int_mask(3) = 0x00000000
[ 5631.941309] syncpt_thresh_int_mask(4) = 0x00000000
[ 5631.946106] syncpt_thresh_int_mask(5) = 0x00000000
[ 5631.950903] syncpt_thresh_int_mask(6) = 0x00000000
[ 5631.955699] syncpt_thresh_int_mask(7) = 0x00000000
[ 5631.960494] syncpt_thresh_int_mask(8) = 0x00000000
[ 5631.965290] syncpt_thresh_int_mask(9) = 0x00000000
[ 5631.970085] syncpt_thresh_int_mask(10) = 0x00000000
[ 5631.974968] syncpt_thresh_int_mask(11) = 0x00000000
[ 5631.979852]
               ---- host syncpt irq status ----

[ 5631.987084] syncpt_thresh_cpu0_int_status(0) = 0x00000000
[ 5631.992487] syncpt_thresh_cpu0_int_status(1) = 0x00000000
[ 5631.997891] syncpt_thresh_cpu0_int_status(2) = 0x00000000
[ 5632.003294] syncpt_thresh_cpu0_int_status(3) = 0x00000000
[ 5632.008709] syncpt_thresh_cpu0_int_status(4) = 0x00000000
[ 5632.014113] syncpt_thresh_cpu0_int_status(5) = 0x00000000
[ 5632.019519]
               ---- host syncpt thresh ----

[ 5632.026400] syncpt_int_thresh_thresh_0(0) = 1
[ 5632.030766] syncpt_int_thresh_thresh_0(7) = 245150
[ 5632.035636] gm20b pbdma 0:
[ 5632.038248] id: 4 (tsg), next_id: 4 (tsg) chan status: valid
[ 5632.043924] PBDMA_PUT: 0000001e0000b5c8 PBDMA_GET: 0000001e0000a410 GP_PUT: 0000055f GP_GET: 0000055d FETCH: 0000055f HEADER: 60030100
               HDR: 60116040 SHADOW0: 0000a3f8 SHADOW1: 0011d01e

[ 5632.061736] gm20b eng 0:
[ 5632.064194] id: 0 (channel), next_id: 4 (tsg), ctx status: load
[ 5632.070196] busy

[ 5632.072141] gm20b eng 1:
[ 5632.074581] id: 0 (channel), next_id: 0 (channel), ctx status: invalid


[ 5632.084385] 506-gm20b, pid 10688, refs 2:
[ 5632.088322] channel status:  in use idle not busy
[ 5632.093032] RAMFC : TOP: 8000001f0000c030 PUT: 0000001f0000c030 GET: 0000001f0000c030 FETCH: 0000001f0000c030
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000801 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.116490] 507-gm20b, pid 10688, refs 10:
[ 5632.120512] channel status:  in use on_pbdma_and_eng busy
[ 5632.125919] RAMFC : TOP: 8000001f00014e80 PUT: 0000001f00014e80 GET: 0000001f00014e80 FETCH: 0000001f00014e80
               HEADER: 60400000 COUNT: 80000000
               SYNCPOINT 00000000 00000701 SEMAPHORE 0000000d fc000000 01cc28fe 01100002

[ 5632.149381] 508-gm20b, pid 3975, refs 2:
[ 5632.153226] channel status:  in use idle not busy
[ 5632.157932] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.181387] 509-gm20b, pid 3975, refs 2:
[ 5632.185251] channel status:  in use idle not busy
[ 5632.189959] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.213416] 510-gm20b, pid 3975, refs 2:
[ 5632.217263] channel status:  in use idle not busy
[ 5632.221971] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.245416] 511-gm20b, pid 3975, refs 2:
[ 5632.249258] channel status:  in use idle not busy
[ 5632.254001] RAMFC : TOP: 0000000000000000 PUT: 0000000000000000 GET: 0000000000000000 FETCH: 0000000000000000
               HEADER: 60400000 COUNT: 00000000
               SYNCPOINT 00000000 00000000 SEMAPHORE 00000000 00000000 00000000 00000000

[ 5632.277760] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1721 [ERR]  fake mmu fault on engine 0, engine subid 0 (gpc), client 8 (pe 2), addr 0x547b109000, type 4 (unbound inst), access_type 0x00000001,inst_ptr 0x20f88000
[ 5632.298245] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:129  [ERR]  gr_fecs_os_r : 0
[ 5632.306874] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:131  [ERR]  gr_fecs_cpuctl_r : 0x40
[ 5632.316139] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:133  [ERR]  gr_fecs_idlestate_r : 0x1
[ 5632.325543] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:135  [ERR]  gr_fecs_mailbox0_r : 0x0
[ 5632.334882] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:137  [ERR]  gr_fecs_mailbox1_r : 0x0
[ 5632.344199] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:139  [ERR]  gr_fecs_irqstat_r : 0x0
[ 5632.353448] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:141  [ERR]  gr_fecs_irqmode_r : 0x4
[ 5632.362675] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:143  [ERR]  gr_fecs_irqmask_r : 0x8704
[ 5632.372215] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:145  [ERR]  gr_fecs_irqdest_r : 0x0
[ 5632.381475] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:147  [ERR]  gr_fecs_debug1_r : 0x40
[ 5632.390723] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:149  [ERR]  gr_fecs_debuginfo_r : 0x0
[ 5632.400130] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:151  [ERR]  gr_fecs_ctxsw_status_1_r : 0x304
[ 5632.410186] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(0) : 0x1
[ 5632.420225] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(1) : 0x0
[ 5632.430261] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(2) : 0x90009
[ 5632.440621] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(3) : 0x14000
[ 5632.451009] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(4) : 0x1ffda0
[ 5632.461457] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(5) : 0x0
[ 5632.471508] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(6) : 0x0
[ 5632.481534] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(7) : 0x0
[ 5632.491605] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(8) : 0x0
[ 5632.501630] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(9) : 0x0
[ 5632.511680] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(10) : 0x0
[ 5632.521793] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(11) : 0x0
[ 5632.531931] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(12) : 0x0
[ 5632.542061] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(13) : 0x0
[ 5632.552194] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(14) : 0x0
[ 5632.562318] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:155  [ERR]  gr_fecs_ctxsw_mailbox_r(15) : 0x0
[ 5632.572449] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:159  [ERR]  gr_fecs_engctl_r : 0x0
[ 5632.581606] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:161  [ERR]  gr_fecs_curctx_r : 0x0
[ 5632.590780] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:163  [ERR]  gr_fecs_nxtctx_r : 0x0
[ 5632.599937] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:169  [ERR]  FECS_FALCON_REG_IMB : 0xbadfbadf
[ 5632.609983] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:175  [ERR]  FECS_FALCON_REG_DMB : 0xbadfbadf
[ 5632.620009] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:181  [ERR]  FECS_FALCON_REG_CSW : 0xbadfbadf
[ 5632.630056] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:187  [ERR]  FECS_FALCON_REG_CTX : 0xbadfbadf
[ 5632.640105] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:193  [ERR]  FECS_FALCON_REG_EXCI : 0xbadfbadf
[ 5632.650270] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.660220] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.670192] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.680155] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.690174] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.700117] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.710052] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:200  [ERR]  FECS_FALCON_REG_PC : 0xbadfbadf
[ 5632.720049] nvgpu: 57000000.gpu      gk20a_fecs_dump_falcon_stats:206  [ERR]  FECS_FALCON_REG_SP : 0xbadfbadf
[ 5632.729981] nvgpu: 57000000.gpu gk20a_fifo_handle_mmu_fault_locked:1726 [ERR]  gr_status_r : 0x81
[ 5632.739919] nvgpu: 57000000.gpu                    fifo_error_isr:2605 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100
[ 5806.419639] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 6122.382657] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready
[ 6438.400266] IPv6: ADDRCONF(NETDEV_UP): wlan0: link is not ready

yes ,

Hi @drcnyc,

Does your system have high gpu usage right after boot up?
Do you still have tegrastats tool or anything else that can check the gpu clock/loading there?

@wangzhaofeng

Hi,

I have run 18 hours for this but still cannot reproduce issue.
I cannot occupy 1 jetson nano for couple of days because other colleagues also need it too.

Then I think we should change to usecase oriented.
Why do you want to run 15 oceanFFT? Is it a stress test? What will be your finalized gpu application?

yeah i run a stress test

What is your criteria to define this test is passed ?

Hi @drcnyc and @wangzhaofeng

Also want to ask you guys to move to rel-32.4.3 because 32.4.2 is just a developer preview and it is always good to use the latest release when debug.

when i make the gpu test ,i unplugged the hdmi monitor ,then after several days or hours i connect the hdmi monitor ,it can display normal not black with nothing

I just wanted to point out that this is a USB mouse error, not a GPU error:

@linuxdev I should have clarified, I was referring to the lines below that, starting at [5274.596] which correspond to the GPU error times in the dmesg logs.

@WayneWWW, there is no GPU usage at all on startup. I even removed anything remotely GPU-related from startup scripts, such as loading nvargus-daemon, etc. Tegrastats shows zero GPU usage when the errors happen.

@WayneWWW it appears other users are having this issue as well, I found this other thread:

Which refers to the emmc module, but the error is almost identical - and I’m wondering if it could be related? It happens sporadically for me, with or without GPU usage - could be every few minutes and then not again for a few hours. Most of the time it does not crash the system (i.e. as you can see from the X logs above, it is able to recover), but occasionally it does.