Unable to get SD Card to init on custom carrier board.

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

I have a custom carrier board with an SD Card. As far as we can tell, it is the exact same schematic as the dev kit and OEM guide. Even probed it all to make sure the traces go to the correct pins.

dmesg section on custom board:

[    6.323470] sdhci-tegra 3400000.sdhci: Got CD GPIO
[    6.328340] sdhci-tegra 3400000.sdhci: Got WP GPIO
[    6.333194] sdhci-tegra 3400000.sdhci: Parent select= pll_p rate=408000000
[    6.340098] sdhci-tegra 3400000.sdhci: wakeup init done, cd_irq: 256
[    6.346753] sdhci-tegra 3400000.sdhci: Client registration for eMC Successful
[    6.396219] mmc2: SDHCI controller on 3400000.sdhci [3400000.sdhci] using ADMA 64-bit with 64 bit addr
[    6.474626] EXT4-fs (mmcblk0p1): 8 orphan inodes deleted
[    6.479957] EXT4-fs (mmcblk0p1): recovery complete
[    6.493949] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)
[    6.502056] VFS: Mounted root (ext4 filesystem) on device 179:1.
[    6.509782] mmc1 tuning done saved tap delay=19
[    6.510069] devtmpfs: mounted
[    6.510356] Freeing unused kernel memory: 1168K (ffffffc001137000 - ffffffc00125b000)
[    6.510380] Freeing alternatives memory: 100K (ffffffc00125b000 - ffffffc001274000)
[    6.517546] btb inv war enabled
[    6.535915] mmc1: hw tuning done ...
[    6.539493] mmc1: tuning_window[0]: 0xffffffff
[    6.543937] mmc1: tuning_window[1]: 0xfffff8ff
[    6.548391] mmc1: tuning_window[2]: 0xffc7ffff
[    6.548401] mmc1: tuning_window[3]: 0x3fffffff
[    6.548406] mmc1: tuning_window[4]: 0x0
[    6.548411] mmc1: tuning_window[5]: 0x0
[    6.548415] mmc1: tuning_window[6]: 0x0
[    6.548420] mmc1: tuning_window[7]: 0x0
[    6.548422] sdhci: Tap value: 19 | Trim value: 5
[    6.548425] sdhci: SDMMC_VENDOR_INTR_STATUS[0x108]: 0x40000
[    6.588380] mmc1: new ultra high speed SDR104 SDIO card at address 0001

Here it is on the devkit:

6.375873] sdhci-tegra 3400000.sdhci: Got CD GPIO
[    6.380743] sdhci-tegra 3400000.sdhci: Got WP GPIO
[    6.385600] sdhci-tegra 3400000.sdhci: Parent select= pll_p rate=408000000
[    6.392503] sdhci-tegra 3400000.sdhci: wakeup init done, cd_irq: 256
[    6.399170] sdhci-tegra 3400000.sdhci: Client registration for eMC Successful
[    6.448657] mmc2: SDHCI controller on 3400000.sdhci [3400000.sdhci] using ADMA 64-bit with 64 bit addr
[    6.534522] EXT4-fs (mmcblk0p1): 13 orphan inodes deleted
[    6.539942] EXT4-fs (mmcblk0p1): recovery complete
[    6.554891] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)
[    6.562999] VFS: Mounted root (ext4 filesystem) on device 179:1.
[    6.571028] devtmpfs: mounted
[    6.571075] mmc1 tuning done saved tap delay=19
[    6.571077] mmc1: hw tuning done ...
[    6.571084] mmc1: tuning_window[0]: 0xffffffff
[    6.571088] mmc1: tuning_window[1]: 0xfffff8ff
[    6.571093] mmc1: tuning_window[2]: 0xffc7ffff
[    6.571097] mmc1: tuning_window[3]: 0x3fffffff
[    6.571102] mmc1: tuning_window[4]: 0x0
[    6.571107] mmc1: tuning_window[5]: 0x0
[    6.571131] mmc1: tuning_window[6]: 0x0
[    6.571135] mmc1: tuning_window[7]: 0x0
[    6.571138] sdhci: Tap value: 19 | Trim value: 5
[    6.571140] sdhci: SDMMC_VENDOR_INTR_STATUS[0x108]: 0x40000
[    6.576041] mmc1: new ultra high speed SDR104 SDIO card at address 0001
[    6.632300] Freeing unused kernel memory: 1168K (ffffffc001137000 - ffffffc00125b000)
[    6.640197] Freeing alternatives memory: 100K (ffffffc00125b000 - ffffffc001274000)
[    6.657429] btb inv war enabled
[    6.715002] random: systemd: uninitialized urandom read (16 bytes read, 115 bits of entropy available)
[    6.725506] random: systemd: uninitialized urandom read (16 bytes read, 115 bits of entropy available)
[    6.734873] random: systemd: uninitialized urandom read (16 bytes read, 115 bits of entropy available)
[    6.745117] systemd[1]: systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN)
[    6.763431] systemd[1]: Detected architecture arm64.
[    6.772976] systemd[1]: Set hostname to <tegra-ubuntu>.
[    6.789937] random: systemd: uninitialized urandom read (16 bytes read, 117 bits of entropy available)
[    6.793776] random: systemd-cryptse: uninitialized urandom read (16 bytes read, 118 bits of entropy available)
[    6.794381] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 118 bits of entropy available)
[    6.794444] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 118 bits of entropy available)
[    6.794459] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 118 bits of entropy available)
[    6.794573] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 118 bits of entropy available)
[    6.795701] random: systemd-sysv-ge: uninitialized urandom read (16 bytes read, 118 bits of entropy available)
[    6.896565] random: nonblocking pool is initialized
[    6.907054] mmc2 tuning done saved tap delay=78
[    6.911597] mmc2: hw tuning done ...
[    6.915178] mmc2: tuning_window[0]: 0xfffc0fff
[    6.919625] mmc2: tuning_window[1]: 0xf03fffff
[    6.924070] mmc2: tuning_window[2]: 0xffffffff
[    6.925601] systemd[1]: Listening on Journal Audit Socket.
[    6.934009] mmc2: tuning_window[3]: 0x7fffff81
[    6.934019] mmc2: tuning_window[4]: 0x0
[    6.934024] mmc2: tuning_window[5]: 0x0
[    6.934028] mmc2: tuning_window[6]: 0x0
[    6.934033] mmc2: tuning_window[7]: 0x0
[    6.934036] sdhci: Tap value: 78 | Trim value: 5
[    6.934039] sdhci: SDMMC_VENDOR_INTR_STATUS[0x108]: 0x40000
[    6.934130] mmc2: new ultra high speed SDR104 SDXC card at address aaaa
[    6.934289] mmcblk mmc2:aaaa: Card claimed for testing.
[    6.934593] mmcblk1: mmc2:aaaa SC64G 59.5 GiB

You can see that on the devkit, mmcblk1(mmc2) appears.

Tests I’ve run:

  1. probing all electrical to make sure connections are correct.
  2. Probed SDCARD_CD to make sure that it pulls low when the card is inserted.
  3. Place Jetson module in devkit to make sure the sdcard is detected.
  4. Place Jetson module in another carrier board to make sure it is detected.
  5. Increase log level of MMC driver, nothing extra shows up.
  6. Probe clock and data signals, all appear to initialize after boot and sending clock signals.
  7. Verify on the devkit that dmesg receives card insert/eject logs.
2

You probably want to mention the L4T release (“head -n 1 /etc/nv_tegra_release”) along with whether it was all “ok” from “sha1sum -c /etc/nv_tegra_release”.

For the log on the custom board, was this booted with the SD card? Without? Did logging stop at the last entry, or was there more (if more it might be important to know what happened next even if it isn’t part of the SD card):

[    6.588380] mmc1: new ultra high speed SDR104 SDIO card at address 0001

If you boot without the SD card and monitor “dmesg --follow”, what shows up specifically from inserting the SD card? I guess that is an extension to the first question of whether it stopped booting at that point.

About the SD card itself…how is it formatted? On a host PC (or better yet, on a Jetson dev kit carrier board) what do the following show (monitor “dmesg --follow” while doing this and see if any error shows):

sudo -s
# "whatever" would be mmcblk1 for the Jetson, but PC host would differ...try both host and Jetson:
gdisk -l /dev/whatever_it_is
# Just on Jetson:
lsblk -p
lsblk -p -m
blkid
blkid -i /dev/mmcblk1
exit

If you “sudo -s”, then cd to “/sys/kernel/debug/mmc2/” you should see files “clock”, “ios”, and “speed”. If no SD card is present no subdirectory will exist. Once an SD card is detected you should see a subdirectory. Mine is “mmc2:59b4”. I don’t have the list of what the values in the files should be, but you might wnat to show the log of this:

egrep '*' `find /sys/kernel/debug/mmc2 -type f`

If you have custom device tree changes you might want to mention what they are. Because device tree install instructions have changed a lot over recent releases you’ll want to verify they actually made it into the tree. Extract a copy of the running system’s device tree for comparison:

dtc -I fs -O dts -o extracted.dts /proc/device-tree
3

Thanks for your help.
Here is the log from the custom carrier board. It was booted with an SD Card inserted.
The SDCARD formatting unknown, but I’ll check next time I plug in the dev kit.

[    0.000000] Booting Linux on physical CPU 0x100
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.38DNNCAM-HW4.4 (d@Blackieubuntu) (gcc version 4.8.5 (GCC) ) #8 SMP PREEMPT Fri Jun 29 15:52:20 MDT 2018
[    0.000000] Boot CPU: AArch64 Processor [411fd073]
[    0.000000] earlycon: Early serial console at MMIO32 0x3100000 (options '')
[    0.000000] bootconsole [uart0] enabled
[    0.000000] Reserved memory: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fc000000
[    0.000000] On node 0 totalpages: 2052096
[    0.000000]   DMA zone: 8192 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 519680 pages, LIFO batch:31
[    0.000000]   Normal zone: 23968 pages used for memmap
[    0.000000]   Normal zone: 1532416 pages, LIFO batch:31
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] PERCPU: Embedded 17 pages/cpu @ffffffc1f665b000 s31360 r8192 d30080 u69632
[    0.000000] pcpu-alloc: s31360 r8192 d30080 u69632 alloc=17*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2019936
[    0.000000] Kernel command line: root=/dev/mmcblk0p1 rw rootwait console=ttyS0,115200n8 console=tty0 OS=l4t fbcon=map:0 net.ifnames=0 memtype=0 video=tegrafb no_console_suspend=1 earlycon=uart8250,mmio32,0x03100000 nvdumper_reserved=0x2772e0000 gpt tegraid=18.1.2.0.0 tegra_keep_boot_clocks maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.2.1 androidboot.serialno=0421318044549 bl_prof_dataptr=0x10000@0x277040000 sdhci_tegra.en_boot_part_access=1 root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Memory: 7976500K/8208384K available (11288K kernel code, 1825K rwdata, 5824K rodata, 1168K init, 692K bss, 166348K reserved, 65536K cma-reserved)
[    0.000000] Virtual kernel memory layout:
                   vmalloc : 0xffffff8000000000 - 0xffffffbdbfff0000   (   246 GB)
                   vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB maximum)
                             0xffffffbdc2000000 - 0xffffffbdc9da0000   (   125 MB actual)
                   fixed   : 0xffffffbffa7fd000 - 0xffffffbffac00000   (  4108 KB)
                   PCI I/O : 0xffffffbffae00000 - 0xffffffbffbe00000   (    16 MB)
                   modules : 0xffffffbffc000000 - 0xffffffc000000000   (    64 MB)
                   memory  : 0xffffffc000000000 - 0xffffffc1f6800000   (  8040 MB)
                     .init : 0xffffffc001137000 - 0xffffffc00125b000   (  1168 KB)
                     .text : 0xffffffc000080000 - 0xffffffc001137000   ( 17116 KB)
                     .data : 0xffffffc00127d000 - 0xffffffc001445440   (  1826 KB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=6.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=6
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] Architected cp15 timer(s) running at 31.25MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xe6a171046, max_idle_ns: 881590405314 ns
[    0.000002] sched_clock: 56 bits at 31MHz, resolution 32ns, wraps every 4398046511088ns
[    0.009451] Console: colour dummy device 80x25
[    0.014111] console [tty0] enabled
[    0.017668] bootconsole [uart0] disabled
[    0.021770] kmemleak: Kernel memory leak detector disabled
[    0.021782] Calibrating delay loop (skipped), value calculated using timer frequency.. 62.50 BogoMIPS (lpj=125000)
[    0.021793] pid_max: default: 32768 minimum: 301
[    0.021841] Security Framework initialized
[    0.021879] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.021889] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.022273] Initializing cgroup subsys io
[    0.022287] Initializing cgroup subsys memory
[    0.022304] Initializing cgroup subsys devices
[    0.022314] Initializing cgroup subsys freezer
[    0.022323] Initializing cgroup subsys net_cls
[    0.022333] Initializing cgroup subsys perf_event
[    0.022343] Initializing cgroup subsys net_prio
[    0.022353] Initializing cgroup subsys pids
[    0.022362] Initializing cgroup subsys debug
[    0.022503] CPU0 ipc=752
[    0.022513] CPU1 ipc=1024
[    0.022522] CPU2 ipc=1024
[    0.022530] CPU3 ipc=752
[    0.022540] CPU4 ipc=752
[    0.022549] CPU5 ipc=752
[    0.022588] ASID allocator initialised with 65536 entries
[    0.044732] tegra-id: chipid=21817.
[    0.044750] tegra-id: opt_subrevision=1.
[    0.044763] Tegra Revision: A02p SKU: 0xdc CPU Process: 0 SoC Process: 0
[    0.045257] DTS File Name: /home/d/Downloads/Jetpack3.2/64_TX2/Linux_for_Tegra/sources/kernel/kernel-4.4/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t18x/quill/kernel-dts/tegra186-quill-p3310-1000-c03-00-base.dts
[    0.045285] DTB Build time: Jun 29 2018 16:04:42
[    0.062385] CPU1: Booted secondary processor [4e0f0030]
[    0.073932] CPU2: Booted secondary processor [4e0f0030]
[    0.085617] CPU3: Booted secondary processor [411fd073]
[    0.097636] CPU4: Booted secondary processor [411fd073]
[    0.109625] CPU5: Booted secondary processor [411fd073]
[    0.109689] Brought up 6 CPUs
[    0.109749] SMP: Total of 6 processors activated.
[    0.109762] CPU: All CPU(s) started at EL2
[    0.110182] devtmpfs: initialized
[    0.128812] Initilizing CustomIPI irq domain
[    0.129000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.129788] pinctrl core: initialized pinctrl subsystem
[    0.129946] OS set in device tree is not L4T.
[    0.130134] regulator-dummy: no parameters
[    0.130242] Initializing plugin-manager
[    0.130298] Plugin module not found
[    0.130719] node /plugin-manager/soc-prod-a02-fragment match with chip-id A02P
[    0.131018] node /plugin-manager/fragement@0 match with odm-data enable-denver-wdt
[    0.131597] node /plugin-manager/fragement@4 match with odm-data enable-denver-wdt
[    0.132739] node /plugin-manager/fragment-sdwake-p3310-1000-300 match with board >=3310-1000-300
[    0.133066] node /plugin-manager/fragement-pmon-p3310-1000-300 match with board >=3310-1000-300
[    0.133353] node /plugin-manager/fragement-pmon-p3310-1000-800 match with board >=3310-1000-800
[    0.133582] node /plugin-manager/fragment-devslp@0 match with board >=3310-1000-200
[    0.133989] node /plugin-manager/fragment-500-pcie-config match with board >=3310-1000-500
[    0.134410] node /plugin-manager/fragment-500-xusb-config match with board >=3310-1000-500
[    0.134688] node /plugin-manager/fragment-500-e3325-pcie match with board >=3310-1000-500
[    0.135068] node /plugin-manager/fragment-p3310-c00-comm match with board >=3310-1000-800
[    0.135302] node /plugin-manager/fragment-p3310-c00-pmic match with board >=3310-1000-800
[    0.135511] node /plugin-manager/fragment-p3310-c01 match with board >=3310-1000-900
[    0.135753] node /plugin-manager/fragment-p3310-c03 match with board >=3310-1000-B00
[    0.137666] Adding domain adsp-pd to PM domain ape-pd
[    0.139323] NET: Registered protocol family 16
[    0.140134] console [pstore-1] enabled
[    0.140146] pstore: Registered ramoops as persistent store backend
[    0.140160] ramoops: attached 0x200000@0x277080000, ecc: 0/0
[    0.149162] cpuidle: using governor ladder
[    0.153617] cpuidle: using governor menu
[    0.154493] bpmp: waiting for handshake
[    0.154508] bpmp: handshake completed
[    0.154550] bpmp: synchronizing channels
[    0.154574] bpmp: channels synchronized
[    0.154584] bpmp: mail init ok
[    0.154926] vdso: 2 pages (1 code @ ffffffc001285000, 1 data @ ffffffc001284000)
[    0.154957] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.155902] atomic_pool_init():526: DMA: preallocated 1024 KiB pool for atomic allocations
[    0.157320] tegra_powergate_init: DONE
[    0.157344] DTS File Name: /home/d/Downloads/Jetpack3.2/64_TX2/Linux_for_Tegra/sources/kernel/kernel-4.4/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t18x/quill/kernel-dts/tegra186-quill-p3310-1000-c03-00-base.dts
[    0.157372] DTB Build time: Jun 29 2018 16:04:42
[    0.158225] Tegra reboot handler registered.
[    0.158873] Registering Tegra186 clocks (this may take a while)...done
[    0.160215] arm-smmu 12000000.iommu: probing hardware configuration...
[    0.160232] arm-smmu 12000000.iommu: SMMUv2 with:
[    0.160245] arm-smmu 12000000.iommu: 	stage 1 translation
[    0.160258] arm-smmu 12000000.iommu: 	stage 2 translation
[    0.160270] arm-smmu 12000000.iommu: 	nested translation
[    0.160285] arm-smmu 12000000.iommu: 	stream matching with 128 register groups, mask 0x7f80
[    0.160303] arm-smmu 12000000.iommu: SMMU address space size (0x800000) differs from mapped region size (0x1000000)!
[    0.160322] arm-smmu 12000000.iommu: 	64 context banks (0 stage-2 only)
[    0.160336] arm-smmu 12000000.iommu: 	Stage-1: 39-bit VA -> 48-bit IPA
[    0.160349] arm-smmu 12000000.iommu: 	Stage-2: 39-bit IPA -> 48-bit PA
[    0.197106] arm-smmu 12000000.iommu: registered 67 master devices
[    0.199327] iommu: Adding device 3460000.sdhci to group 0
[    0.199752] iommu: Adding device 3400000.sdhci to group 1
[    0.201082] iommu: Adding device 3507000.ahci-sata to group 2
[    0.201333] iommu: Adding device 3160000.i2c to group 3
[    0.201561] iommu: Adding device c240000.i2c to group 4
[    0.201784] iommu: Adding device 3180000.i2c to group 5
[    0.201984] iommu: Adding device 3190000.i2c to group 6
[    0.202271] iommu: Adding device 31b0000.i2c to group 7
[    0.202486] iommu: Adding device 31c0000.i2c to group 8
[    0.202698] iommu: Adding device c250000.i2c to group 9
[    0.202897] iommu: Adding device 31e0000.i2c to group 10
[    0.204032] iommu: Adding device 3210000.spi to group 11
[    0.204237] iommu: Adding device c260000.spi to group 12
[    0.204434] iommu: Adding device 3240000.spi to group 13
[    0.204909] iommu: Adding device 3100000.serial to group 14
[    0.205117] iommu: Adding device 3110000.serial to group 15
[    0.205331] iommu: Adding device c280000.serial to group 16
[    0.205541] iommu: Adding device 3130000.serial to group 17
[    0.206423] iommu: Adding device 2490000.ether_qos to group 18
[    0.206810] iommu: Adding device b000000.rtcpu to group 19
[    0.207115] iommu: Adding device c1a0000.aon to group 20
[    0.207702] Wake73 for irq=42
[    0.207893] iommu: Adding device smmu_test to group 21
[    0.208269] mc: mapped MMIO address: 0xffffff8000560000 -> 0x2c10000
[    0.208292] mc: mapped MMIO address: 0xffffff8000640000 -> 0x2c20000
[    0.208313] mc: mapped MMIO address: 0xffffff8000660000 -> 0x2c30000
[    0.208333] mc: mapped MMIO address: 0xffffff8000fa0000 -> 0x2c40000
[    0.208353] mc: mapped MMIO address: 0xffffff8000fc0000 -> 0x2c50000
[    0.208373] mc-err: Set intmask: 0xf3140
[    0.208513] ecc-err: dram ecc disabled-MC_ECC_CONTROL:0x0000000c
[    0.209057] Wake76 for irq=199
[    0.209070] Wake77 for irq=199
[    0.209080] Wake78 for irq=199
[    0.209090] Wake79 for irq=199
[    0.209099] Wake80 for irq=199
[    0.209109] Wake81 for irq=199
[    0.209118] Wake82 for irq=199
[    0.209189] iommu: Adding device 3530000.xhci to group 22
[    0.209455] iommu: Adding device 3550000.xudc to group 23
[    0.209978] tegra-pmc c360000.pmc: scratch reg offset dts data not present
[    0.209995] tegra-pmc: ### PMC reset source: Power on reset
[    0.210008] tegra-pmc: ### PMC reset level: L0
[    0.210019] tegra-pmc: ### PMC reset status reg: 0x0
[    0.210157] padctrl padctrl.0: Pad control driver tegra-pmc-padctrl registered
[    0.210177] tegra-pmc c360000.pmc: IO padctrl driver initialized
[    0.210275] tegra186-aowake c370000.pmc: WAKE_AOWAKE_CTRL_0 = 3
[    0.210291] tegra186-aowake c370000.pmc: WAKE_AOWAKE_CNTRL_24(PMU_INT) = 320
[    0.210594] iommu: Adding device 13e10000.host1x to group 24
[    0.210752] iommu: Adding device 13e10000.host1x:ctx0 to group 25
[    0.210906] iommu: Adding device 13e10000.host1x:ctx1 to group 26
[    0.211070] iommu: Adding device 13e10000.host1x:ctx2 to group 27
[    0.211225] iommu: Adding device 13e10000.host1x:ctx3 to group 28
[    0.211391] iommu: Adding device 13e10000.host1x:ctx4 to group 29
[    0.211540] iommu: Adding device 13e10000.host1x:ctx5 to group 30
[    0.211686] iommu: Adding device 13e10000.host1x:ctx6 to group 31
[    0.211844] iommu: Adding device 13e10000.host1x:ctx7 to group 32
[    0.212119] iommu: Adding device 150c0000.nvcsi to group 33
[    0.212495] iommu: Adding device 15700000.vi to group 34
[    0.212718] iommu: Adding device 15600000.isp to group 35
[    0.212940] iommu: Adding device 15210000.nvdisplay to group 36
[    0.213077] iommu: Adding device 15340000.vic to group 37
[    0.213206] iommu: Adding device 154c0000.nvenc to group 38
[    0.213334] iommu: Adding device 15480000.nvdec to group 39
[    0.213549] iommu: Adding device 15380000.nvjpg to group 40
[    0.213690] iommu: Adding device 15500000.tsec to group 41
[    0.213824] iommu: Adding device 15100000.tsecb to group 42
[    0.214310] iommu: Adding device 15810000.se to group 43
[    0.214661] iommu: Adding device 15820000.se to group 44
[    0.215139] iommu: Adding device 15830000.se to group 45
[    0.215501] iommu: Adding device 15840000.se to group 46
[    0.216416] iommu: Adding device 17000000.gp10b to group 47
[    0.219037] tegra-reset 5000000.clock: registered 193 resets.
[    0.221526] iommu: Adding device d000000.bpmp to group 48
[    0.221798] bpmp: ping status is 0
[    0.221972] bpmp d000000.bpmp: firmware tag is 83385b61e34297b2f37e76e0cd1ff81e
[    0.222410] bpmp d000000.bpmp: probe ok
[    0.225046] iommu: Adding device 2600000.dma to group 49
[    0.225933] GPIO line 424 (wifi-enable) hogged as output/high
[    0.225989] GPIO line 445 (sdmmc-wake-input) hogged as input
[    0.226041] GPIO line 446 (sdmmc-wake-output) hogged as output/low
[    0.226172] gpiochip_add_data: registered GPIOs 320 to 511 on device: tegra-gpio
[    0.228069] GPIO line 315 (wifi-wake-ap) hogged as input
[    0.228215] gpiochip_add_data: registered GPIOs 256 to 319 on device: tegra-gpio-aon
[    0.229120] iommu: Adding device 10003000.pcie-controller to group 50
[    0.229395] iommu: Adding device sound to group 51
[    0.231118] iommu: Adding device 3510000.hda to group 52
[    0.232194] iommu: Adding device adsp_audio to group 53
[    0.233531] iommu: Adding device 2993000.adsp to group 54
[    0.243686] vdd-ac-bat: 5000 mV 
[    0.246356] vdd-1v8-ap: 1800 mV 
[    0.264779] SCSI subsystem initialized
[    0.264950] libata version 3.00 loaded.
[    0.265147] usbcore: registered new interface driver usbfs
[    0.265208] usbcore: registered new interface driver hub
[    0.265323] usbcore: registered new device driver usb
[    0.268230] tegra-i2c 3160000.i2c: no acknowledge from address 0x74
[    0.268437] pca953x 0-0074: failed reading register
[    0.268509] pca953x: probe of 0-0074 failed with error -121
[    0.277698] tegra-i2c c250000.i2c: no acknowledge from address 0x77
[    0.277923] pca953x 7-0077: failed reading register
[    0.277996] pca953x: probe of 7-0077 failed with error -121
[    0.280053] media: Linux media interface: v0.10
[    0.280118] Linux video capture interface: v2.00
[    0.284084] pps_core: LinuxPPS API ver. 1 registered
[    0.284115] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.284169] PTP clock support registered
[    0.285122] tegra_wdt_t18x 30c0000.watchdog: Tegra WDT init timeout = 120 sec
[    0.285181] tegra_wdt_t18x 30c0000.watchdog: Registered successfully
[    0.286784] max77620 4-003c: PMIC Version OTP:0x45 and ES:0x8
[    0.288990] GPIO line 253 (spmic_gpio_input_5) hogged as input
[    0.289185] GPIO line 254 (spmic_gpio_input_6) hogged as input
[    0.289347] gpiochip_add_data: registered GPIOs 248 to 255 on device: max77620-gpio
[    0.294592] vddio-ddr: at 1125 mV 
[    0.298401] avdd_dsi_csi: 1200 mV 
[    0.302426] vdd-1v8: 1800 mV 
[    0.306399] vdd-3v3-sys: 3300 mV 
[    0.307473] spmic-ldo0: at 1800 mV 
[    0.308325] spmic-ldo1: at 800 mV 
[    0.314398] vddio-3v3: 3300 mV 
[    0.315016] vddio-sdmmc1: 1800 <--> 3300 mV at 3300 mV 
[    0.317161] vdd-rtc: at 800 mV 
[    0.317873] avdd-ts-hv: 1800 <--> 3300 mV at 1800 mV 
[    0.321160] spmic-ldo6: at 1500 mV 
[    0.325158] vdd-pex-1v05: 1000 mV 
[    0.325712] dvdd-pex: 1000 mV 
[    0.325976] max77620 4-003c: max77620 probe successful
[    0.326229] Advanced Linux Sound Architecture Driver Initialized.
[    0.327215] en-vdd-sd: 3300 mV 
[    0.327354] en-vdd-cam: 1800 mV 
[    0.327560] vdd-usb0-5v: 5000 mV 
[    0.327724] vdd-usb1-5v: 5000 mV 
[    0.328004] en-vdd-cam-hv-2v8: 2800 mV 
[    0.328120] en-vdd-cam-1v2: 1200 mV 
[    0.328279] vdd-3v3: 3300 mV 
[    0.328413] en-vdd-vcm-2v8: 2800 mV 
[    0.328542] vdd-sys-bl: 3300 mV 
[    0.331744] extcon-gpio-states external-connection:extcon@1: Cable state 0
[    0.332144] clocksource: Switched to clocksource arch_sys_counter
[    0.351018] nvmap_heap_init: nvmap_heap_init: created heap block cache
[    0.351285] nvmap: nvmap_override_cache_ops() set roc flush ops to replace cache ops by set/ways
[    0.351755] nvmap_page_pool_init: Total RAM pages: 2010509
[    0.351774] nvmap_page_pool_init: nvmap page pool size: 251313 pages (981 MB)
[    0.351960] nvmap_background_zero_thread: PP zeroing thread starting.
[    0.356394] Wake76 for irq=199
[    0.356408] Wake77 for irq=199
[    0.356419] Wake78 for irq=199
[    0.356430] Wake79 for irq=199
[    0.356439] Wake80 for irq=199
[    0.356449] Wake81 for irq=199
[    0.356459] Wake82 for irq=199
[    0.357169] thermal thermal_zone0: Registering thermal zone thermal_zone0 for type BCPU-therm
[    0.357373] thermal thermal_zone1: Registering thermal zone thermal_zone1 for type MCPU-therm
[    0.357708] thermal thermal_zone2: Registering thermal zone thermal_zone2 for type GPU-therm
[    0.357784] thermal thermal_zone3: Registering thermal zone thermal_zone3 for type PLL-therm
[    0.357930] thermal thermal_zone4: Registering thermal zone thermal_zone4 for type Tboard_tegra
[    0.358071] thermal thermal_zone5: Registering thermal zone thermal_zone5 for type Tdiode_tegra
[    0.358210] thermal thermal_zone6: Registering thermal zone thermal_zone6 for type PMIC-Die
[    0.358505] isomgr_init(): iso emc max clk=1866000KHz
[    0.358518] isomgr_init(): max_iso_bw=26870400KB
[    0.358603] NET: Registered protocol family 2
[    0.358946] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[    0.359167] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.359459] TCP: Hash tables configured (established 65536 bind 65536)
[    0.359515] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    0.359568] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    0.359727] NET: Registered protocol family 1
[    0.359912] RPC: Registered named UNIX socket transport module.
[    0.359927] RPC: Registered udp transport module.
[    0.359940] RPC: Registered tcp transport module.
[    0.359952] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.359972] PCI: CLS 0 bytes, default 128
[    0.360071] Trying to unpack rootfs image as initramfs...
[    0.373585] host1x 13e10000.host1x: initialized
[    0.374343] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
[    0.376492] futex hash table entries: 2048 (order: 6, 262144 bytes)
[    0.376594] audit: initializing netlink subsys (disabled)
[    0.376624] audit: type=2000 audit(0.364:1): initialized
[    0.383697] io scheduler noop registered
[    0.383805] io scheduler cfq registered (default)
[    0.386735] tegra-pwm 3280000.pwm: PWM clk can sleep in ops
[    0.387897] tegra-pwm 3290000.pwm: PWM clk can sleep in ops
[    0.388977] tegra-pwm 32a0000.pwm: PWM clk can sleep in ops
[    0.390010] tegra-pwm c340000.pwm: PWM clk can sleep in ops
[    0.391690] PCIE: tegra_pcie_probe(4681)
[    0.391708] PCIE: tegra_pcie_read_plat_data(3233)
[    0.391785] PCIE: tegra_pcie_parse_dt(3321)
[    0.391861] tegra-pcie 10003000.pcie-controller: 4x1, 1x1 configuration
[    0.391902] PCIE: tegra_pcie_probe_complete(4553)
[    0.391914] PCIE: tegra_pcie_init(2841)
[    0.391926] PCIE: tegra_pcie_get_resources(1952)
[    0.391938] PCIE: tegra_pcie_get_clocks(1351)
[    0.393287] PCIE: tegra_pcie_enable_regulators(1577)
[    0.393305] tegra-pcie 10003000.pcie-controller: PCIE: Enable power rails
[    0.393379] PCIE: tegra_pcie_power_on(1833)
[    0.393395] PCIE: tegra_pcie_module_power_ungate(1744)
[    0.393407] PCIE: tegra_pcie_enable_regulators(1577)
[    0.393582] PCIE: tegra_pcie_restore_device(1785)
[    0.393595] PCIE: tegra_pcie_map_resources(1622)
[    0.393623] PCIE: tegra_pcie_enable_pads(1466)
[    0.393724] PCIE: tegra_pcie_enable_controller(1509)
[    0.393768] PCIE: tegra_pcie_enable_msi(3059)
[    0.393838] PCIE: tegra_pcie_check_ports(2504)
[    0.393851] tegra-pcie 10003000.pcie-controller: probing port 0, using 4 lanes
[    0.393867] PCIE: tegra_pcie_port_enable(2036)
[    0.393878] PCIE: tegra_pcie_port_reset(2007)
[    0.394590] tsec 15500000.tsec: initialized
[    0.395619] tsec 15100000.tsecb: initialized
[    0.395906] PCIE: tegra_pcie_enable_rp_features(2375)
[    0.395929] PCIE: tegra_pcie_enable_aer(1117)
[    0.395947] PCIE: tegra_pcie_apply_sw_war(2217)
[    0.396189] PCIE: tegra_pcie_prsnt_map_override(1175)
[    0.396204] tegra-pcie 10003000.pcie-controller: probing port 2, using 1 lanes
[    0.396220] PCIE: tegra_pcie_port_enable(2036)
[    0.396231] PCIE: tegra_pcie_port_reset(2007)
[    0.397244] nvdec 15480000.nvdec: initialized
[    0.398257] PCIE: tegra_pcie_enable_rp_features(2375)
[    0.398280] PCIE: tegra_pcie_enable_aer(1117)
[    0.398295] PCIE: tegra_pcie_apply_sw_war(2217)
[    0.398421] PCIE: tegra_pcie_prsnt_map_override(1175)
[    0.400009] falcon 15340000.vic: initialized
[    0.401006] falcon 154c0000.nvenc: initialized
[    0.401958] falcon 15380000.nvjpg: initialized
[    0.402641] iommu_context_dev 13e10000.host1x:ctx0: initialized (streamid=56)
[    0.403173] iommu_context_dev 13e10000.host1x:ctx1: initialized (streamid=57)
[    0.403675] iommu_context_dev 13e10000.host1x:ctx2: initialized (streamid=58)
[    0.404190] iommu_context_dev 13e10000.host1x:ctx3: initialized (streamid=59)
[    0.404724] iommu_context_dev 13e10000.host1x:ctx4: initialized (streamid=60)
[    0.405229] iommu_context_dev 13e10000.host1x:ctx5: initialized (streamid=61)
[    0.405739] iommu_context_dev 13e10000.host1x:ctx6: initialized (streamid=62)
[    0.406265] iommu_context_dev 13e10000.host1x:ctx7: initialized (streamid=63)
[    0.407341] DC OR NODE connected to /host1x/sor1
[    0.407418] can not determine display type
[    0.407429] parse_dc_out_type err
[    0.407443] tegra_camera_platform tegra-camera-platform: tegra_camera_probe:camera_platform_driver probe
[    0.407544] misc tegra_camera_ctrl: tegra_camera_isomgr_register isp_iso_bw=4687500, vi_iso_bw=1500000, max_bw=4687500
[    0.412974] tegra-adma 2930000.adma: Tegra ADMA driver register 10 channels
[    0.413919] tegra-fuse-burn 3820000.efuse:efuse-burn: Fuse burn driver initialized
[    0.414203] kfuse 3830000.kfuse: initialized
[    0.415279] tegra-pmc-iopower pmc-iopower: NO_IOPOWER setting 0x0
[    0.415665] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.417555] console [ttyS0] disabled
[    0.417601] 3100000.serial: ttyS0 at MMIO 0x3100000 (irq = 37, base_baud = 25500000) is a Tegra
[    0.821965] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    0.821967] PCIE: tegra_pcie_port_reset(2007)
[    1.226356] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    1.226358] PCIE: tegra_pcie_port_reset(2007)
[    1.630740] tegra-pcie 10003000.pcie-controller: link 0 down, retrying
[    1.630742] PCIE: tegra_pcie_port_reset(2007)
[    1.632758] tegra-pcie 10003000.pcie-controller: link 0 down, ignoring
[    1.632760] PCIE: tegra_pcie_port_disable(2067)
[    2.035118] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    2.035119] PCIE: tegra_pcie_port_reset(2007)
[    2.439492] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    2.439493] PCIE: tegra_pcie_port_reset(2007)
[    2.597744] console [ttyS0] enabled
[    2.603473] 3110000.serial: ttyTHS1 at MMIO 0x3110000 (irq = 38, base_baud = 0) is a TEGRA_UART
[    2.613286] c280000.serial: ttyTHS2 at MMIO 0xc280000 (irq = 39, base_baud = 0) is a TEGRA_UART
[    2.622269] serial-tegra 3130000.serial: RX in PIO mode
[    2.628476] 3130000.serial: ttyTHS3 at MMIO 0x3130000 (irq = 40, base_baud = 0) is a TEGRA_UART
[    2.648548] brd: module loaded
[    2.656704] loop: module loaded
[    2.660191] nct1008_nct72 7-004c: find device tree node, parsing dt
[    2.664647] gk20a 17000000.gp10b: failed to allocate secure buffer -12
[    2.673104] nct1008_nct72 7-004c: starting parse dt
[    2.678109] nct1008_nct72 7-004c: success parsing dt
[    2.683261] nct1008_nct72 7-004c: success in enabling tmp451 VDD rail
[    2.726289] tmp451: Enabled overheat logging at 104.00C
[    2.731717] nct1008_nct72 7-004c: nct1008_probe: initialized
[    2.739597] THERMAL EST: found 3 subdevs
[    2.743598] THERMAL EST num_resources: 0
[    2.747582] [THERMAL EST subdev 0]
[    2.751038] [THERMAL EST subdev 1]
[    2.754505] [THERMAL EST subdev 2]
[    2.758342] thermal thermal_zone7: Registering thermal zone thermal_zone7 for type thermal-fan-est
[    2.767339] THERMAL EST: thz register success.
[    2.772027] THERMAL EST: end of probe, return err: 0
[    2.777149] tegra_profiler: Branch: Dev
[    2.781020] tegra_profiler: Version: 1.117
[    2.785173] tegra_profiler: Samples version: 41
[    2.789795] tegra_profiler: IO version: 22
[    2.793926] armv8_pmu: imp: 0x41, idcode: 0x1
[    2.798313] armv8_pmu: [0] arch: AA64 PmuV3 ARM CORTEX-A57, type: 5, ver: 0, pmu ver: 0x1
[    2.806520] armv8_pmu: imp: 0x4e, idcode: 0x1
[    2.810906] armv8_pmu: [1] arch: AA64 PmuV3 NVIDIA (Denver), type: 6, ver: 2, pmu ver: 0x1
[    2.819196] armv8_pmu: imp: 0x4e, idcode: 0x1
[    2.823580] armv8_pmu: [2] arch: AA64 PmuV3 NVIDIA (Denver), type: 6, ver: 2, pmu ver: 0x1
[    2.831873] armv8_pmu: imp: 0x41, idcode: 0x1
[    2.836264] armv8_pmu: [3] arch: AA64 PmuV3 ARM CORTEX-A57, type: 5, ver: 0, pmu ver: 0x1
[    2.843850] tegra-pcie 10003000.pcie-controller: link 2 down, retrying
[    2.843852] PCIE: tegra_pcie_port_reset(2007)
[    2.845868] tegra-pcie 10003000.pcie-controller: link 2 down, ignoring
[    2.845869] PCIE: tegra_pcie_port_disable(2067)
[    2.845875] PCIE: tegra_pcie_conf_gpios(2556)
[    2.845877] tegra-pcie 10003000.pcie-controller: PCIE: no end points detected
[    2.845878] PCIE: tegra_pcie_power_off(1915)
[    2.845882] PCIE: tegra_pcie_save_device(1848)
[    2.845884] PCIE: tegra_pcie_prsnt_map_override(1175)
[    2.845886] PCIE: tegra_pcie_prsnt_map_override(1175)
[    2.845888] PCIE: tegra_pcie_pme_turnoff(1720)
[    2.845894] PCIE: tegra_pcie_enable_pads(1466)
[    2.845896] PCIE: tegra_pcie_unmap_resources(1678)
[    2.846086] PCIE: tegra_pcie_module_power_gate(1882)
[    2.846189] PCIE: tegra_pcie_disable_regulators(1599)
[    2.846192] tegra-pcie 10003000.pcie-controller: PCIE: Disable power rails
[    2.927573] armv8_pmu: imp: 0x41, idcode: 0x1
[    2.931963] armv8_pmu: [4] arch: AA64 PmuV3 ARM CORTEX-A57, type: 5, ver: 0, pmu ver: 0x1
[    2.940165] armv8_pmu: imp: 0x41, idcode: 0x1
[    2.944544] armv8_pmu: [5] arch: AA64 PmuV3 ARM CORTEX-A57, type: 5, ver: 0, pmu ver: 0x1
[    2.952974] tegra_profiler: auth: init
[    2.960193] tegra-ahci 3507000.ahci-sata: AHCI 0001.0301 32 slots 2 ports 3 Gbps 0x1 impl platform mode
[    2.969653] tegra-ahci 3507000.ahci-sata: flags: 64bit ncq sntf pm led pmp pio slum part deso sadm apst 
[    2.980284] scsi host0: tegra_ahci
[    2.984099] scsi host1: tegra_ahci
[    2.987729] ata1: SATA max UDMA/133 mmio [mem 0x03507000-0x03508fff] port 0x100 irq 25
[    2.995679] ata2: DUMMY
[    2.999210] spi-tegra114 3210000.spi: Static pin configuration used
[    3.005971] spi-tegra114 c260000.spi: Static pin configuration used
[    3.012685] spi-tegra114 3240000.spi: Static pin configuration used
[    3.019971] tun: Universal TUN/TAP device driver, 1.6
[    3.025093] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    3.031547] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    3.037409] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    3.043425] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
[    3.050407] igb: Copyright (c) 2007-2014 Intel Corporation.
[    3.057230] PPP generic driver version 2.4.2
[    3.059873] eqos 2490000.ether_qos: Setting local MAC: 0 4 4b a5 2f b2
[    3.060307] libphy: dwc_phy: probed
[    3.071765] PPP BSD Compression module registered
[    3.076504] PPP Deflate Compression module registered
[    3.081620] PPP MPPE Compression module registered
[    3.086456] NET: Registered protocol family 24
[    3.091024] usbcore: registered new interface driver r8152
[    3.096602] usbcore: registered new interface driver asix
[    3.102070] usbcore: registered new interface driver ax88179_178a
[    3.108258] usbcore: registered new interface driver cdc_ether
[    3.114180] usbcore: registered new interface driver smsc75xx
[    3.120004] usbcore: registered new interface driver net1080
[    3.125286] bcm54xx_low_power_mode(): put phy in iddq-lp mode
[    3.131508] usbcore: registered new interface driver cdc_subset
[    3.137515] usbcore: registered new interface driver zaurus
[    3.143183] usbcore: registered new interface driver cdc_ncm
[    3.149174] Wake76 for irq=199
[    3.152240] Wake77 for irq=199
[    3.155299] Wake78 for irq=199
[    3.158358] Wake79 for irq=199
[    3.161416] Wake80 for irq=199
[    3.164474] Wake81 for irq=199
[    3.167532] Wake82 for irq=199
[    3.170664] tegra-xotg xotg: usb2 phy is not available yet
[    3.176805] usbcore: registered new interface driver cdc_acm
[    3.182494] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[    3.190607] usbcore: registered new interface driver usb-storage
[    3.196728] usbcore: registered new interface driver usbserial
[    3.202616] usbcore: registered new interface driver cp210x
[    3.208232] usbserial: USB Serial support registered for cp210x
[    3.214236] usbcore: registered new interface driver ftdi_sio
[    3.220031] usbserial: USB Serial support registered for FTDI USB Serial Device
[    3.227433] usbcore: registered new interface driver option
[    3.233047] usbserial: USB Serial support registered for GSM modem (1-port)
[    3.240090] usbcore: registered new interface driver pl2303
[    3.245709] usbserial: USB Serial support registered for pl2303
[    3.251924] tegra-usb-cd usb_cd: otg phy is not available yet
[    3.259653] tegra-xudc-new 3550000.xudc: usb2 phy is not available yet
[    3.320145] ata1: SATA link down (SStatus 0 SControl 300)
[    3.398606] max77686-rtc max77620-rtc: rtc core: registered max77620-rtc as rtc0
[    3.408265] tegra_rtc c2a0000.rtc: rtc core: registered c2a0000.rtc as rtc1
[    3.415249] tegra_rtc c2a0000.rtc: Tegra internal Real Time Clock
[    3.421512] i2c /dev entries driver
[    3.425823] [IMX274]: probing v4l2 sensor.
[    3.430420] pca9570 2-0024: pca9570_icr_move, forward val=5
[    3.546178] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.552614] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    3.559545] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.565962] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 2
[    3.682816] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.689239] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    3.696171] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.702593] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 0
[    3.819446] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.825874] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    3.832799] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.839215] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 2
[    3.956072] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.962488] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    3.969415] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    3.975833] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 0
[    4.088257] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.094687] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    4.101635] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.108056] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 2
[    4.224912] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.231331] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    4.238257] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.244674] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 0
[    4.361526] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.367948] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    4.374874] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.381292] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 2
[    4.498145] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.504565] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    4.511488] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.517907] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 0
[    4.634763] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.641182] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    4.648112] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.654529] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 2
[    4.771380] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.777794] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 48
[    4.784739] tegra-i2c 3180000.i2c: no acknowledge from address 0x24
[    4.791158] pca9570 2-0024: pca9570_write_reg:i2c write failed, 0x24 = 0
[    4.797863] pca9570 2-0024: pca9570_icr_move:i2c write failed
[    4.803655] pca9570: probe of 2-0024 failed with error -121
[    4.809278] gspca_main: v2.14.0 registered
[    4.813432] usbcore: registered new interface driver gspca_zc3xx
[    4.819722] max77620-power max77620-power: Event recorder REG_NVERC : 0x0
[    4.828370] FAN: coudln't get the regulator
[    4.833253] device-mapper: uevent: version 1.0.3
[    4.838076] device-mapper: ioctl: 4.34.0-ioctl (2015-10-28) initialised: dm-devel@redhat.com
[    4.846925] sdhci: Secure Digital Host Controller Interface driver
[    4.853115] sdhci: Copyright(c) Pierre Ossman
[    4.858024] sdhci-pltfm: SDHCI platform and OF driver helper
[    4.864149] sdhci-tegra 3460000.sdhci: Parent select= pll_p rate=408000000
[    4.866240] tegra-se-elp 3ad0000.se_elp: tegra_se_elp_probe: complete
[    4.866431] hidraw: raw HID events driver (C) Jiri Kosina
[    4.868676] usbcore: registered new interface driver usbhid
[    4.868678] usbhid: USB HID core driver
[    4.872000] tegra-i2c 3160000.i2c: no acknowledge from address 0x42
[    4.872192] ina3221x 0-0042: ina3221 reset failure status: 0xffffff87
[    4.872371] ina3221x: probe of 0-0042 failed with error -121
[    4.872854] tegra-i2c 3160000.i2c: no acknowledge from address 0x43
[    4.873038] ina3221x 0-0043: ina3221 reset failure status: 0xffffff87
[    4.873197] ina3221x: probe of 0-0043 failed with error -121
[    4.873670] nvadsp 2993000.adsp: in probe()...
[    4.875662] nvadsp 2993000.adsp: Registering AMC Error Interrupt
[    4.875702] nvadsp 2993000.adsp: AMC/ARAM initialized.
[    4.875958] nvadsp 2993000.adsp: nvadsp_app_module_probe
[    4.876769] Denver: backdoor interface is NOT available.
[    4.879096] coresight-tmc 8030000.etf: TMC initialized
[    4.884022] coresight-tmc 8050000.etr: TMC initialized
[    4.884530] coresight-tpiu 8060000.tpiu: TPIU initialized
[    4.885079] coresight-funnel 9010000.funnel_bccplex: FUNNEL initialized
[    4.885637] coresight-funnel 8010000.funnel_major: FUNNEL initialized
[    4.886126] coresight-funnel 8820000.funnel_minor: FUNNEL initialized
[    4.886586] coresight-replicator 8040000.replicator: REPLICATOR initialized
[    4.887288] coresight-etm4x 9840000.ptm: ETM 4.0 initialized
[    4.887992] coresight-etm4x 9940000.ptm: ETM 4.0 initialized
[    4.888647] coresight-etm4x 9a40000.ptm: ETM 4.0 initialized
[    4.898980] coresight-etm4x 9b40000.ptm: ETM 4.0 initialized
[    4.899042] Module initialized successfully 
[    4.902217] tegra186-cam-rtcpu b000000.rtcpu: Trace buffer configured at IOVA=0xa0000000
[    4.903889] tegra186-cam-rtcpu b000000.rtcpu: already unhalted
[    4.904233] tegra-ivc ivc-b000000.rtcpu: region 0: iova=0xa0110000-0xa011957f size=38272
[    4.904673] tegra-ivc ivc-b000000.rtcpu:echo@0: echo: ver=0 grp=1 RX[16x64]=0x1000-0x1480 TX[16x64]=0x1480-0x1900
[    4.904736] tegra-ivc ivc-b000000.rtcpu:vinotify@12c0: vinotify: ver=0 grp=1 RX[64x128]=0x1900-0x3980 TX[64x128]=0x3980-0x5a00
[    4.904786] tegra-ivc ivc-b000000.rtcpu:mods@32c0: mods: ver=0 grp=1 RX[1x64]=0x5a00-0x5ac0 TX[1x64]=0x5ac0-0x5b80
[    4.904835] tegra-ivc ivc-b000000.rtcpu:ivccontrol@52c0: ivccontrol: ver=0 grp=1 RX[16x320]=0x5b80-0x7000 TX[16x320]=0x7000-0x8480
[    4.904886] tegra-ivc ivc-b000000.rtcpu:ivccapture@72c0: ivccapture: ver=0 grp=1 RX[16x64]=0x8480-0x8900 TX[16x64]=0x8900-0x8d80
[    4.904940] tegra-ivc ivc-b000000.rtcpu:dbg@7c00: dbg: ver=0 grp=1 RX[1x384]=0x8d80-0x8f80 TX[1x384]=0x8f80-0x9180
[    4.904996] tegra-ivc ivc-b000000.rtcpu:dbg@7e00: dbg: ver=0 grp=1 RX[1x384]=0x9180-0x9380 TX[1x384]=0x9380-0x9580
[    4.905213] tegra186-cam-rtcpu b000000.rtcpu: using cam RTCPU IRQ (52)
[    4.905217] tegra186-cam-rtcpu b000000.rtcpu: tegra_camrtc_mon_create is successful
[    4.906395] tegra-ivc ivc-b000000.rtcpu:echo@0: ivc channel driver missing
[    4.906400] tegra-ivc ivc-b000000.rtcpu:vinotify@12c0: ivc channel driver missing
[    4.906404] tegra-ivc ivc-b000000.rtcpu:mods@32c0: ivc channel driver missing
[    4.906407] tegra-ivc ivc-b000000.rtcpu:ivccontrol@52c0: ivc channel driver missing
[    4.906411] tegra-ivc ivc-b000000.rtcpu:ivccapture@72c0: ivc channel driver missing
[    4.906414] tegra-ivc ivc-b000000.rtcpu:dbg@7c00: ivc channel driver missing
[    4.906417] tegra-ivc ivc-b000000.rtcpu:dbg@7e00: ivc channel driver missing
[    4.906502] tegra186-cam-rtcpu b000000.rtcpu: firmware version cpu=sce cmd=4 sha1=a8b1ecdd9615cd817a56df035be363fce3b0b88f
[    4.911450] tegra-ivc-cdev ivc-b000000.rtcpu:echo@0: probing /dev/camchar-echo
[    4.912448] tegra-ivc-cdev ivc-b000000.rtcpu:dbg@7c00: probing /dev/camchar-dbg
[    4.912637] camchar: rtcpu character device driver loaded
[    4.916541] tegra186-aondbg aondbg: aondbg driver probe() OK
[    4.916823] register_ari_mca_banks: Registered MCA ROC:IOB
[    4.916910] register_ari_mca_banks: Registered MCA ROC:CCE
[    4.917255] tegra18-bridge 2390000.axi2apb: bridge probed OK
[    4.917291] tegra18-bridge 23a0000.axi2apb: bridge probed OK
[    4.917323] tegra18-bridge 23b0000.axi2apb: bridge probed OK
[    4.917359] tegra18-bridge 23c0000.axi2apb: bridge probed OK
[    4.917392] tegra18-bridge 23d0000.axi2apb: bridge probed OK
[    4.917424] tegra18-bridge 2100000.axip2p: bridge probed OK
[    4.917455] tegra18-bridge 2110000.axip2p: bridge probed OK
[    4.917486] tegra18-bridge 2120000.axip2p: bridge probed OK
[    4.917520] tegra18-bridge 2130000.axip2p: bridge probed OK
[    4.917551] tegra18-bridge 2140000.axip2p: bridge probed OK
[    4.917581] tegra18-bridge 2150000.axip2p: bridge probed OK
[    4.917615] tegra18-bridge 2160000.axip2p: bridge probed OK
[    4.917678] tegra18-bridge 2170000.axip2p: bridge probed OK
[    4.917710] tegra18-bridge 2180000.axip2p: bridge probed OK
[    4.917740] tegra18-bridge 2190000.axip2p: bridge probed OK
[    4.917791] tegra18_a57_serr_init: on CPU 2 a Denver Core
[    4.917954] parse_throttle_dt_data: Num cap clks = 4
[    4.917958] parse_throttle_dt_data: clk=mcpu type=1
[    4.917961] parse_throttle_dt_data: clk=bcpu type=2
[    4.917968] parse_throttle_dt_data: clk=gpu type=4
[    4.917997] parse_throttle_dt_data: clk=emc type=3
[    4.918612] tegra_throttle_probe: probe successful. #cdevs=4
[    4.918713] tegra18x_actmon d230000.actmon: in actmon_register()...
[    4.919330] tegra18x_actmon d230000.actmon: initialization Completed for the device mc_all
[    4.919724] hw perfevents: enabled with denver15_uncore_pmu PMU driver, 3 counters available
[    4.921841] tegra-se-nvhost 15810000.se: initialized
[    4.922429] tegra-se-nvhost 15810000.se: tegra_se_probe: complete
[    4.922787] tegra-se-nvhost 15820000.se: initialized
[    4.923429] tegra-se-nvhost 15820000.se: tegra_se_probe: complete
[    4.923796] tegra-se-nvhost 15830000.se: initialized
[    4.923996] tegra-se-nvhost 15830000.se: tegra_se_probe: complete
[    4.924362] tegra-se-nvhost 15840000.se: initialized
[    4.924946] tegra-se-nvhost 15840000.se: tegra_se_probe: complete
[    4.930626] tegra-gpcdma 2600000.dma: GPC DMA driver register 32 channels
[    4.933934] cpufreq: platform driver Initialization: pass
[    4.934037] tegra_cluster_clk e090000.cluster_clk_priv: in probe()...
[    4.934142] tegra_cluster_clk e090000.cluster_clk_priv: passed
[    4.934288] tegra186-padctl 3520000.pinctrl: padctl mmio start 0x0000000003520000 end 0x0000000003520fff
[    4.934296] tegra186-padctl 3520000.pinctrl: ao mmio start 0x0000000003540000 end 0x0000000003540fff
[    4.934334] tegra186-padctl 3520000.pinctrl: FUSE_SKU_USB_CALIB_0 0x34cc99
[    4.934339] tegra186-padctl 3520000.pinctrl: FUSE_USB_CALIB_EXT_0 0x8
[    4.935658] tegra186-padctl 3520000.pinctrl: using UTMI port 0 for otg
[    4.936298] tegra186-padctl 3520000.pinctrl: VBUS over-current detection enabled
[    4.936858] tegra-bpmp-thermal d000000.bpmp:bpmpthermal: zone 0 not supported
[    4.937252] tegra-bpmp-thermal d000000.bpmp:bpmpthermal: zone 3 not supported
[    4.939614] cpuidle: Initializing cpuidle driver init for Denver cluster
[    4.939753] cpuidle: Initializing cpuidle driver init for A57 cluster
[    4.940971] tachometer tegra-tachometer: Tachometer driver tegra-tachometer registered
[    4.940975] Tachometer driver initialized with pulse_per_rev: 2 and win_len: 2
[    4.944865] tegra210_adsp_audio_platform_probe: platform probe started
[    4.945948] tegra210-adsp adsp_audio: Default param-type to BYTE for mp3-dec1
[    4.950883] input: tegra-hda HDMI/DP,pcm=3 as /devices/3510000.hda/sound/card0/input0
[    4.951072] input: tegra-hda HDMI/DP,pcm=7 as /devices/3510000.hda/sound/card0/input1
[    4.951319] tegra210-adsp adsp_audio: Default param-type to BYTE for spkprot
[    4.951976] tegra210-adsp adsp_audio: Default param-type to BYTE for src
[    4.952360] tegra210-adsp adsp_audio: Default param-type to BYTE for aac-dec1
[    4.952794] tegra210-adsp adsp_audio: Default param-type to BYTE for aec
[    4.952800] tegra210-adsp adsp_audio: adma channel page address dt entry not found
[    4.952802] tegra210-adsp adsp_audio: using adma channel page 0
[    4.965011] tegra210_adsp_audio_platform_probe probe successfull.
[    4.965012] OPE platform probe
[    4.965120] OPE platform probe successful
[    5.055226] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF1 <-> ADMAIF1 mapping ok
[    5.055352] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF2 <-> ADMAIF2 mapping ok
[    5.055447] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF3 <-> ADMAIF3 mapping ok
[    5.055540] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF4 <-> ADMAIF4 mapping ok
[    5.055640] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF5 <-> ADMAIF5 mapping ok
[    5.055735] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF6 <-> ADMAIF6 mapping ok
[    5.055836] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF7 <-> ADMAIF7 mapping ok
[    5.055929] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF8 <-> ADMAIF8 mapping ok
[    5.056028] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF9 <-> ADMAIF9 mapping ok
[    5.056171] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF10 <-> ADMAIF10 mapping ok
[    5.056268] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF11 <-> ADMAIF11 mapping ok
[    5.056372] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF12 <-> ADMAIF12 mapping ok
[    5.056466] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF13 <-> ADMAIF13 mapping ok
[    5.056559] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF14 <-> ADMAIF14 mapping ok
[    5.056654] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF15 <-> ADMAIF15 mapping ok
[    5.056748] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF16 <-> ADMAIF16 mapping ok
[    5.056849] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF17 <-> ADMAIF17 mapping ok
[    5.056946] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF18 <-> ADMAIF18 mapping ok
[    5.057041] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF19 <-> ADMAIF19 mapping ok
[    5.057183] tegra-snd-t186ref-mobile-rt565x sound: ADMAIF20 <-> ADMAIF20 mapping ok
[    5.060743] tegra-snd-t186ref-mobile-rt565x sound: ADSP-FE1 <-> ADSP PCM1 mapping ok
[    5.060842] tegra-snd-t186ref-mobile-rt565x sound: ADSP-FE2 <-> ADSP PCM2 mapping ok
[    5.060867] compress asoc: ADSP-FE3 <-> ADSP COMPR1 mapping ok
[    5.060892] compress asoc: ADSP-FE4 <-> ADSP COMPR2 mapping ok
[    5.094904] input: tegra-snd-t186ref-mobile-rt565x Headphone Jack as /devices/sound/sound/card1/input2
[    5.095611] tegra-snd-t186ref-mobile-rt565x sound: codec-dai "dit-hifi" registered
[    5.095614] tegra-snd-t186ref-mobile-rt565x sound: This is a dummy codec
[    5.096056] GACT probability NOT on
[    5.096064] Mirror/redirect action on
[    5.096074] u32 classifier
[    5.096075]     Actions configured
[    5.096096] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
[    5.096719] IPVS: Registered protocols ()
[    5.096744] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[    5.096844] IPVS: Creating netns size=1424 id=0
[    5.096868] IPVS: ipvs loaded.
[    5.096873] IPVS: [rr] scheduler registered.
[    5.096963] ipip: IPv4 over IPv4 tunneling driver
[    5.097288] Initializing XFRM netlink socket
[    5.097563] NET: Registered protocol family 10
[    5.098256] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    5.098318] NET: Registered protocol family 17
[    5.098330] NET: Registered protocol family 15
[    5.098352] bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.
[    5.098357] 8021q: 802.1Q VLAN Support v1.8
[    5.101536] Registered cp15_barrier emulation handler
[    5.111879] Registered setend emulation handler
[    5.113362] registered taskstats version 1
[    5.115869] isp 15600000.isp: initialized
[    5.119184] nvcsi 150c0000.nvcsi: initialized
[    5.120802] Wake76 for irq=199
[    5.120803] Wake77 for irq=199
[    5.120804] Wake78 for irq=199
[    5.120804] Wake79 for irq=199
[    5.120805] Wake80 for irq=199
[    5.120806] Wake81 for irq=199
[    5.120806] Wake82 for irq=199
[    5.121062] xhci-tegra 3530000.xhci: UTMI port 0 has OTG_CAP
[    5.121064] xhci-tegra 3530000.xhci: No USB3 port has OTG_CAP
[    5.144178] xhci-tegra 3530000.xhci: Direct firmware load for tegra18x_xusb_firmware failed with error -2
[    5.144179] xhci-tegra 3530000.xhci: Falling back to user helper
[    5.144425] spi-tegra114 3210000.spi: Static pin configuration used
[    5.145064] spi-tegra114 c260000.spi: Static pin configuration used
[    5.145746] spi-tegra114 3240000.spi: Static pin configuration used
[    5.146419] tegra-xotg xotg: OTG rev:0200, ADP:0, SRP:1, HNP:1, RSP:0
[    5.146454] tegra-xotg xotg: update_id_state: ID floating
[    5.146458] tegra-xotg xotg: update_vbus_state: VBUS not detected
[    5.146477] tegra-xotg xotg: Nvidia XUSB OTG Controller
[    5.147891] tegra-xotg xotg: otg: gadget gadget registered
[    5.147893] tegra-xotg xotg: set gadget otg_caps from OTG controller
[    5.147894] tegra-xotg xotg: otg: host not registered yet
[    5.147896] tegra-xotg xotg: otg: start OTG finite state machine
[    5.147994] tegra-xudc-new 3550000.xudc: entering ELPG
[    5.149073] tegra-xudc-new 3550000.xudc: entering ELPG done
[    5.149440] input: gpio-keys as /devices/gpio-keys/input/input3
[    5.187282] tegra_rtc c2a0000.rtc: setting system clock to 2000-01-01 01:00:16 UTC (946688416)
[    5.216805] bpmp: mounted debugfs mirror
[    5.217268] [dram-timers] DRAM derating cdev registered.
[    5.221038] spmic-ldo0: disabling
[    5.221210] spmic-ldo1: disabling
[    5.221357] vddio-sdmmc1: disabling
[    5.221526] avdd-ts-hv: disabling
[    5.221689] en-vdd-sd: disabling
[    5.221691] en-vdd-cam: disabling
[    5.221693] vdd-usb0-5v: disabling
[    5.221694] vdd-usb1-5v: disabling
[    5.221695] en-vdd-cam-hv-2v8: disabling
[    5.221697] en-vdd-cam-1v2: disabling
[    5.221698] vdd-3v3: disabling
[    5.221700] en-vdd-vcm-2v8: disabling
[    5.221701] vdd-sys-bl: disabling
[    5.228441] ALSA device list:
[    5.228444]   #0: tegra-hda at 0x3518000 irq 400
[    5.228445]   #1: tegra-snd-t186ref-mobile-rt565x
[    5.229268] tegra-vi4 15700000.vi: initialized
[    5.230462] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi-0 bound
[    5.230465] tegra-vi4 15700000.vi: subdev imx274 2-001a bound
[    6.150084] sdhci-tegra 3460000.sdhci: Parent select= pll_c4_out0 rate=196249804
[    6.157924] sdhci-tegra 3460000.sdhci: Client registration for eMC Successful
[    6.200155] mmc0: SDHCI controller on 3460000.sdhci [3460000.sdhci] using ADMA 64-bit with 64 bit addr
[    6.209795] sdhci-tegra 3440000.sdhci: Parent select= pll_p rate=408000000
[    6.216966] sdhci-tegra 3440000.sdhci: Client registration for eMC Successful
[    6.254152] mmc0: MAN_BKOPS_EN bit is not set
[    6.260139] mmc1: SDHCI controller on 3440000.sdhci [3440000.sdhci] using ADMA 64-bit with 64 bit addr
[    6.266327] mmc0: Skipping tuning since strobe enabled
[    6.273772] mmc0: periodic cache flush enabled
[    6.273778] mmc0: new HS400 MMC card at address 0001
[    6.273876] mmcblk mmc0:0001: Card claimed for testing.
[    6.274040] mmcblk0: mmc0:0001 032G34 29.1 GiB 
[    6.274171] mmcblk0boot0: mmc0:0001 032G34 partition 1 4.00 MiB
[    6.274289] mmcblk0boot1: mmc0:0001 032G34 partition 2 4.00 MiB
[    6.274402] mmcblk0rpmb: mmc0:0001 032G34 partition 3 4.00 MiB
[    6.275618]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 p29
[    6.323470] sdhci-tegra 3400000.sdhci: Got CD GPIO
[    6.328340] sdhci-tegra 3400000.sdhci: Got WP GPIO
[    6.333194] sdhci-tegra 3400000.sdhci: Parent select= pll_p rate=408000000
[    6.340098] sdhci-tegra 3400000.sdhci: wakeup init done, cd_irq: 256
[    6.346753] sdhci-tegra 3400000.sdhci: Client registration for eMC Successful
[    6.396219] mmc2: SDHCI controller on 3400000.sdhci [3400000.sdhci] using ADMA 64-bit with 64 bit addr
[    6.474626] EXT4-fs (mmcblk0p1): 8 orphan inodes deleted
[    6.479957] EXT4-fs (mmcblk0p1): recovery complete
[    6.493949] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)
[    6.502056] VFS: Mounted root (ext4 filesystem) on device 179:1.
[    6.509782] mmc1 tuning done saved tap delay=19
[    6.510069] devtmpfs: mounted
[    6.510356] Freeing unused kernel memory: 1168K (ffffffc001137000 - ffffffc00125b000)
[    6.510380] Freeing alternatives memory: 100K (ffffffc00125b000 - ffffffc001274000)
[    6.517546] btb inv war enabled
[    6.535915] mmc1: hw tuning done ...
[    6.539493] mmc1: tuning_window[0]: 0xffffffff
[    6.543937] mmc1: tuning_window[1]: 0xfffff8ff
[    6.548391] mmc1: tuning_window[2]: 0xffc7ffff
[    6.548401] mmc1: tuning_window[3]: 0x3fffffff
[    6.548406] mmc1: tuning_window[4]: 0x0
[    6.548411] mmc1: tuning_window[5]: 0x0
[    6.548415] mmc1: tuning_window[6]: 0x0
[    6.548420] mmc1: tuning_window[7]: 0x0
[    6.548422] sdhci: Tap value: 19 | Trim value: 5
[    6.548425] sdhci: SDMMC_VENDOR_INTR_STATUS[0x108]: 0x40000
[    6.588380] mmc1: new ultra high speed SDR104 SDIO card at address 0001
[    6.595053] systemd[1]: System time before build time, advancing clock.
[    6.615101] random: systemd: uninitialized urandom read (16 bytes read, 112 bits of entropy available)
[    6.625682] random: systemd: uninitialized urandom read (16 bytes read, 112 bits of entropy available)
[    6.635037] random: systemd: uninitialized urandom read (16 bytes read, 112 bits of entropy available)
[    6.645354] systemd[1]: systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN)
[    6.663700] systemd[1]: Detected architecture arm64.
[    6.673413] systemd[1]: Set hostname to <tegra-ubuntu>.
[    6.691523] random: systemd: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.695042] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.695109] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.695126] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.695225] random: systemd-gpt-aut: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.695603] random: systemd-cryptse: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.697379] random: systemd-sysv-ge: uninitialized urandom read (16 bytes read, 114 bits of entropy available)
[    6.825133] systemd[1]: Listening on Journal Socket (/dev/log).
[    6.831287] systemd[1]: Reached target Swap.
[    6.835696] systemd[1]: Listening on /dev/initctl Compatibility Named Pipe.
[    6.842730] systemd[1]: Reached target Encrypted Volumes.
[    6.848260] systemd[1]: Listening on udev Control Socket.
[    6.853757] systemd[1]: Listening on Syslog Socket.
[    6.858703] systemd[1]: Reached target Remote File Systems (Pre).
[    6.864860] systemd[1]: Reached target Remote File Systems.
[    6.870598] systemd[1]: Listening on Journal Audit Socket.
[    6.876469] systemd[1]: Listening on udev Kernel Socket.
[    6.882280] systemd[1]: Created slice System Slice.
[    6.887616] systemd[1]: Created slice system-serial\x2dgetty.slice.
[    6.894187] systemd[1]: Listening on Journal Socket.
[    6.900680] systemd[1]: Mounting Debug File System...
[    6.905805] random: nonblocking pool is initialized
[    6.913769] systemd[1]: Starting Load Kernel Modules...
[    6.921384] systemd[1]: Starting Create Static Device Nodes in /dev...
[    6.930323] systemd[1]: Starting Remount Root and Kernel File Systems...
[    6.935651] EXT4-fs (mmcblk0p1): re-mounted. Opts: (null)
[    6.944687] systemd[1]: Mounting POSIX Message Queue File System...
[    6.953500] systemd[1]: Created slice User and Session Slice.
[    6.959422] systemd[1]: Reached target Slices.
[    6.965285] systemd[1]: Started Braille Device Support.
[    6.971562] systemd[1]: Created slice system-getty.slice.
[    6.977210] systemd[1]: Reached target User and Group Name Lookups.
[    6.985094] systemd[1]: Starting Journal Service...
[    6.990365] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[    7.001563] systemd[1]: Mounted Debug File System.
[    7.006713] systemd[1]: Mounted POSIX Message Queue File System.
[    7.013401] systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
[    7.023482] systemd[1]: Failed to start Load Kernel Modules.
[    7.029437] systemd[1]: systemd-modules-load.service: Unit entered failed state.
[    7.036958] systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.
[    7.045490] systemd[1]: Started Create Static Device Nodes in /dev.
[    7.052552] systemd[1]: Started Remount Root and Kernel File Systems.
[    7.060671] systemd[1]: Started Journal Service.
[    7.158527] systemd-journald[278]: Received request to flush runtime journal from PID 1
[    7.351545] xhci-tegra 3530000.xhci: cannot find firmware....retry after 1 second
[    7.624985] using random self ethernet address
[    7.629620] using random host ethernet address
[    7.643554] Mass Storage Function, version: 2009/09/11
[    7.648753] LUN: removable file: (no medium)
[    7.657844] using random self ethernet address
[    7.663007] using random host ethernet address
[    7.673405] usb0: HOST MAC 12:ae:6a:5a:a7:3f
[    7.680331] usb0: MAC da:9e:c4:e8:96:04
[    7.684545] usb1: HOST MAC 2e:d4:ec:d0:19:c5
[    7.688981] usb1: MAC 0a:9d:e7:15:52:82
[    7.692979] tegra-xudc-new 3550000.xudc: exiting ELPG
[    7.698977] tegra-xudc-new 3550000.xudc: exiting ELPG done
[    7.704508] tegra-xudc-new 3550000.xudc: ep 0 (type: 0, dir: out) enabled
[    7.711344] tegra-xudc-new 3550000.xudc: entering ELPG
[    7.716997] tegra-xudc-new 3550000.xudc: entering ELPG done
[    7.722623] tegra-xudc-new 3550000.xudc: exiting ELPG
[    7.728539] tegra-xudc-new 3550000.xudc: exiting ELPG done
[    7.734063] tegra-xudc-new 3550000.xudc: entering ELPG
[    7.739704] tegra-xudc-new 3550000.xudc: entering ELPG done
[    7.833848] device usb0 entered promiscuous mode
[    7.876568] IPv6: ADDRCONF(NETDEV_UP): usb0: link is not ready
[    7.900051] device usb1 entered promiscuous mode
[    7.910488] IPv6: ADDRCONF(NETDEV_UP): usb1: link is not ready
[    7.916861] l4tbr0: port 2(usb1) entered forwarding state
[    7.922321] l4tbr0: port 2(usb1) entered forwarding state
[    8.137934] IPVS: Creating netns size=1424 id=1
[    8.203839] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    8.214816] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    8.304579] gk20a 17000000.gp10b: railgate is disabled.
[    8.367304] xhci-tegra 3530000.xhci: Firmware timestamp: 2017-12-07 10:50:08 UTC, Version: 55.09 release
[    8.378877] xhci-tegra 3530000.xhci: xHCI Host Controller
[    8.384382] xhci-tegra 3530000.xhci: new USB bus registered, assigned bus number 1
[    8.392863] xhci-tegra 3530000.xhci: hcc params 0x0184fd25 hci version 0x100 quirks 0x00010810
[    8.401796] xhci-tegra 3530000.xhci: irq 59, io mem 0x03530000
[    8.407761] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    8.414574] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.421818] usb usb1: Product: xHCI Host Controller
[    8.426733] usb usb1: Manufacturer: Linux 4.4.38DNNCAM-HW4.4 xhci-hcd
[    8.433196] usb usb1: SerialNumber: 3530000.xhci
[    8.438128] hub 1-0:1.0: USB hub found
[    8.441933] hub 1-0:1.0: 4 ports detected
[    8.472143] xhci-tegra 3530000.xhci: xHCI Host Controller
[    8.477587] xhci-tegra 3530000.xhci: new USB bus registered, assigned bus number 2
[    8.485363] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    8.492180] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.499582] usb usb2: Product: xHCI Host Controller
[    8.504489] usb usb2: Manufacturer: Linux 4.4.38DNNCAM-HW4.4 xhci-hcd
[    8.510952] usb usb2: SerialNumber: 3530000.xhci
[    8.516140] hub 2-0:1.0: USB hub found
[    8.519925] hub 2-0:1.0: 3 ports detected
[    8.524371] tegra-xotg xotg: otg: host 3530000.xhci registered
[    8.553190] CPU1: shutdown
[    8.555900] psci: CPU1 killed.
[    8.577450] CPU2: shutdown
[    8.580232] psci: CPU2 killed.
[    8.624313] xhci-tegra 3530000.xhci: entering ELPG
[    8.632187] xhci-tegra 3530000.xhci: entering ELPG done
[    8.680156] l4tbr0: port 2(usb1) entered disabled state
[    9.804851] cfg80211: World regulatory domain updated:
[    9.810041] cfg80211:  DFS Master region: unset
[    9.814410] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp), (dfs_cac_time)
[    9.824196] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    9.832206] cfg80211:   (2457000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm), (N/A)
[    9.840300] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (N/A, 2000 mBm), (N/A)
[    9.848315] cfg80211:   (5170000 KHz - 5250000 KHz @ 80000 KHz, 160000 KHz AUTO), (N/A, 2000 mBm), (N/A)
[    9.857805] cfg80211:   (5250000 KHz - 5330000 KHz @ 80000 KHz, 160000 KHz AUTO), (N/A, 2000 mBm), (0 s)
[    9.867300] cfg80211:   (5490000 KHz - 5730000 KHz @ 160000 KHz), (N/A, 2000 mBm), (0 s)
[    9.875389] cfg80211:   (5735000 KHz - 5835000 KHz @ 80000 KHz), (N/A, 2000 mBm), (N/A)
[    9.883388] cfg80211:   (57240000 KHz - 63720000 KHz @ 2160000 KHz), (N/A, 0 mBm), (N/A)
[   11.012829] eqos 2490000.ether_qos eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   11.021717] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   11.978386] IPVS: Creating netns size=1424 id=2
[   12.610979] IPVS: Creating netns size=1424 id=3

Here is the output of egrep ‘*’ find /sys/kernel/debug/mmc2 -type f
with card removed:

/sys/kernel/debug/mmc2/speed:0
/sys/kernel/debug/mmc2/clock:0
/sys/kernel/debug/mmc2/ios:clock:		0 Hz
/sys/kernel/debug/mmc2/ios:actual clock:	375000 Hz
/sys/kernel/debug/mmc2/ios:vdd:		0 (invalid)
/sys/kernel/debug/mmc2/ios:bus mode:	2 (push-pull)
/sys/kernel/debug/mmc2/ios:chip select:	0 (don't care)
/sys/kernel/debug/mmc2/ios:power mode:	0 (off)
/sys/kernel/debug/mmc2/ios:bus width:	0 (1 bits)
/sys/kernel/debug/mmc2/ios:timing spec:	0 (legacy)
/sys/kernel/debug/mmc2/ios:signal voltage:	0 (3.30 V)
/sys/kernel/debug/mmc2/ios:driver type:	0 (driver type B)

with card inserted:

/sys/kernel/debug/mmc2/speed:0
/sys/kernel/debug/mmc2/clock:0
/sys/kernel/debug/mmc2/ios:clock:		0 Hz
/sys/kernel/debug/mmc2/ios:actual clock:	100000 Hz
/sys/kernel/debug/mmc2/ios:vdd:		0 (invalid)
/sys/kernel/debug/mmc2/ios:bus mode:	2 (push-pull)
/sys/kernel/debug/mmc2/ios:chip select:	0 (don't care)
/sys/kernel/debug/mmc2/ios:power mode:	0 (off)
/sys/kernel/debug/mmc2/ios:bus width:	0 (1 bits)
/sys/kernel/debug/mmc2/ios:timing spec:	0 (legacy)
/sys/kernel/debug/mmc2/ios:signal voltage:	0 (3.30 V)
/sys/kernel/debug/mmc2/ios:driver type:	0 (driver type B)

This is the first time I’ve seen any changes in the system when the card is inserted/removed. So that’s something.

dmesg --follow on the dev kit shows each time the card is removed. On the carrier board it does not.

head -n 1 /etc/nv_tegra_release
# R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t186ref, EABI: aarch64, DATE: Fri Mar  2 04:57:01 UTC 2018

[code]
sha1sum -c /etc/nv_tegra_release
/usr/lib/xorg/modules/extensions/libglx.so: OK
/usr/lib/xorg/modules/drivers/nvidia_drv.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libtegrav4l2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libargus_socketserver.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvomxilclient.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvidia-egl-wayland.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamerautils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvosd.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libargus_socketclient.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libargus.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtestresults.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvddk_vic.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamlog.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvparser.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_image.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_parser.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_utils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcameratools.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvimp.so: OK
/usr/lib/aarch64-linux-gnu/

4

When a card is inserted, looks like the clock frequency falls back from 375khz, 196khz, then 100khz.

970×652 45 KB

987×670 50.2 KB

986×668 54.9 KB

5

Just taking some notes. I see this which seems of interest related to the eMMC:

[    6.396219] mmc2: SDHCI controller on 3400000.sdhci [3400000.sdhci] using ADMA 64-bit with 64 bit addr
[    6.474626] EXT4-fs (mmcblk0p1): <b>8 orphan inodes deleted</b>
[    6.479957] EXT4-fs (mmcblk0p1): recovery complete
[    6.493949] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)

Is the system going down without a correct shutdown command? mmcblk0p1 is the rootfs, and apparently part of it is being lost (the orphan nodes and recovery process keeps the file system non-corrupt, but it means changes to disk which were cached but not yet committed are being lost). With some of the software not necessarily guaranteed some issues may be suspect even if they have nothing to do with lost nodes.

I’m not sure of the exact relation between XUSB and the SD card (there are some differences between a TX1 and TX2 where ethernet used to be through USB…I’m thinking perhaps SD card has a dependency on USB for the TX2, but someone from NVIDIA will have to confirm if this is correct or not). Is everything USB working correctly for you?

What is your “/proc/cmdline” content? You might adjust the extlinux.conf “APPEND” key/value pair to disable USB suspend just for testing…and verify USB is always on. You’d insert this in the APPEND of the correct boot entry (then verify it ends up in “/proc/cmdline” after reboot):

usbcore.autosuspend=-1

I find this suspicious:

/sys/kernel/debug/mmc2/clock:0

…mine shows “208000000”. Your clock is apparently not running (I can think of a number of jokes for that statement…none intended :P). Perhaps a device tree change is needed for the clock or a rail powering the clock (and of course if it depends on a bus which is in suspend state this could also do it). There are other related similar missing or “suspended” clock entries.

Someone may need to comment on why clocks would be stopped…or at least specifics of what to check for rail power and device tree. Meanwhile you might compare your extracted device tree under your board with the extracted device tree under the dev board and see if there might be a power rail or clock setting which seems suspicious. Otherwise there could be some software not kicking in as sensing the SD card being present. Since you have checked wiring my bets are on device tree (which could be either a power rail or a clock).

Do double check though if there is a case of any kind of sense for SD present detect…my bet is this is ok since you get a plug-in event noticed by dmesg.

6

One I missed which verifies the rail isn’t powered (should be “21 (3.3 ~ 3.4 V)”):

/sys/kernel/debug/mmc2/ios:vdd:		0 (invalid)
7

The orphaned nodes are because of instant power off and incorrect shutdowns on my part.

cat /proc/cmdline

root=/dev/mmcblk0p1 rw rootwait console=ttyS0,115200n8 console=tty0 OS=l4t fbcon=map:0 net.ifnames=0 memtype=0 video=tegrafb no_console_suspend=1 earlycon=uart8250,mmio32,0x03100000 nvdumper_reserved=0x2772e0000 gpt tegraid=18.1.2.0.0 tegra_keep_boot_clocks maxcpus=6 boot.slot_suffix= boot.ratchetvalues=0.2.1 androidboot.serialno=0421318044549 bl_prof_dataptr=0x10000@0x277040000 sdhci_tegra.en_boot_part_access=1 root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 usbcore.autosuspend=-1

I do not actually get a plug-in event from SDCARD_CD. It is detected on the devkit and captured in dmesg, but not the custom board, even though they are electrically the same. (SDCARD_CD) shorts to ground when card is inserted.

8

Pin 4 on the SDCard is 3.3V, so its powered, but perhaps the system doesn’t think its powered?
…also SDCARD_POWER_EN is low, and it is active-low, so that means the system has enabled power.

9

Here is the running device tree:

There doesn’t seem to be a difference between the one that is running on the dev kit and the one that is running on the custom carrier:

diff extracted.dts extracted2.dts
3529,3530c3529,3530
< 				reset-source = "MAINSWRST";
< 				reset-level = [31 00];
---
> 				reset-source = "SYS_RESET_N";
> 				reset-level = [30 00];
3534,3535c3534,3535
< 				register-value = "0x00";
< 				reason = "NIL_OR_MORE_THAN_1_BIT";
---
> 				register-value = "0x40";
> 				reason = "MBU";
10

It isn’t the SD card power which the information is pointing to…it’s the clock or something related to the clock power. Is this R28.2? What do you see in:

head -n 1 /etc/nv_tegra_release

Device tree requirements change depending on release so it isn’t possible to talk about too much detail without knowing which release is involved.

Just a thought…normally a custom board would have a lot of device tree changes. Is this board intended to be an exact match to the schematic for the dev board? If not, the lack of differences would be a big warning sign (e.g., a detect pin not wired to the same BGA pin of the SoC would need a change in device tree…e.g., different GPIO pins can be used for different purposes, but the device tree has to be set up for that…else the pin being wired might as well be a cut trace or worse).

So what comes to mind is that you have a different carrier board and barely any differences in your device tree. Each carrier should have a much different device tree. I just now booted mine without SD card and my vdd and clock also revert the same as yours. So I suspect the issue with the clock is that the device tree doesn’t set up the detect pin and the clock is disabled because it doesn’t think it needs a clock to waste power on with no SD card present (earlier I did not check the values without the SD card because I thought detect was working on your carrier).

I’ll suggest that you rename your extracted tree “extracted.dts.txt” and attach it here as a file (forum software likes “.txt” files better than “.dts”). You can’t attach to a new post, but you can attach to an existing post. If you hover the mouse over the quote icon in the upper right of your existing post you’ll see other icons show up. The paper clip icon can attach. You can skip this if the next paragraph indicates you have the wrong device tree running due to an install issue.

One warning about device tree confusion which I seem to run into daily is that in older releases it was possible to edit a dtb file name in extlinux.conf to get the device tree changes (along with placing the dtb in “/boot”). After that (anything R28.x) the device tree instead became part of a partition instead of a file (and the “dd” tool worked for dtb install if you didn’t want to use the flash tool and if the old partition had enough room). Then the device tree required being signed (starting at R28.2), and you could no longer use dd to put the dtb in place unless you know how to sign it (we’re still waiting for information on how to sign without the flash script). I doubt the Jetson would even boot if there were an unsigned tree so obviously nobody used dd to put an unsigned tree in place with R28.2+. Look very closely and see if someone tried to place the device tree in “/boot” and point at it with an “FDT” key/value pair in extlinux.conf. If “FDT” is present in extlinux.conf then this would probably be the direct cause of an invalid tree for your board (there are often extra device trees in “/boot”, but they won’t have an extlinux.conf FDT entry…those are left over baggage from other boards or previous releases and are harmless…the FDT entry is what makes it an active tree and is also a cause of failure in newer releases).

If it does turn out that there is an FDT entry, then the file it points at is probably what you need as a fix (but you’d have to delete the FDT entry and use flash.sh to install the dtb…this doesn’t guarantee the dtb is correct, but it does guarantee the source of the problem).

FYI, if you want to see the content of a dtb you can reverse compile it like this:

dtc -I dtb -O dts -o reverse_compiled.dts binary_dtb.dtb

If this is the issue then you’ll need some information on flashing the dtb (which in turn depends on the L4T release).

11

linuxdev- thanks so much for the help-
Starting at the top:

# R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t186ref, EABI: aarch64, DATE: Fri Mar  2 04:57:01 UTC 2018

This board is as close to an exact match to the dev board as possible. It is missing HDMI and SATA, and the GPIO breakout, but pretty much everything else is the same. I can actually run the stock image right out of jetpack and everything works. (Minus the SDCard).
That said, we are running a custom device tree that simply turns off the interfaces that we don’t need.

The quandary here is that when plugging the same tx2 module from our board to auvidea carrier board to the dev kit, the tx2 picks up the SDCard for the auvidea and the dev kit boards(and responds to IRQs: the card detect signal).

I’m going to look into strapping signals- I’ve seen that cause some strange errors before. We had a GPS unit tied to one of the UARTS one time and it took the mipi channels out.

Here is the extlinux.conf file:

TIMEOUT 30
DEFAULT primary

MENU TITLE p2771-0000 eMMC boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 usbcore.autosuspend=-1

Extracted DTS file attached.

Verified that the SD Card card detect signal is working… so this confirms that the card is being detected, but yet there are no events being triggered.

Card inserted:

# sudo cat /sys/kernel/debug/gpio
GPIOs 248-255, platform/max77620-gpio, max77620-gpio, can sleep:
 gpio-248 (                    |external-connection:) in  hi    
 gpio-253 (                    |spmic_gpio_input_5  ) in  lo    
 gpio-254 (                    |spmic_gpio_input_6  ) in  hi    

GPIOs 256-319, platform/c2f0000.gpio, tegra-gpio-aon:
 gpio-272 (                    |temp_alert          ) in  hi    
 gpio-312 (                    |Power               ) in  hi    
 gpio-313 (                    |Volume Up           ) in  hi    
 gpio-314 (                    |Volume Down         ) in  hi    
 gpio-315 (                    |wifi-wake-ap        ) in  lo    

GPIOs 320-511, platform/2200000.gpio, tegra-gpio:
 gpio-412 (                    |vdd-usb0-5v         ) out lo    
 gpio-413 (                    |vdd-usb1-5v         ) out lo    
 gpio-420 (                    |eqos_phy_reset      ) out hi    
 gpio-421 (                    |eqos_phy_intr       ) in  hi    
 gpio-424 (                    |wifi-enable         ) out hi    
 gpio-444 (                    |wp                  ) in  lo    
<b> gpio-445 (                    |cd                  ) in  lo   </b> 
 gpio-446 (                    |en-vdd-sd           ) out lo    
 gpio-479 (                    |external-connection:) in  hi

Card removed:

nvidia@tegra-ubuntu:~$ sudo cat /sys/kernel/debug/gpio
GPIOs 248-255, platform/max77620-gpio, max77620-gpio, can sleep:
 gpio-248 (                    |external-connection:) in  hi    
 gpio-253 (                    |spmic_gpio_input_5  ) in  lo    
 gpio-254 (                    |spmic_gpio_input_6  ) in  hi    

GPIOs 256-319, platform/c2f0000.gpio, tegra-gpio-aon:
 gpio-272 (                    |temp_alert          ) in  hi    
 gpio-312 (                    |Power               ) in  hi    
 gpio-313 (                    |Volume Up           ) in  hi    
 gpio-314 (                    |Volume Down         ) in  hi    
 gpio-315 (                    |wifi-wake-ap        ) in  lo    

GPIOs 320-511, platform/2200000.gpio, tegra-gpio:
 gpio-412 (                    |vdd-usb0-5v         ) out lo    
 gpio-413 (                    |vdd-usb1-5v         ) out lo    
 gpio-420 (                    |eqos_phy_reset      ) out hi    
 gpio-421 (                    |eqos_phy_intr       ) in  hi    
 gpio-424 (                    |wifi-enable         ) out hi    
 gpio-444 (                    |wp                  ) in  lo    
<b> gpio-445 (                    |cd                  ) in  hi </b>   
 gpio-446 (                    |en-vdd-sd           ) out lo    
 gpio-479 (                    |external-connection:) in  hi

extracted.dts.txt (306 KB)

12

Keep in mind that I don’t know details of most of the device tree changes I see, but what I am doing is comparing against my extracted device tree from R28.2 and looking at what might be related. What follows isn’t an organized answer, it’s just a random unordered list of things (in no particular order) which catch my eye. I’m poking around in the dark hoping something sticks out…someone who knows the SD card device tree requirements could probably go right to the answer.

After I post this I’ll attach a copy of the extracted device tree I have from a stock R28.2 system. You can compare to what you have and you’ll see a lot of differences.

For reference (in table 53 of OEM Product Design Guide) I see pin F17 on the module has TX2 pin name “SDCARD_CD#”, and Tegra signal name “GPIO_EDP2”. Do you have any way to verify this particular pin is actually getting toggled between 0V and 1.8V when an SD card is inserted or removed on your carrier board? If there is any sort of level shifter can you verify 1.8V is being reached?

I’m using a utility called “diffoscope” to compare, you might want to try it:

diffoscope --text-color=always my_extracted.dts your_extracted.dts | less -r

“vimdiff” is good too if you use vi…it allows going to the end of the “ahub {” curly brace and moving to the closing entry with “%”.

Was the PINMUX spreadsheet used to set up the device tree? If so, do you have a copy of the spreadsheet used to generate the tree? If not, do you have a detailed document or comments on changes to the device tree (reverse compile erases comments)?

If you don’t have the “Platform Adaptation and Bring-Up Guide” already (it’s in the “baggage/” subdirectory of the L4T docs download), then you can go here and get it:
https://developer.nvidia.com/embedded/downloads#?search=bring-up

Sometimes a power rail or clock will have its fingers into more parts of the pie than what is obvious. Something you’ve disabled for one interface may in fact have an effect on the detect of the SD card. As an example I see a difference in your device tree audio hub (“ahub {…”) power-domains versus mine. I have 0x2a, you have 0x32…I have no idea about the meaning of 0x2a or 0x32 for audio hub power-domains, so I can’t say if it really matters…but if for example the audio hub had some power source in common with the SD card’s detect then this would be a cause of the detect acting as if no card were inserted. So this is speculation.

I don’t care about the i2s differences; so far as I know audio and SD card are unrelated. However, any power bus related to both audio and SD might be of interest. Looks like your custom carrier might have audio enhancements, but I’m more or less ignoring the audio and browsing this because of power rails.

I see BPMP differences, and although I don’t know if the boot power management would have an effect on this I suspect it can. The SD card is normally available within the boot loader itself, and so power management and clocks which are part of perhaps choosing the SD card as a boot medium must work early on. I see PMC changes as well (pmc-iopower). Within this you will see “pad-names”, and “pad-names” in my version contains some sdmmc entries. I am just speculating, but there is a chance the changes to pmc might have powered off the ability to detect SD card (or at least changed which pin would be used). The data pads are impedance controlled, so different trace layout might change whether data works…but detecting the presence of an SD card wouldn’t care about impedance for a detect pin (the failure is before data access is ever attempted).

I do not see any differences in your “gpio_edp” entries.

Because U-Boot can use the extlinux.conf entry on an SD card for alternate boot media you might test something in the serial console while in the U-Boot command line. Check that this shows the same as mine:

Tegra186 (P2771-0000-500) # printenv boot_targets
boot_targets=mmc1 mmc0 usb0 pxe dhcp

Notice that mmc1 is the first target to check, and mmc1 is the SD card. Any edits to the environment which don’t use “saveenv” will be temporary and won’t stay around after boot (you can test something without it being written in stone). So if you do this and boot it’ll limit to trying to boot from SD card for a single boot (after reboot this edit will go away):

setenv boot_targets=mmc1
boot

It’ll say “MMC: no card present” if the boot loader itself does not see the card. The card could be empty or unrelated to Linux, I just want to know if the card is visible in U-Boot because the way the device tree is merged and handed down at different boot stages might mean a software problem would go away and SD could be detected. Try several different cards and see if it always says no card present.
original_extracted_dts_R28.2_4.4.38-tegra_default.dts.txt (390 KB)

13

–TEST: Check CD Signal–
“For reference (in table 53 of OEM Product Design Guide) I see pin F17 on the module has TX2 pin name “SDCARD_CD#”, and Tegra signal name “GPIO_EDP2”. Do you have any way to verify this particular pin is actually getting toggled between 0V and 1.8V when an SD card is inserted or removed on your carrier board? If there is any sort of level shifter can you verify 1.8V is being reached?”

Yes- I’ve probed it and verified that the system is actually reading a change on SDCARD_CD.

–TEST: Flash back to known standard image–
I just flashed back to standard vanilla Jetpack 3.2, 28.2 just to make sure, and still nothing. dmesg reads the same, and physically, the SD Card clock first tries at 375khz then falls back. I would chalk this up to bad layout, but the one fishy thing is that F17 (SDCARD_CD#), (GPIO_EDP2) does not generate an event (although the system can read that is toggling). I would expect it to tell me “MM2 removed” just as the other boards do.

—TEST: Bootloader see the card?–
mmc0 and mmc1 seem to be the on the system’s build-in sdhci. mmc2 is what comes up on the devkit when an sdcard is inserted into the devkit sdcard reader.
I will look via the serial console - but should it be mmc2, or mmc1?

–TEST: Strapping pins—
Out of all of the strapping pins, we are only influencing UART0_RTS- with a level converter. To be safe, I took the level converter out. Same results.

This device tree (and pinmux) tests and validation are appreciated, but do not address the fact that this TX2 module with the same device tree, when placed in other boards actually works (brings up the SDCard and has mmc2 events show in the log). To me, that says that either there is something fundamentally different in the hardware, or something about the custom board is causing the system not to look for the SD Card, and not look for the IRQ that handles the SDCARD_CD signal.

Thank you again for your amazing insights and detailed responses. Very, very much appreciated.

14

—TEST: Check /proc/interrupts for interrupts being triggered.----

CPU0       CPU3       CPU4       CPU5       
  3:          0          0          0          0     GICv2  30 Edge      arch_timer
  6:      12209          0          0          0     GICv2  32 Level     tegra186_timer0
  7:          0          0          0          0     GICv2  33 Level     tegra186_timer1
  8:          0          0          0          0     GICv2  34 Level     tegra186_timer2
  9:          0      12206          0          0     GICv2  35 Level     tegra186_timer3
 10:          0          0      10200          0     GICv2  36 Level     tegra186_timer4
 11:          0          0          0      19816     GICv2  37 Level     tegra186_timer5
 12:      11642          0          0          0     GICv2 208 Level     hsp
 13:          0          0          0          0     GICv2 202 Level     arm-smmu global fault
 14:          0          0          0          0     GICv2 203 Level     arm-smmu global fault
 22:      23268          0          0          0     GICv2  97 Level     3460000.sdhci
 23:        545          0          0          0     GICv2  96 Level     3440000.sdhci
 24:        208          0          0          0     GICv2  94 Level     3400000.sdhci
 25:          0          0          0          0     GICv2 229 Level     3507000.ahci-sata
 26:         11          0          0          0     GICv2  57 Level     3160000.i2c
 27:          0          0          0          0     GICv2  58 Level     c240000.i2c
 28:         20          0          0          0     GICv2  59 Level     3180000.i2c
 29:          0          0          0          0     GICv2  60 Level     3190000.i2c
 30:          0          0          0          0     GICv2  62 Level     31b0000.i2c
 31:          0          0          0          0     GICv2  63 Level     31c0000.i2c
 32:         42          0          0          0     GICv2  64 Level     c250000.i2c
 33:          0          0          0          0     GICv2  65 Level     31e0000.i2c
 34:          0          0          0          0     GICv2  68 Level     3210000.spi
 35:          0          0          0          0     GICv2  69 Level     c260000.spi
 36:          0          0          0          0     GICv2  71 Level     3240000.spi
 37:         46          0          0          0     GICv2 144 Level     serial
 42:        111          0          0          0     GICv2 226 Level     ether_qos.common_irq
 44:       1916          0          0          0     GICv2 222 Level     2490000.ether_qos.rx0
 45:       2714          0          0          0     GICv2 218 Level     2490000.ether_qos.tx0
 52:          0          0          0          0     GICv2  48 Level     b000000.rtcpu
 53:        169          0          0          0     GICv2 242 Level     d230000.actmon
 54:          0          0          0          0        PM  42 Level     tegra_rtc
 55:          0          0          0          0     GICv2 255 Level     mc_status
 57:          0          0          0          0     GICv2 196 Level     3538000.mailbox
 59:          0          0          0          0        PM 195 Level     xhci-hcd:usb1
 60:          0          0          0          0        PM 199 Level     3530000.xhci, xotg
 61:          0          0          0          0     GICv2 198 Level     3550000.xudc
 62:       4346          0          0          0     GICv2 297 Level     host_syncpt
 63:          0          0          0          0     GICv2 295 Level     host_status
 64:          1          0          0          0     GICv2 151 Level     150c0000.nvcsi
 65:          0          0          0          0     GICv2 233 Level     15700000.vi
 68:          0          0          0          0     GICv2 237 Level     tegra-isp-isr
 73:       2150          0          0          0     GICv2 102 Level     gk20a_stall
 74:          0          0          0          0     GICv2 103 Level     gk20a_nonstall
 76:          0          0          0          0     GICv2 315 Level     3ad0000.se_elp
 77:         54          0          0          0     GICv2 173 Level     b150000.tegra-hsp
 78:         53          0          0          0     GICv2 174 Level     b150000.tegra-hsp, b150000.tegra-hsp
 81:          0          0          0          0     GICv2 165 Level     c150000.tegra-hsp
 92:          0          0          0          0     GICv2 107 Level     gpcdma.0
 93:          0          0          0          0     GICv2 108 Level     gpcdma.1
 94:          0          0          0          0     GICv2 109 Level     gpcdma.2
 95:          0          0          0          0     GICv2 110 Level     gpcdma.3
 96:          0          0          0          0     GICv2 111 Level     gpcdma.4
 97:          0          0          0          0     GICv2 112 Level     gpcdma.5
 98:          0          0          0          0     GICv2 113 Level     gpcdma.6
 99:          0          0          0          0     GICv2 114 Level     gpcdma.7
100:          0          0          0          0     GICv2 115 Level     gpcdma.8
101:          0          0          0          0     GICv2 116 Level     gpcdma.9
102:          0          0          0          0     GICv2 117 Level     gpcdma.10
103:          0          0          0          0     GICv2 118 Level     gpcdma.11
104:          0          0          0          0     GICv2 119 Level     gpcdma.12
105:          0          0          0          0     GICv2 120 Level     gpcdma.13
106:          0          0          0          0     GICv2 121 Level     gpcdma.14
107:          0          0          0          0     GICv2 122 Level     gpcdma.15
108:          0          0          0          0     GICv2 123 Level     gpcdma.16
109:          0          0          0          0     GICv2 124 Level     gpcdma.17
110:          0          0          0          0     GICv2 125 Level     gpcdma.18
111:          0          0          0          0     GICv2 126 Level     gpcdma.19
112:          0          0          0          0     GICv2 127 Level     gpcdma.20
113:          0          0          0          0     GICv2 128 Level     gpcdma.21
114:          0          0          0          0     GICv2 129 Level     gpcdma.22
115:          0          0          0          0     GICv2 130 Level     gpcdma.23
116:          0          0          0          0     GICv2 131 Level     gpcdma.24
117:          0          0          0          0     GICv2 132 Level     gpcdma.25
118:          0          0          0          0     GICv2 133 Level     gpcdma.26
119:          0          0          0          0     GICv2 134 Level     gpcdma.27
120:          0          0          0          0     GICv2 135 Level     gpcdma.28
121:          0          0          0          0     GICv2 136 Level     gpcdma.29
122:          0          0          0          0     GICv2 137 Level     gpcdma.30
123:          0          0          0          0     GICv2 138 Level     gpcdma.31
232:          9          0          0          0  tegra-gpio 101 Level     phy_interrupt
256:          8          0          0          0  tegra-gpio 125 Edge      3400000.sdhci cd
290:          0          0          0          0  tegra-gpio 159 Edge      external-connection:extcon@1
340:          0          0          0          0  tegra-gpio-aon  16 Level     tmp451
380:          0          0          0          0  tegra-gpio-aon  56 Edge      Power
381:          0          0          0          0  tegra-gpio-aon  57 Edge      Volume Up
382:          0          0          0          0  tegra-gpio-aon  58 Edge      Volume Down
390:         54          0          0          0       GIC  32 Level     adma.0
391:         53          0          0          0       GIC  33 Level     adma.1
392:          0          0          0          0       GIC  34 Level     adma.2
393:          0          0          0          0       GIC  35 Level     adma.3
394:          0          0          0          0       GIC  36 Level     adma.4
395:          0          0          0          0       GIC  37 Level     adma.5
396:          0          0          0          0       GIC  38 Level     adma.6
397:          0          0          0          0       GIC  39 Level     adma.7
398:          0          0          0          0       GIC  40 Level     adma.8
399:          0          0          0          0       GIC  41 Level     adma.9
400:        237          0          0          0     GICv2 193 Level     snd_hda_tegra
411:          0          0          0          0       GIC  73 Edge      hwmbox1_send_empty
412:          0          0          0          0       GIC  64 Edge      hwmbox0_recv_full
413:          0          0          0          0       GIC 115 Edge      adsp watchdog
414:          0          0          0          0       GIC  94 Edge      adsp wfi
415:          0          0          0          0       GIC  89 Level     AMC error int
424:         17          0          0          0     GICv2  39 Level     30c0000.watchdog
431:          0          0          0          0        PM 241 Edge      max77620-top
435:          0          0          0          0  max77620-top   3 Edge      max77620-gpio
436:          0          0          0          0  max77620-top   4 Edge      max77686-rtc
440:          0          0          0          0  max77620-top   8 Edge      max77620-thermal
441:          0          0          0          0  max77620-top   9 Edge      max77620-thermal
442:          0          0          0          0  max77620-gpio   0 Edge      external-connection:extcon@1
450:          0          0          0          0  max77686-rtc   1 Edge      rtc-alarm1
IPI0:     12792      20040      20444      37386       Rescheduling interrupts
IPI1:        35         28         26         34       Function call interrupts
IPI2:         0          0          0          0       CPU stop interrupts
IPI3:         0          0          0          0       Timer broadcast interrupts
IPI4:       758        508        340        492       IRQ work interrupts

–RESULT: IRQ256 increases every time the card is inserted or extracted.–
–CONCLUSION: The system is receiving the SDCARD_CD IRQ, but not doing anything with it.

15

I forgot to attach the actual dts I had extracted, it’s being attached to my previous reply.

“I will look via the serial console - but should it be mmc2, or mmc1?”
The SD card shows as “/dev/mmcblk1*” for block devices, but the controller would be mmc2 in dmesg (I think sometimes it is a case of 0-based and 1-based numbering, but I’m not entirely sure where naming is from). In the OEM product guide it refers to VDDIO_SDMMC1_AP (I believe zero-based). In dmesg it refers to mmc2 (1-based), and in this example, when I eject an SD card dmesg shows mmc2:

[  246.590188] mmc2: card 59b4 removed
[  246.590188] mmc2: card 59b4 removed
[  246.645588] sdhci-tegra 3400000.sdhci: Tuning already done, restoring the best tap value : 28

…you have to differentiate between whether we are talking about the module or the carrier…and I have no idea how to guarantee the right one is being used without actually testing it.

Incidentally, I just realized there is a clue there. It’s the address from “3400000.sdhci”. After I eject and then add the SD card back in, this is the attach message:

[  318.222855] mmc2 tuning done saved tap delay=28
[  318.227381] mmc2: hw tuning done ...
[  318.230952] mmc2: tuning_window[0]: 0xfffffc1f
[  318.235389] mmc2: tuning_window[1]: 0xffe0ffff
[  318.239827] mmc2: tuning_window[2]: 0x87ffffff
[  318.244263] mmc2: tuning_window[3]: 0x7fffffff
[  318.248699] mmc2: tuning_window[4]: 0x0
[  318.252528] mmc2: tuning_window[5]: 0x0
[  318.256356] mmc2: tuning_window[6]: 0x0
[  318.260184] mmc2: tuning_window[7]: 0x0
[  318.264010] sdhci: Tap value: 28 | Trim value: 5
[  318.268617] sdhci: SDMMC_VENDOR_INTR_STATUS[0x108]: 0x40000
[  318.274209] mmc2: new ultra high speed SDR104 SDHC card at address 59b4
[  318.283043] mmcblk1: mmc2:59b4 SD    14.7 GiB 
[  318.297810]  mmcblk1: p1

Again, mmc2 is the controller, but it doesn’t mention “sdhci-tegra 3400000.sdhci”. In the device tree I see:

# within aliases...
sdhci0 = "/sdhci@3400000";
...
# It's own block...the alias would be the same as this:
sdhci@3400000 {

Within the “sdhci@3400000” block you have some differences. I do not know how those differences may or may not matter. Is the PINMUX spreadsheet used for this available?

It is a good point that this works with the same tree on the dev board, and if the wiring were a match, then in theory it should work. Having the interfaces re-enabled by going back to the stock device tree adds evidence in favor of that, but I think that isn’t entirely conclusive. The state of some hardware depends on what is detected during boot. Something which is intentionally missing may have a side effect regardless of whether it is missing in device tree or missing through lack of detect during boot. If it is a case of failing to detect something, rather than hardware omission, then there would be a software means to fix it.

You have been able to guarantee the detect pin itself changes level correctly and that with both boards being a near identical layout and using identical device trees detect never shows up on the custom carrier, and yet it shows up on the dev carrier. Toggling that pin should be enough unless something in software is telling it to ignore the pin. Device tree is almost always the culprit when correct layout results in hardware being ignored. The trouble is that I haven’t worked with SD card enough to say, but there are a number of components (such as bluetooth) interrelated.

Incidentally, I am curious about how the clock initially starts up and then backs off. Is this correct for the custom carrier? If so, are you able to monitor the data lines for activity as this occurs? Can you compare to what shows up in a working dev carrier? Perhaps it is looking for a data response before it marks that something was detected in dmesg. Some sort of handshake maybe. Look for any difference between the custom board and the dev board at the moment the detect pin asserts.

16

Another note. Notice the “59b4” ID in the logs when a card is inserted or removed. If you look in “/sys/devices/3400000.sdhci/mmc_host/mmc2/” you will find the existence of a specific subdirectory depends on whether the card is detected:

/sys/devices/3400000.sdhci/mmc_host/mmc2/mmc2:59b4

…the kernel definitely thinks of it as mmc2.

On your carrier does this directory show up with the card inserted?

17

Sadly, no.

root@tegra-ubuntu:/sys/devices/3400000.sdhci/mmc_host/mmc2# ls
device  power  subsystem  uevent
18

You mentioned watching the clock. Are you able to see this on a scope, and if so, is this pin 5 of the SD card? Are you able to simultaneously scope pin 2, the SD card command lane, and perhaps pin 7, the data#0 lane?

Can you compare the actual voltage delivered to the SD card pin 4 (power) when the SD is not connected on both the dev board and custom board to see if they start at the same voltage?

EDIT: How much direct access do you have with a scope to the SD card pins?

19

According to the SD card protocols once an attempt to detect is triggered the clock will try to strobe and sends 48 bit commands. Commands go over pin 2 synchronously, so the clock itself should run in time with activity on pin 2.

If commands succeed the initial 3.3V interface can be changed to 1.8V…so I’m curious if the power is correctly starting at 3.3V. If it does drop to 1.8V be sure it actually reaches 1.8V and not something out of spec.

20

What devices show up in “/sys/bus/mmc/devices/*”? In addition to the SD card itself (“mmc2:59b4”) I see:

mmc0:0001  mmc1:0001

…if either of those are missing, then whatever is stopping card recognition is probably the same reason as the missing SD card. If this is the case, then we might be able to track down what was done to disable those other devices and make it possible for mmc2:59b4 to also be recognized.

EDIT: Does this change from “0” to “1” with the card inserted (I’m guessing “no”, but want to check):

cat /sys/kernel/debug/340000.sdhci/card_insert