Ubuntu automatic login OK but crash on mmc2: Timeout waiting for hardware interrupt.

  1. Flash OS with JetPack failed, I tried many times. Finally flashed successfully with “uncommenting the board id in jetson-tk1.conf file”
  2. Now I can enter U-Boot menu and everything looks great, but right after the login screen shows up the monitor go black
  3. below is the message I captured from the serial port with a usb-to-serial cable

thanks for any tips and help

--------------------------Serial console message below-----------------------

Ubuntu 14.04.1 LTS tegra-ubuntu ttyS0

tegra-ubuntu login: ubuntu (automatic login)

Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.10.40-gdacac96 armv7l)

0 packages can be updated.
0 updates are security updates.

ubuntu@tegra-ubuntu:~$ [ 18.330625] mmc2: Timeout waiting for hardware interrupt.
[ 18.336020] sdhci: =========== REGISTER DUMP (mmc2)===========
[ 18.341846] sdhci: Sys addr: 0x00000000 | Version: 0x00000303
[ 18.347669] sdhci: Blk size: 0x00000000 | Blk cnt: 0x00000000
[ 18.353494] sdhci: Argument: 0x00000c00 | Trn mode: 0x00000000
[ 18.359316] sdhci: Present: 0x01fb00f1 | Host ctl: 0x00000001
[ 18.365138] sdhci: Power: 0x0000000f | Blk gap: 0x00000000
[ 18.370961] sdhci: Wake-up: 0x00000000 | Clock: 0x00000405
[ 18.376784] sdhci: Timeout: 0x00000000 | Int stat: 0x00000000
[ 18.382607] sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00fc0003
[ 18.388429] sdhci: AC12 err: 0x00000000 | Slot int: 0x00000000
[ 18.394252] sdhci: Caps: 0x376fd080 | Caps_1: 0x10002f73
[ 18.400074] sdhci: Cmd: 0x0000341a | Max curr: 0x00000000
[ 18.405892] sdhci: Host ctl2: 0x00003000
[ 18.409805] sdhci: ADMA Err: 0x00000000 | ADMA Ptr: 0x00000000
[ 18.415624] sdhci: ===========================================

Is this custom hardware? If this a Jetson TK1 it should never ask you to uncomment the board ID.

Regardless, if there is any attempt to add the L4T apply_binaries.sh changes, then later changing one of the files it installed could cause video failure. You can check individual files via:

sha1sum -c /etc/nv_tegra_release

(e.g., via ssh or serial console if video fails)

Was this issue ever addressed? I am getting this same exact error before my system becomes unresponsive due to the fact that my OS is running on internal mmc.

130|shell@armv7_cortex_a:/ $ [   37.878633] mmc2: Timeout waiting for hardware interrupt.
[   37.884023] sdhci: ================== REGISTER DUMP (mmc2)==================
[   37.891059] sdhci: Sys addr[0x000]: 0x00000000 | Version[0x0fe]:  0x00000303
[   37.898095] sdhci: Blk size[0x004]: 0x00000000 | Blk cnt[0x006]:  0x00000000
[   37.905128] sdhci: Argument[0x008]: 0x00000c00 | Trn mode[0x00c]: 0x00000000
[   37.912159] sdhci: Present[0x024]:  0x01fb00f1 | Host ctl[0x028]: 0x00000001
[   37.919191] sdhci: Power[0x029]:    0x0000000f | Blk gap[0x02a]:  0x00000000
[   37.926222] sdhci: Wake-up[0x02b]:  0x00000000 | Clock[0x02c]:    0x00000405
[   37.933253] sdhci: Timeout[0x02e]:  0x00000000 | Int stat[0x030]: 0x00000000
[   37.940284] sdhci: Int enab[0x034]: 0x00ff0003 | Sig enab[0x038]: 0x00fc0003
[   37.947315] sdhci: AC12 err[0x03c]: 0x00000000 | Slot int[0x0fc]: 0x00000000
[   37.954346] sdhci: Caps[0x040]:     0x376fd080 | Caps_1[0x044]:   0x10002f73
[   37.961378] sdhci: Cmd[0x00e]:      0x0000341a | Max curr[0x048]: 0x00000000
[   37.968406] sdhci: Host ctl2[0x03e]: 0x00003000
[   37.972924] sdhci: ADMA Err[0x054]: 0x00000000 | ADMA Ptr[0x058]: 0x00000000
[   37.979953] sdhci: Tap value: 0 | Trim value: 2
[   37.984469] sdhci: SDMMC Interrupt status: 0x00000000
[   37.989503] sdhci: =========================================================

Full log on pastebin: http://pastebin.com/6Peqtm45

I was thinking it was my DTB file because of all the dt errors but I am not totally sure.

I am using kernel 3.10.67 modified to run on Jetson TK1 (android) using GCC arm-linux-gnueabi-4.9-linaro

Normally eMMC corresponds to mmc0, and SD card corresponds to mmc1. More information on what mmc2 is for your boards would help.

Board is the standard Jetson TK1 dev kit.

There may be a legitimate mmc2 on a stock Jetson TK1, but I’ve not seen reference to it. I’m wondering if part of the flash went to memory in mmc0 which is corrupt/failing. There may be something more to test, but it is possible it is time for RMA. The one thing I can think of is to verify if the “bootloader/system.img.raw” produced in the flash is valid. Can you check for the exact byte size of that file? Also, are you able to successfully mount this via:

sudo mount -o loop system.img.raw /mnt

Finally, on the host where flash was performed, is there still free space? E.g.:

df -H

No need for an RMA. Im sure there is an mmc2 in the upstream 3.10.67 kernel our jetson tk1 doesnt use. Possibly a sdhci wifi device?

I dont see a log for mmc2 on 3.10.40

$ dmesg|grep mmc2
/ $ dmesg|grep mmc                                              
<6>[    0.000000] Lowering sdmmc1 maximum rate from 208000000 to 204000000
<6>[    0.000000] Lowering sdmmc3 maximum rate from 208000000 to 204000000
<6>[    1.015051] vddio-sdmmc-2: 1800 <--> 3300 mV at 150 mA 
<6>[    1.126408] sdmmc-en-supply: 3300 mV 
<6>[    8.699769] mmc0: no vqmmc regulator found
<6>[    8.703894] mmc0: no vmmc regulator found
<6>[    8.707912] mmc0: Invalid maximum block size, assuming 512 bytes
<6>[    8.747458] mmc0: SDHCI controller on sdhci-tegra.3 [sdhci-tegra.3] using ADMA
<6>[    8.770498] mmc1: no vqmmc regulator found
<6>[    8.775781] mmc1: no vmmc regulator found
<6>[    8.779927] mmc1: Invalid maximum block size, assuming 512 bytes
<6>[    8.819460] mmc1: SDHCI controller on sdhci-tegra.2 [sdhci-tegra.2] using ADMA
<6>[    8.819665] mmc0: BKOPS_EN bit is not set
<6>[    9.371723] mmc0: new HS200 MMC card at address 0001
<6>[    9.371948] mmcblk mmc0:0001: Card claimed for testing.
<6>[    9.372151] mmcblk0: mmc0:0001 SEM16G 14.6 GiB 
<6>[    9.372310] mmcblk0rpmb: mmc0:0001 SEM16G partition 3 4.00 MiB
<6>[    9.374850]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
<4>[    9.809947]    sdmmc4_ddr
<4>[    9.812580]    sdmmc3_ddr
<6>[   10.183383] vddio-sdmmc-2: incomplete constraints, leaving on
<6>[   12.476463] EXT4-fs (mmcblk0p4): mounted filesystem with ordered data mode. Opts: (null)
<7>[   12.484557] SELinux: initialized (dev mmcblk0p4, type ext4), uses xattr
<6>[   12.498356] EXT4-fs (mmcblk0p5): mounted filesystem with ordered data mode. Opts: (null)
<7>[   12.506450] SELinux: initialized (dev mmcblk0p5, type ext4), uses xattr
<6>[   12.521643] EXT4-fs (mmcblk0p6): mounted filesystem with writeback data mode. Opts: data=writeback,nodelalloc
<7>[   12.531557] SELinux: initialized (dev mmcblk0p6, type ext4), uses xattr
<4>[   12.541893] EXT4-fs (mmcblk0p11): Ignoring removed nomblk_io_submit option
<4>[   12.549172] EXT4-fs (mmcblk0p11): bad geometry: block count 3195904 exceeds size of device (3194880 blocks)
<4>[   12.653122] EXT4-fs (mmcblk0p11): bad geometry: block count 3195904 exceeds size of device (3194880 blocks)
<6>[   32.120075] EXT4-fs (mmcblk0p10): mounted filesystem with ordered data mode. Opts: 
<7>[   32.127772] SELinux: initialized (dev mmcblk0p10, type ext4), uses mountpoint labeling

It’s odd to see mmc2 on a standard Jetson even if the kernel differs. It’s something you’ll want to post the config for (or how the config differs from the default kernel), as well as whether you used a dtb from the original kernel or from kernel build.

Here is my defconfig on pastbin: http://pastebin.com/mYZy3Hus i have compared both defconfigs and voltage defconfigs from 3.10.40 and my configs should work. Ill git compare them later…

If I build the kernel without CONFIG_MMC_SDHCI_TEGRA=y

everything runs fine so the lockup must be a mmc voltage issue or a node being created for tegra.sdhci.0 mmc2 which jetson doesnt have and its in the dtb/dts includes.

Not sure how to fix this issue on 3.10.67. I wouldn’t know where to start…

Can I enable more debugging output on the serial console somehow for more detailed information on whats causing the mmc2 lockup?

Have you updated the dtb file in “/boot” (or preferably added an alternate entry while leaving the original alone)? Also, are you cross-compiling or native compiling?

I am using the DTB built with kernel, I am cross compiling with linaro gcc 4.9. I started a separate topic due to the fact this topic is ubuntu and I am running android. Here is the other topic: https://devtalk.nvidia.com/default/topic/954715/jetson-tk1/mmc2-timeout-waiting-for-hardware-interrupt-/