TX1 Move Filesystem to SD, L4T R32 Rev 2.3

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi all,

I have been enjoying trying to move the filesystem of my TX1 from my eMMC to an SD card for a while now to no avail. I have been following both the Jetson Hacks guide and also this topic https://devtalk.nvidia.com/default/topic/974861/jetson-tx1/exact-steps-to-extend-move-file-system-root-on-tx1-to-use-vnand-m-2-on-j120/post/5319373/#5319373 (mainly because it was recent), but it seems they’re both working for L4T builds prior to 28.1.

I know that some extra steps occur during boot for L4T >28.1 that make the simple modification of the /boot/extlinux/extlinux.conf file to point to an external device as root not a viable method any more, so I would love to know the method for moving the root filesystem over to an SD device. In my case: /dev/mmcblkp2.

System Info
lsblk output:

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 16M 1 loop
mmcblk0 179:0 0 14.7G 0 disk
├─mmcblk0p1 179:1 0 14G 0 part /
├─mmcblk0p2 179:2 0 2M 0 part
├─mmcblk0p3 179:3 0 4M 0 part
├─mmcblk0p4 179:4 0 2M 0 part
├─mmcblk0p5 179:5 0 6M 0 part
├─mmcblk0p6 179:6 0 4M 0 part
├─mmcblk0p7 179:7 0 6M 0 part
├─mmcblk0p8 179:8 0 2M 0 part
├─mmcblk0p9 179:9 0 2M 0 part
├─mmcblk0p10 179:10 0 128M 0 part
├─mmcblk0p11 179:11 0 20M 0 part
├─mmcblk0p12 179:12 0 64M 0 part
├─mmcblk0p13 179:13 0 64M 0 part
├─mmcblk0p14 179:14 0 4M 0 part
├─mmcblk0p15 179:15 0 2M 0 part
├─mmcblk0p16 179:16 0 6M 0 part
├─mmcblk0p17 179:17 0 6M 0 part
├─mmcblk0p18 179:18 0 2M 0 part
└─mmcblk0p19 179:19 0 368M 0 part
mmcblk0boot0 179:32 0 4M 1 disk
mmcblk0boot1 179:64 0 4M 1 disk
mmcblk0rpmb 179:96 0 4M 0 disk
mmcblk2 179:128 0 59.8G 0 disk
└─mmcblk2p1 179:129 0 59.8G 0 part /media/nvidia/tx1_sd
zram0 252:0 0 494.5M 0 disk [SWAP]
zram1 252:1 0 494.5M 0 disk [SWAP]
zram2 252:2 0 494.5M 0 disk [SWAP]
zram3 252:3 0 494.5M 0 disk [SWAP]

head -n 1 /etc/nv_tegra_release output:

R32 (release), REVISION: 2.3, GCID: 17644089, BOARD: t210ref, EABI: aarch64, DATE: Tue Nov 5 21:56:03 UTC 2019

Method 1
So, I’ve tried the copying the rootfs to my sd card with the following following the above topic’s accepted answer:

  1. sudo umount /dev/mmcblk2p1
  2. sudo mkfs.ext4 /dev/mmcblk2p1 -L tx1_sd
  3. sudo rsync -avxHAX --progress / /media/nvidia/tx1_sd/
  4. sudo vim /boot/extlinux/extlinux.conf
  5. Modify /boot/extlinux/extlinux.conf to:

TIMEOUT 30
DEFAULT primary

MENU TITLE L4T boot options

LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
INITRD /boot/initrd
FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
APPEND fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=${lp0_vec} nvdumper_reserved=${nvdumper_reserved} core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 net.ifnames=0 root=/dev/mmcblk2p1 rw rootwait

LABEL backup
MENU LABEL backup kernel
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} quiet

  1. sync
  2. sudo shutdown -r now

No success on that one, the system boots and then uses the filesystem on the eMMC. One side question about the above, would having the backup label there be useful at all, say if the SD card wasn’t inserted, then the jetson would boot on the backup label and load the eMMC?

I’ve also tried a clean install of the rootfs using the first method of the accepted answer in the linked post above. I got the same outcome so my method for specifying the root filesystem during boot must be bunk.

So, any better posts out there that have this problem fixed? Or perhaps this topic can serve as a sign-post for everybody fumbling around with the post 28.1 L4T releases.

linuxdev, I know you know something :)

Cheers.

2

Really sorry for late reply. Are you trying to mount rootfs on rel-32 + TX1?

3

Hi WayneWWW, yes my original intention was to clone rootfs onto /dev/mmcblk2p1 (rel-32, TX1), but still perform the boot from /dev/mmcblk0p1 like in rel<=28.1. I understand that procedure is no longer viable and was wondering how to mount rootfs on an external drive (in my case an SD card that points to /dev/mmcblk2).

I have seen a post in TX2 support to mount certain parts of the rootfs (/usr/local in the case of https://devtalk.nvidia.com/default/topic/1067987/jetson-tx2/memory-issue-in-jetson-tx2/post/5410765/#5410765) onto partitions on an external drive, but I was hoping for a more complete move of the entire rootfs like in the old days.

4

Please refer to this post. The new method starts from rel-32.

https://devtalk.nvidia.com/default/topic/1066556/jetson-agx-xavier/how-to-use-rootfs-in-nvme-ssd-/post/5401782

5

Thanks WayneWWW, I’ll take a look at this in the new year and get back to you before Jan-12-2020. Cheers.

6

Hi again,

So, I followed the instructions provided in https://devtalk.nvidia.com/default/topic/1066556/jetson-agx-xavier/how-to-use-rootfs-in-nvme-ssd-/post/5401782, that link to the post https://devtalk.nvidia.com/default/topic/1060950/jetson-tx2/how-can-we-flash-everything-apart-bootloader-to-extern-sdcard-on-tx2-like-jetson-nano-with-no-intern-/post/5373638/#5373638. But the tx1 is unable to find the SD card to mount rootfs during boot.

To reproduce what I did:

  1. Mounted 64 GB SD card on host machine and formatted the space with an ext4 partition.
  2. Installed Jetpack 4.3 (L4T 32 RE-3.1) to host machine.
  3. Copied rootfs folder to SD card. ``` sudo mount /dev/mmcblk0p1 /mnt/sd sudo cp -r ./rootfs /mnt/sd/. ```
  4. Got the PARTUUID of my SD card partition: ``` sudo blkid /dev/mmcblk0p1 /dev/mmcblk0p1: LABEL="Linux filesystem" UUID="e7e6dcd2-c38e-4805-a853-200efb7f99fc" TYPE="ext4" PARTUUID="b5af891d-01" ```
  5. Created the l4t-rootfs-uuid.txt file in ./bootloader. ``` echo 'e7e6dcd2-c38e-4805-a853-200efb7f99fc' > ./bootloader/l4t-rootfs-uuid.txt ```
  6. Put the tx1 in forced recovery mode and flashed it. ``` sudo ./flash.sh jetson-tx1 external ```

The flashing was successful, and I removed the SD card from my host machine, inserted it into the dev carrier board’s SD card reader and rebooted the tx1. The tx1 then repeats the following about 30 or so times before falling back to a bash-4.38 shell:

mount: can't find PARTUUID=e7e6dcd2-c38e-4805-a853-200efb7f99fc
#above repeats
/mnt/ is not a mountpoint
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

EDIT: I don’t know why the tx1 complains that it cannot find the PARTUUID, I thought that it should be looking for the UUID of the partition, which is a different value.

I’ve double-checked the PARTUUID UUID value for my SD card and can confirm that it is e7e6dcd2-c38e-4805-a853-200efb7f99fc on my host machine. lsblk and other useful commands don’t work on the bash-4.38 shell that the tx1 boots into. dmesg seems to indicate that the SD card was detected and is listed on /dev as mmcblk2p1.

I’m wondering what I’ve missed here. Is the tx1 different from the tx2 in some way for SD card boot? Perhaps I should copy the contents of rootfs to the SD card rather than the folder. Any suggestions?

1 Like
7

I have not tried this method of booting from SD, but here are some comments.

You might want to limit your copy to one file system. “/proc” and “/sys” (and sometimes “/dev”) are not necessarily “real” files, but are instead part of kernel drivers in RAM. Also, you want to preserve all permissions. So something like “sudo cp -axr …”. If you choose this method.

On the other hand, your file system may be changing during the copy and thus end up not being a valid copy. You might want to instead clone the rootfs while in recovery mode. The clone is a bit-for-bit exact copy of the file system…this is a true duplicate of the partition. Then something like “dd” can be used to copy this to the partition of the SD card.

If you’ve ever flashed, then your host PC will have a subdirectory “Linux_for_Tegra/”. Within this is “flash.sh”. A clone (with the micro-B USB connected, and Jetson in recovery mode) would be something like this (depending on release to some extent…earlier releases will differ, some releases need a patch to “flash.sh”):

sudo ./flash.sh -r -k APP -G my_backup.img jetson-tx1 mmcblk0p1

Note that the above will provide “my_backup.img” and also “my_backup.img.raw”. The raw image is larger, and this can be loopback mounted, examined, edited, so on. The smaller image is a “sparse” image and can only be used for flash (I tend to delete the smaller image…the large image is quite large, and I run bzip -9 on this when not using it…which takes a very long time). The raw image will be around 16GB on a TX1.

You will want to find the exact size of the raw image (which will be an even multiple of either 10241024n, or 102410241024*n). You have to create an ext4 partition on your SD card as the first partition which is the same size. Then dd to this first partition. If you install to the SD card as a whole, then this is not a partition. Assuming the first partition shows up on your host PC (not mounted) as “/dev/sdb1”, then it would go like this (implying you already created the partition on the SD card):

dd if=/where/ever/it/is/my_backup.img.raw /dev/sdb1 bs=1M
# Prior to removing the SD card you will want to "sudo sync /dev/sdb"...adjust for actual SD card name.

(the “bs=1M” just buffers one MB for faster operation)

If this succeeds and you want to change the size of the partition, then you can just use something like “gparted” to extend the partition.

You will also want to verify IDs via “lsblk -f /dev/sdb” (assumes your SD card is “/dev/sdb”). Keep in mind you are interested in the partition UID, not the disk ID. If you run “sudo gdisk -l /dev/sdb”, then you’ll see the disk ID. Note that “lsblk -f” shows every partition’s ID.

Don’t know if that is enough, but this would make a valid image. Depending on release you may also need to edit the SD card image’s “/boot/extlinux/extlinux.conf” (the “root=” part would need to point at “/dev/mmcblk1p1”). You can loopback mount the image and edit that file there, or you can edit once it is on the SD card.

8

Hi linuxdev,

For reference, I’m following the instructions in this post and adapting it for the latest L4T (32 rev-3.1 at the moment):https://devtalk.nvidia.com/default/topic/1060950/jetson-tx2/how-can-we-flash-everything-apart-bootloader-to-extern-sdcard-on-tx2-like-jetson-nano-with-no-intern-/post/5373638/#5373638.

The idea is that rootfs, located in the …/nvidia/nvidia_sdk/JetPack_4.3_Linux_P2180/Linux_for_Tegra directory on my host machine, is copied to an ext4 partition on an SD card. I’ll reference my commands from that directory from now on. As far as I know, this rootfs that I’m copying to the SD card from my host machine shouldn’t be changing at all while on my host machine, right?

I reformatted the SD card with an ext4 partition in gparted on my host machine and found the UUID:

lsblk -f /dev/mmcblk0p1
NAME      FSTYPE LABEL UUID                                 MOUNTPOINT
mmcblk0p1 ext4         bd46aeaa-6556-497c-8179-8843237b95f3

Then I copied the rootfs over to the new partition as linuxdev has suggested:

sudo mount /dev/mmcblk0p1 /mnt/sd
sudo cp -axr ./rootfs/* /mnt/sd/
sudo umount /dev/mmcblk0p1

That took some time. Now I write the UUID found in the command above to ./bootloader/l4t-rootfs-uuid.txt:

echo 'bd46aeaa-6556-497c-8179-8843237b95f3' > ./bootloader/l4t-rootfs-uuid.txt

Now I’m ready to flash to the TX1. Put it in recovery mode and run the following on my host machine:

sudo ./flash.sh jetson-tx1 external

I’ve attached the output from flash.sh in “flash_dump.txt” https://pastebin.com/7x2w2wHC

The flashing process didn’t take very long. I turned off the TX1, inserted the SD card into the devkit carrier board, connected a serial cable and booted it up. The serial output is attached in “bootup.txt” https://pastebin.com/FB0aJrud

During the boot I had a HDMI display connected too, it outputted something funny:

mount: can't find PARTUUID=bd46aeaa-6556-497c-8179-8843237b95f3
**REPEAT THE ABOVE ABOUT 30 TIMES**
/mnt/ is not a mountpoint
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
bash-4.38

Not so sure about the last few instructions concerning the backup_image and dd. Could you run me through how to get such a file?

I did notice that in flash.sh there is a viable argument for rootdev called “mmcblk1p1” which flags the tx1 to mount the SD card for rootfs. However, that appears to not work for devkit carrier, the SD card is mounted on /dev/mmcblk2p1, mmcblk1 must be reserved for some reason…

Cheers.

9

I have not tried the “external” target before (the part which is new to later releases).

One comment prior to saying more: During a normal flash the “rootfs/” content has edits to the “rootfs/boot/” area. This is one reason why options during flash change the image and may edit the boot content of the “rootfs/”. This is also one reason why the rootfs of an earlier and later release are often not compatible. I am unsure of how “external” works with this, but if you copied your SD card image prior to running the flash, or if you flashed before copying the UUID in, then perhaps the image itself is missing something. Right now I lack an SD card for this, so I cannot test.

One other option for copy of the image is to dd the “bootloader/system.img.raw” to the SD card’s first partition instead of individual file copy, but this would not change the partition ID error issue. “dd” would guarantee a bit-for-bit exact match, and would also guarantee any boot content edit made its way in to the SD card rootfs. I do not believe this is the problem though.

One immediate thought of a possible simple bug is that perhaps the method of testing for partition UID is case sensitive and looking for upper case (hex is without case, but the matching might not have accounted for this)…don’t know, and no SD card so it isn’t possible for me to check. Otherwise, the log indicates an exact match to the partition ID.

You are correct that SD card in earlier releases should be “/dev/mmcblk1”, and the rootfs “/dev/mmcblk1p1”. I do not know the meaning of the mmcblk2 change, but if an SD card is used in a non-boot capacity, then this is the new device number (I cannot verify if the device number change exists with an SD card as boot device).

Moving on to the U-Boot log, it seems the partition was found and extlinux.conf was retrieved (this contradicts the filesystem missing partition UUID issue):

mmc1 is current device
Scanning mmc 1:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf

We know for a fact that up to this point not only was a filesystem with extlinux.conf found (though perhaps not via partition UID), we also know that the file could be read. The initrd is also read. There is mention of the device tree being found and read, but this tree is no longer used as it used to be. Perhaps there is a conflict in the device tree?

In the flash logs this device tree is listed as flashed (several lines show copy and signing of this):

tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb

The dtb should still be present on the host PC. Location should be:

/home/owro/nvidia/nvidia_sdk/JetPack_4.3_Linux_P2180/Linux_for_Tegra/kernel/dtb/<b>tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb</b>

If you run this command you should be able to convert from binary format to source format:

cd /home/owro/nvidia/nvidia_sdk/JetPack_4.3_Linux_P2180/Linux_for_Tegra/kernel/dtb/
dtc -I dtb -O dts -o extracted_tegra210-jetson-tx1-p2597-2180-a01-devkit.dts ./tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb

You could rename this with a “.txt” extension if you like. If you can, attach this file. I am curious if the “chosen” node conflicts (or for that matter, where the UID is stored).

I don’t know if the initrd can be attached here or not, but if possible, perhaps attach this file:

/home/owro/nvidia/nvidia_sdk/JetPack_4.3_Linux_P2180/Linux_for_Tegra/bootloader/l4t_initrd.img

If the forum does not want to attach the “.img”, then perhaps you could gzip this and it would accept the “.gz” file (or a “.zip”).

For anyone at NVIDIA, can you verify if this U-Boot message, “Scanning mmc 1:1…Found /boot/extlinux/extlinux.conf”, is the SD card content and not some other content? Of course if the case is that this is not the SD card, then all of the above is a result of old content on the eMMC being found and never looking at the SD card.

10

SUCCESS.

Looks like the problem resided within the formatting of the SD Card. carolyuu suggested the following:

The key here was the GPT partition table specification. I was using gparted and the default table format I think was msdos, which resulted in only a UUID being generated for the subsequent partition and no usable PARTUUID (which is the actual key needed by the booting process!). Creating a new GPT partition table in gparted before creating the ext4 partition gave me the information I needed:

sudo blkid /dev/mmcblk0p1
/dev/mmcblk0p1: LABEL="TX1ROOTFS" UUID="69e5a285-cb29-4f62-89a5-9f60f53cd823" TYPE="ext4" PARTUUID="5da6675d-1fcc-45ae-9faa-1d6578be093"

Writing the PARTUUID for mmcblk0p1 to the l4t-rootfs-uuid.txt file in ./bootloader. Then I’m good to copy the rootfs directory to the partition.

echo '5da6675d-1fcc-45ae-9faa-1d6578be093' > ./bootloader/l4t-rootfs-uuid.txt
sudo mount /dev/mmcblk0p1 /mnt/sd
sudo cp -axr ./rootfs/* /mnt/sd/.

Then put the TX1 in recovery mode and flash.

sudo ./flash.sh jetson-tx1 external

Then, I unmounted the SD card from my host machine, inserted on the TX1 dev carrier board, and rebooted the TX1. The TX1 successfully booted with the SD card set to rootfs:

lsblk /dev/mmcblk2p1
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk2p1   179:129  0 59.8G  0 part /

Thanks a lot for your time and help!

11

On my Xavier, I mounted /usr, /home, /tmp, and /var/lib/docker on an ssd. All was working fine for months, even after updates but this latest update seems to have left my system in an unbootable state:

bash: cannot set terminal process group (-1): Inappropriate ioctl for device

It’s kinda a shame to have an nvme ssd slot and not be able to use it for anything. I figured I was safe with just those few partitions copied over, but i guess not. At least I can pull the ssd, stick it in a reader, and recover my /home.

Edit: per discussion below, this issue seems unrelated ot NVMe.

12

This implies a driver changed. The whole idea that “in *NIX everything is a file” is only for standard functions which might occur with a file. When the IOCTL comes into play, then it is something custom for that driver, and the inappropriate IOCTL implies that if you have the right driver in place, then it might work again (or conversely, something using the driver could have changed and depend on a different driver). Don’t know what it is, but if you can reproduce it on command line, then “strace” could point out which specific call failed (which in turn could show which driver or which software called the driver).

1 Like
13

Thanks for the insight on the error, @linuxdev. The only custom driver on the system is for Coral, but I tried removing that first and it still didn’t boot. It’s a DKMS driver, so it’s supposed to build with every new kernel, but if the NVidia kernel packages are wildly different from Canonicals, perhaps that has something to do with it. I did see something about it finishing the kernel update in the background after the last apt-upgrade but I didn’t take note of it.

Do you know of any way I can modify the kernel command line in this non-bootable state so I can try and blocklist the edgetpu module? It shouldn’t be loading since the hardware is not installed but I can’t think of anything else at the moment. It’s not a tragedy or anything since I can just reflash and my $HOME and such will still be there on the ssd, but it’d be nice not to have to reconfigure my system.

I get stuck at a bash prompt in minicom after the above message but it won’t seem to accept any input. I rarely use minicom so i’m not sure if the system is locked up the parameters are wrong.

14

I’m thinking it is a patch level change on an existing non-custom driver. strace could help identify though without guessing. One of the drivers for the NVMe is probably the same “name”, but updated in some way requiring a different IOCTL (or software aware of the IOCTL change). I have no NVMe, so I cannot test. It is quite possible the issue is completely unrelated to the NVIDIA drivers.

I assume you mean “blacklist” and that “blocklist” is a typo, but I am unfamiliar with edgetpu so I don’t really know. Unfortunately, since your config is in “/etc”, and “/etc” is on eMMC, I don’t think it is entirely easy to just blacklist. If there is an initrd that may also require a blacklist, but if you can get to the kernel command line, then if you know the module name, you can blacklist via:
module_blacklist=name
(“name” is a comma delimited set of names)

Perhaps the “extlinux.conf” file is reachable? Or the device tree can be modified without losing the drive (“chosen->bootargs”)?

If you interrupt U-Boot on its command line, then you can see all of its parameters with “printenv”. Possibly you could temporarily tell it “/boot” is from an SD card, and the SD card would contain the same extlinux.conf as your regular “/boot” which points at the NVMe. Sorry, it is hard to guess so many details without actually sitting there and watching and examining as it boots.

15

assume you mean “blacklist” and that “blocklist” is a typo

In light of the proposal from some kernel developers to use more inclusive language. I’m trying to be as accurate as possible in anticipation of future changes. I assume it’ll become an alias at some point.

So something odd just happened. I was trying to get you a serial boot log. I booted it up once, and got this:

[0000.177] W> RATCHET: MB1 binary ratchet value 4 is too large than ratchet level 0 from HW fuses.
[0000.186] I> MB1 (prd-version: 1.5.1.3-t194-41334769-d2a21c57)
[0000.191] I> Boot-mode: Coldboot
[0000.194] I> Chip revision : A02 
[0000.197] I> Bootrom patch version : 7 (correctly patched)
[0000.202] I> ATE fuse revision : 0x200
[0000.206] I> Ram repair fuse : 0x0
[0000.209] I> Ram Code : 0x0
[0000.211] I> rst_source : 0x0
[0000.214] I> rst_level : 0x0
[0000.217] I> Boot-device: eMMC
[0000.232] I> sdmmc DDR50 mode
[0000.237] I> Active Boot chain : 1
[0000.240] I> Boot-device: eMMC
[0000.243] W> MB1_PLATFORM_CONFIG: device prod data is empty in MB1 BCT.
[0000.251] I> Temperature = 22000
[0000.254] W> Skipping boost for clk: BPMP_CPU_NIC
[0000.259] W> Skipping boost for clk: BPMP_APB
[0000.263] W> Skipping boost for clk: AXI_CBB
[0000.267] W> Skipping boost for clk: AON_CPU_NIC
[0000.271] W> Skipping boost for clk: CAN1
[0000.275] W> Skipping boost for clk: CAN2
[0000.279] I> Boot-device: eMMC
[0000.282] I> Boot-device: eMMC
[0000.291] I> Sdmmc: HS400 mode enabled
[0000.296] I> ECC region[0]: Start:0x0, End:0x0
[0000.300] I> ECC region[1]: Start:0x0, End:0x0
[0000.304] I> ECC region[2]: Start:0x0, End:0x0
[0000.308] I> ECC region[3]: Start:0x0, End:0x0
[0000.312] I> ECC region[4]: Start:0x0, End:0x0
[0000.316] I> Non-ECC region[0]: Start:0x80000000, End:0x100000000
[0000.322] I> Non-ECC region[1]: Start:0x0, End:0x0
[0000.326] I> Non-ECC region[2]: Start:0x0, End:0x0
[0000.331] I> Non-ECC region[3]: Start:0x0, End:0x0
[0000.335] I> Non-ECC region[4]: Start:0x0, End:0x0
[0000.341] E> FAILED: Thermal config
[0000.348] E> FAILED: MEMIO rail config
[0000.361] I> Boot-device: eMMC
[0000.371] I> sdmmc bdev is already initialized
[0000.442] I> MB1 done

����main enter
SPE VERSION #: R01.00.14 Created: Sep 19 2018 @ 11:03:21
HW Function test
Start Scheduler.
in late init
��
  [0000.450] I> Welcome to MB2(TBoot-BPMP) (version: 00.00.2018.32-mobile-2dfe4beb)
[0000.451] I> DMA Heap @ [0x526fa000 - 0x52ffa000]
[0000.452] I> Default Heap @ [0xd486400 - 0xd48a400]
[0000.452] E> DEVICE_PROD: Invalid value data = 70020000, size = 0.
[0000.458] W> device prod register failed
[0000.462] I> Boot-device: eMMC
[0000.465] I> Boot_device: SDMMC_BOOT instance: 3
[0000.471] I> sdmmc-3 params source = boot args
[0000.474] I> sdmmc bdev is already initialized
[0000.478] I> sdmmc-3 params source = boot args
[0000.484] I> Found 17 partitions in SDMMC_BOOT (instance 3)
[0000.492] I> Found 42 partitions in SDMMC_USER (instance 3)
[0000.494] I> Active Boot chain : 1
[0000.497] I> parsing oem signed section of bpmp-fw header done
[0000.502] I> bpmp-fw binary init read from storage
[0000.507] I> oem authentication of bpmp-fw header done
[0000.514] I> bpmp-fw binary done read from storage
[0000.516] I> bpmp-fw: Authentication init Done
[0000.521] I> parsing oem signed section of cpubl header done
[0000.526] I> cpubl binary init read from storage
[0000.531] I> bpmp-fw: Authentication Finalize Done
[0000.536] I> oem authentication of cpubl header done
[0000.540] I> cpubl binary done read from storage
[0000.545] I> cpubl: Authentication init Done
[0000.550] I> parsing oem signed section of rce header done
[0000.554] I> rce binary init read from storage
[0000.559] I> Relocating BR-BCT
[0000.561] I> cpubl: Authentication Finalize Done
[0000.566] I> oem authentication of rce header done
[0000.570] I> rce binary done read from storage
[0000.575] I> rce: Authentication init Done
[0000.579] I> parsing oem signed section of ape header done
[0000.584] I> ape binary init read from storage
[0000.588] I> rce: Authentication Finalize Done
[0000.593] I> oem authentication of ape header done
[0000.597] I> ape binary done read from storage
[0000.601] I> ape: Authentication init Done
[0000.606] I> parsing oem signed section of tos header done
[0000.611] I> tos binary init read from storage
[0000.615] I> ape: Authentication Finalize Done
[0000.620] I> oem authentication of tos header done
[0000.624] I> tos binary done read from storage
[0000.628] I> tos: Authentication init Done
[0000.632] I> parsing oem signed section of bpmp-fw-dtb header done
[0000.638] I> bpmp-fw-dtb binary init read from storage
[0000.643] I> tos: Authentication Finalize Done
[0000.649] I> oem authentication of bpmp-fw-dtb header done
[0000.656] I> bpmp-fw-dtb binary done read from storage
[0000.658] I> bpmp-fw-dtb: Authentication init Done
[0000.663] I> parsing oem signed section of cpubl-dtb header done
[0000.668] I> cpubl-dtb binary init read from storage
[0000.673] I> bpmp-fw-dtb: Authentication Finalize Done
[0000.710] I> oem authentication of cpubl-dtb header done
[0000.711] I> cpubl-dtb binary done read from storage
[0000.712] I> cpubl-dtb: Authentication init Done
[0000.714] I> parsing oem signed section of eks header done
[0000.714] I> eks binary init read from storage
[0000.715] I> cpubl-dtb: Authentication Finalize Done
[0000.715] I> oem authentication of eks header done
[0000.719] I> eks binary done read from storage
[0000.723] I> eks: Authentication init Done
[0000.727] I> eks: Authentication Finalize Done
[0000.731] I> EKB detected (length: 0x410) @ VA:0x5270a400
��NOTICE:  BL31: v1.3(release):41d46a9cf
NOTICE:  BL31: Built : 21:14:52, Jun 25 2020
ipc-unittest-main: 1519: Welcome to IPC unittest!!!
ipc-unittest-main: 1531: waiting forever
ipc-unittest-srv: 329: Init unittest services!!!
hwkey-agent: 40: hwkey-agent is running!!
hwkey-agent: 182: key_mgnt_processing .......
hwkey-agent: 153: ekb_verification: EKB_CMAC verification is not match.
hwkey-agent: 219: key_mgnt_processing: failed (-7)
hwkey-agent: 44: main: Failed to verify or extract EKB (-7).
exit called, thread 0xffffffffea89bd58, name trusty_app_2_92b92883-f96a-4177
platform_bootstrap_epilog: trusty bootstrap complete
��

welcome to lk
calling constructors
initializing heap
creating bootstrap completion thread
top of bootstrap2()
initializing platform
bpmp: platform_init
tag is 57f8a77779f848bf2ecf21dabee5645f
tag_show initialized
dt initialized
mail initialized
chipid initialized
fuse initialized
sku initialized
speedo initialized
ec_get_ec_list: found 45 ecs
ec initialized
ec_mrq initialized
vmon_populate_monitors: found 3 monitors
vmon initialized
adc initialized
fmon_populate_monitors: found 73 monitors
fmon initialized
fmon_mrq initialized
reset initialized
nvhs initialized
392 clocks registered
WARNING: pll_c4 has no dyn ramp
clk_mrq_init: mrq handler registered
clk initialized
nvlink initialized
io_dpd initialized
io_dpd initialized
thermal initialized
i2c5 controller initialized
initialized i2c mrq handling
i2c initialized
regulator initialized
avfs_clk_platform initialized
soctherm initialized
aotag initialized
powergate initialized
dvs initialized
pm initialized
pg_late initialized
strap initialized
tag initialized
emc initialized
clk_dt initialized
avfs_ccplex_platform initialized
tj_max: dt node not found
tj_init initialized
uphy_mrq_init: mrq handler registered
uphy_dt initialized
uphy initialized
safereg_init: period 80 ms
ec_late initialized
��
  ��mrq initialized
��[0001.200] I> Welcome to Cboot
��fmon_post initialized
��[0001.200] I> Cboot Version: t194-18fdfe28
[0001.200] I> CPU-BL Params @ 0xf2820000
[0001.201] I>  0) Base:0x00000000 Size:0x00000000
[0001.205] I>  1) Base:0xf1100000 Size:0x00100000
[0001.209] I>  2) Base:0xf2000000 Size:0x00200000
��clk_set_parent failed for clk can1, parent pll_aon (-22)
clk_set_parent failed for clk can2, parent pll_aon (-22)
clk_set_parent failed for clk dmic5, parent pll_aon (-22)
clk_set_parent failed for clk i2c2, parent pll_aon (-22)
clk_set_parent failed for clk i2c8, parent pll_aon (-22)
clk_set_parent failed for clk spi2, parent pll_aon (-22)
clk_set_parent failed for clk pwm4, parent pll_aon (-22)
clk_dt_late initialized
machine_check initialized
pm_post initialized
dbells initialized
avfs_clk_platform_post initialized
dmce initialized
cvc initialized
ccplex_avfs_hw_init: nafll_cluster0: not monitored
ccplex_avfs_hw_init: nafll_cluster1: not monitored
ccplex_avfs_hw_init: nafll_cluster2: not monitored
ccplex_avfs_hw_init: nafll_cluster3: not monitored
avfs_clk_mach_post initialized
regulator_post initialized
rm initialized
sc7_diag initialized
thermal_test initialized
serial_late initialized
clk_post initialized
clk_dt_post initialized
mc_reg initialized
pg_post initialized
dyn_modules initialized
sku_debugfs initialized
speedo_debugfs initialized
adc_debugfs initialized
clk_debugfs initialized
��[0001.213] I>  3) Base:0xf1200000 Size:0x00200000
[0001.318] I>  4) Base:0xf1000000 Size:0x00100000
[0001.323] I>  5) Base:0xf0f00000 Size:0x00100000
[0001.327] I>  6) Base:0xf3800000 Size:0x00400000
[0001.332] I>  7) Base:0xf1c00000 Size:0x00400000
[0001.336] I>  8) Base:0xf0e00000 Size:0x00100000
[0001.341] I>  9) Base:0xf0d00000 Size:0x00100000
[0001.345] I> 10) Base:0xf3000000 Size:0x00800000
[0001.350] I> 11) Base:0x40000000 Size:0x00040000
��emc_debugfs initialized
dvs_debugfs initialized
��[0001.354] I> 12) Base:0xf0c00000 Size:0x00100000
��fmon_debugfs initialized
vmon_debugfs initialized
pg_debugfs initialized
profile_fs initialized
debugfs_cons initialized
mail_fs initialized
profile initialized
cvc_debugfs initialized
dmce_debugfs initialized
ec_debugfs initialized
rm_debugfs initialized
soctherm_debug initialized
gr_reader initialized
mods initialized
dt_fs initialized
debugfs_mrq initialized
debug_mrq initialized
debug_safereg initialized
initializing target
calling apps_init()
starting app shell
entering main console loop
] ��[0001.363] I> 13) Base:0x40046000 Size:0x00002000
[0001.414] I> 14) Base:0x40048000 Size:0x00002000
[0001.418] I> 15) Base:0xac000000 Size:0x00004000
[0001.423] I> 16) Base:0x4004a000 Size:0x00002000
[0001.427] I> 17) Base:0xf0b00000 Size:0x00100000
[0001.432] I> 18) Base:0x4004c000 Size:0x00002000
[0001.436] I> 19) Base:0xf2200000 Size:0x00600000
[0001.441] I> 20) Base:0x4004e000 Size:0x00002000
[0001.445] I> 21) Base:0xf09d0000 Size:0x0000c000
[0001.449] I> 22) Base:0x00000000 Size:0x00000000
[0001.454] I> 23) Base:0xf09e0000 Size:0x00020000
[0001.458] I> 24) Base:0xf6000000 Size:0x02000000
[0001.463] I> 25) Base:0x40050000 Size:0x00002000
[0001.467] I> 26) Base:0x40040000 Size:0x00006000
[0001.472] I> 27) Base:0xf1800000 Size:0x00400000
[0001.476] I> 28) Base:0xf4c00000 Size:0x01400000
[0001.481] I> 29) Base:0xf1400000 Size:0x00400000
[0001.485] I> 30) Base:0xf0a00000 Size:0x00100000
[0001.490] I> 31) Base:0x00000000 Size:0x00000000
[0001.494] I> 32) Base:0xf8000000 Size:0x08000000
[0001.499] I> 33) Base:0x00000000 Size:0x00000000
[0001.503] I> 34) Base:0xf3c00000 Size:0x01000000
[0001.507] I> 35) Base:0xab000000 Size:0x01000000
[0001.512] I> 36) Base:0xa0000000 Size:0x0b000000
[0001.516] I> 37) Base:0xf2800000 Size:0x00800000
[0001.521] I> 38) Base:0x80000000 Size:0x20000000
[0001.525] I> 39) Base:0xb0000000 Size:0x08000000
[0001.530] I> 40) Base:0x00000000 Size:0x00000000
[0001.534] I> 41) Base:0x00000000 Size:0x00000000
[0001.539] I> 42) Base:0x00000000 Size:0x00000000
[0001.543] I> 43) Base:0x00000000 Size:0x00000000
[0001.548] I> 44) Base:0x00000000 Size:0x00000000
[0001.552] I> 45) Base:0x00000000 Size:0x00000000
[0001.557] GIC-SPI Target CPU: 0
[0001.560] Interrupts Init done
[0001.562] calling constructors
[0001.565] initializing heap
[0001.568] I> Heap: [0xa06905a8 ... 0xab000000]
[0001.572] initializing threads
[0001.575] initializing timers
[0001.578] creating bootstrap completion thread
[0001.582] top of bootstrap2()
[0001.585] CPU: MIDR: 0x4E0F0040, MPIDR: 0x80000000
[0001.590] initializing platform
[0001.593] E> DEVICE_PROD: Invalid value data = 0, size = 0.
[0001.598] W> device prod register failed
[0001.602] I> Bl_dtb @0xaaf00000
[0001.608] W> "plugin-manager" doesn't exist, creating
[0001.610] W> "ids" doesn't exist, creating
[0001.614] W> "connection" doesn't exist, creating
[0001.618] W> "configs" doesn't exist, creating
[0001.631] I> Find /i2c@3160000's alias i2c0
[0001.631] I> Reading eeprom i2c=0 address=0x50
[0001.657] I> Device at /i2c@3160000:0x50
[0001.657] I> Reading eeprom i2c=0 address=0x56
[0001.681] I> Device at /i2c@3160000:0x56
[0001.683] I> Find /i2c@3180000's alias i2c2
[0001.684] I> Reading eeprom i2c=2 address=0x54
[0001.685] E> I2C: slave not found in slaves.
[0001.686] E> I2C: Could not write 0 bytes to slave: 0x00a8 with repeat start true.
[0001.687] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.687] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa8 at 0x00000000 via instance 2.
[0001.695] E> eeprom: Failed to read I2C slave device
[0001.700] I> Eeprom read failed 0x3526070d
[0001.704] I> Reading eeprom i2c=2 address=0x57
[0001.709] E> I2C: slave not found in slaves.
[0001.713] E> I2C: Could not write 0 bytes to slave: 0x00ae with repeat start true.
[0001.721] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.726] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xae at 0x00000000 via instance 2.
[0001.736] E> eeprom: Failed to read I2C slave device
[0001.740] I> Eeprom read failed 0x3526070d
[0001.744] I> Reading eeprom i2c=2 address=0x52
[0001.749] E> I2C: slave not found in slaves.
[0001.753] E> I2C: Could not write 0 bytes to slave: 0x00a4 with repeat start true.
[0001.761] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.766] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa4 at 0x00000000 via instance 2.
[0001.776] E> eeprom: Failed to read I2C slave device
[0001.780] I> Eeprom read failed 0x3526070d
[0001.785] I> Find /i2c@c240000's alias i2c1
[0001.788] I> Reading eeprom i2c=1 address=0x52
[0001.794] E> I2C: slave not found in slaves.
[0001.797] E> I2C: Could not write 0 bytes to slave: 0x00a4 with repeat start true.
[0001.805] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.810] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa4 at 0x00000000 via instance 1.
[0001.820] E> eeprom: Retry to read I2C slave device.
[0001.825] E> I2C: slave not found in slaves.
[0001.829] E> I2C: Could not write 0 bytes to slave: 0x00a4 with repeat start true.
[0001.837] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.842] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa4 at 0x00000000 via instance 1.
[0001.852] E> eeprom: Failed to read I2C slave device
[0001.856] I> Eeprom read failed 0x3526070d
[0001.860] I> Reading eeprom i2c=1 address=0x50
[0001.865] E> I2C: slave not found in slaves.
[0001.869] E> I2C: Could not write 0 bytes to slave: 0x00a0 with repeat start true.
[0001.877] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.882] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa0 at 0x00000000 via instance 1.
[0001.892] E> eeprom: Retry to read I2C slave device.
[0001.897] E> I2C: slave not found in slaves.
[0001.901] E> I2C: Could not write 0 bytes to slave: 0x00a0 with repeat start true.
[0001.908] E> I2C_DEV: Failed to send register address 0x00000000.
[0001.914] E> I2C_DEV: Could not read 256 registers of size 1 from slave 0xa0 at 0x00000000 via instance 1.
[0001.924] E> eeprom: Failed to read I2C slave device
[0001.928] I> Eeprom read failed 0x3526070d
[0001.932] I> create_pm_ids: id: 2888-0001-400-H, len: 15
[0001.937] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0001.948] I> create_pm_ids: id: 2822-0000-500-F, len: 15
[0001.954] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0001.965] I> Adding plugin-manager/ids/2888-0001-400=/i2c@3160000:module@0x50
[0001.973] W> "i2c@3160000" doesn't exist, creating
[0001.976] W> "module@0x50" doesn't exist, creating
[0001.981] I> Adding plugin-manager/ids/2822-0000-500=/i2c@3160000:module@0x56
[0001.988] W> "module@0x56" doesn't exist, creating
[0001.995] I> Adding plugin-manager/cvm
[0001.997] W> "chip-id" doesn't exist, creating
[0002.000] I> Adding plugin-manager/chip-id/A02
[0002.005] I> Plugin-manager override starting
[0002.010] I> node /plugin-manager/fragement-tegra-wdt-en matches
[0002.019] I> node /plugin-manager/fragement-soft-wdt matches
[0002.027] I> node /plugin-manager/fragment-pcie-c5-rp matches
[0002.032] I> node /plugin-manager/fragment-tegra-ufs-lane10 matches
[0002.041] I> Disable plugin-manager status in FDT
[0002.041] I> Plugin-manager override finished successfully
[0002.042] I> gpio framework initialized
[0002.047] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio' driver
[0002.054] I> tegrabl_gpio_driver_register: register 'nvidia,tegra194-gpio-aon' driver
[0002.061] I> tegrabl_tca9539_init: i2c bus: 1, slave addr: 0x46
[0002.069] E> fetch_driver_phandle_from_dt: failed to get node with compatible ti,tca9539
[0002.076] E> fetch_driver_phandle_from_dt: failed to get node with compatible nxp,tca9539
[0002.082] W> tegrabl_tca9539_init: failed to fetch phandle from dt
[0002.088] I> tegrabl_tca9539_init: i2c bus: 1, slave addr: 0x44
[0002.096] E> fetch_driver_phandle_from_dt: failed to get node with compatible ti,tca9539
[0002.104] E> fetch_driver_phandle_from_dt: failed to get node with compatible nxp,tca9539
[0002.110] W> tegrabl_tca9539_init: failed to fetch phandle from dt
[0002.118] I> fixed regulator driver initialized
[0002.132] I> register 'maxim' power off handle
[0002.133] I> virtual i2c enabled
[0002.133] I> registered 'maxim,max20024' pmic
[0002.133] I> tegrabl_gpio_driver_register: register 'max20024-gpio' driver
[0002.139] I> Boot-device: eMMC
[0002.142] I> Boot_device: SDMMC_BOOT instance: 3
[0002.151] I> sdmmc-3 params source = boot args
[0002.151] I> create_pm_ids: id: 2888-0001-400-H, len: 15
[0002.156] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0002.167] I> create_pm_ids: id: 2822-0000-500-F, len: 15
[0002.172] I> config: mem-type:00,power-config:00,misc-config:00,modem-config:00,touch-config:00,display-config:00,, len: 93
[0002.183] I> sdmmc bdev is already initialized
[0002.187] I> sdmmc-3 params source = boot args
[0002.218] I> Found 17 partitions in SDMMC_BOOT (instance 3)
[0002.230] I> Found 42 partitions in SDMMC_USER (instance 3)
[0002.239] I> enabling 'vdd-hdmi-5v0' regulator
[0002.244] I> regulator 'vdd-hdmi-5v0' already enabled
[0002.245] I> hdmi cable connected
[0002.247] W> set volts not configured for 'vdd-1v0'
[0002.249] W> set volts not configured for 'vdd-1v8-hs'
[0002.253] E> invalid display type
[0002.256] E> invalid display type
[0002.257] E> cannot find any other nvdisp nodes
[0002.273] I> edid read success
[0002.285] I> edid read success
[0002.285] I> width = 640, height = 480, frequency = 25174825
[0002.286] I> width = 640, height = 480, frequency = 25174825
[0002.286] I> width = 640, height = 480, frequency = 25174825
[0002.287] I> width = 640, height = 480, frequency = 25174825
[0002.287] I> width = 3840, height = 2160, frequency = 594000000
[0002.291] I> width = 1920, height = 1080, frequency = 148500000
[0002.297] I> width = 1920, height = 540, frequency = 74250000
[0002.302] I> width = 3840, height = 2160, frequency = 593406593
[0002.308] I> width = 1920, height = 1080, frequency = 148351648
[0002.314] I> width = 1920, height = 1080, frequency = 148351648
[0002.319] I> width = 1280, height = 720, frequency = 74175824
[0002.325] I> width = 1280, height = 720, frequency = 74175824
[0002.331] I> width = 1920, height = 1080, frequency = 74175824
[0002.336] I> width = 1920, height = 1080, frequency = 74175824
[0002.342] I> width = 1920, height = 1080, frequency = 74175824
[0002.348] I> width = 3840, height = 2160, frequency = 296703296
[0002.354] I> width = 720, height = 480, frequency = 26973026
[0002.359] I> width = 720, height = 576, frequency = 26973026
[0002.365] I> width = 3840, height = 2160, frequency = 296703296
[0002.370] I> width = 3840, height = 2160, frequency = 296703296
[0002.376] I> width = 3840, height = 2160, frequency = 296703296
[0002.382] I> width = 4096, height = 2160, frequency = 296703296
[0002.388] I> Best mode Width = 1920, Height = 1080, freq = 148351648
[0002.398] I> hdmi_enable, starting HDMI initialisation
[0002.403] I> hdmi_enable, HDMI initialisation complete
[0002.413] I> Load in CBoot Boot Options partition and parse it
[0002.420] E> Error -9 when finding node with path /boot-configuration
[0002.420] E> tegrabl_cbo_parse_info: "boot-configuration" not found in CBO file.
[0002.423] I> Hit any key to stop autoboot:     4       3       2       1
[0004.431] initializing target
[0004.431] calling apps_init()
[0004.432] starting app kernel_boot_app
[0004.451] I> found decompressor handler: lz4-legacy
[0004.452] I> decompressing BMP blob ...
[0004.455] I> Kernel type = Normal
[0004.455] I> Loading kernel-bootctrl from partition
[0004.456] I> Loading partition kernel-bootctrl at 0xa4ac0000 from device(0x1)
[0004.462] W> tegrabl_get_kernel_bootctrl: magic number(0x00000000) is invalid
[0004.463] W> tegrabl_get_kernel_bootctrl: use default dummy boot control data
[0004.464] I> ########## SD boot ##########
[0004.468] I> No sdcard
[0004.470] I> -0 params source = 
[0004.473] E> Blockdev open: exit error
[0004.476] E> SD boot failed, err: 724238353
[0004.480] I> ########## USB boot ##########
[0004.501] I> USB Firmware Version: 60.06 release
[0004.557] I> regulator of usb2-0 already enabled
[0004.563] I> regulator of usb2-1 already enabled
[0004.569] I> regulator of usb2-2 already enabled
[0004.578] I> enabling 'vdd-5v-sata' regulator
[0005.583] E> failed to initialize xhci controller
[0005.583] E> Error in init of XUSB host driver, err: 79790026
[0005.583] E> Failed to initialize device 5-0
[0005.584] E> USB boot failed, err: 2037973030
[0005.584] I> ########## Fixed storage boot ##########
[0005.584] I> Already published: 00010003
[0005.585] I> Look for boot partition
[0005.588] I> Fallback: assuming 0th partition is boot partition
[0005.594] I> Detect filesystem
[0005.621] I> Loading extlinux.conf ...
[0005.621] I> rootfs path: /sdmmc_user/boot/extlinux/extlinux.conf
[0005.658] I> L4T boot options
[0005.658] I> [1]: "primary kernel"
[0005.659] I> Enter choice: 
[0008.660] I> Continuing with default option: 1
[0008.660] I> Loading kernel sig file from rootfs ...
[0008.660] I> rootfs path: /sdmmc_user/boot/Image.sig
[0008.679] I> Loading kernel binary from rootfs ...
[0008.679] I> rootfs path: /sdmmc_user/boot/Image
[0008.916] I> Validate kernel ...
[0008.916] I> T19x: Authenticate kernel (bin_type: 37), max size 0x5000000
[0009.227] E> digest on binary did not match!!
[0009.228] C> OEM authentication of kernel payload failed!
[0009.228] W> Failed to validate kernel binary (err=1077936152, fail=0)
[0009.229] W> Security fuse not burned, ignore validation failure
[0009.236] I> No kernel-dtb binary path
[0009.242] I> A/B: bin_type (38) slot 1
[0009.242] I> Loading kernel-dtb_b from partition
[0009.243] I> Loading partition kernel-dtb_b at 0x91000000 from device(0x1)
[0009.253] I> Validate kernel-dtb ...
[0009.254] I> T19x: Authenticate kernel-dtb (bin_type: 38), max size 0x400000
[0009.257] I> Loading ramdisk from rootfs ...
[0009.258] I> rootfs path: /sdmmc_user/boot/initrd
[0009.310] I> Kernel hdr @0xa4ac0000
[0009.310] I> Kernel dtb @0x90000000
[0009.310] I> decompressor handler not found
[0009.311] I> Copying kernel image (34330632 bytes) from 0xa4ac0000 to 0x80080000 ... [0009.321] I> Done
[0009.322] I> Updated bpmp info to DTB
[0009.323] I> Ramdisk: Base: 0x92000000; Size: 0x54ecaf
[0009.323] I> Updated initrd info to DTB
[0009.324] W> WARN: Fail to override "console=none" in commandline
[0009.324] E> tegrabl_linuxboot_add_disp_param, du 1 failed to get display params
[0009.330] E> tegrabl_linuxboot_add_disp_param, du 1 failed to get display params
[0009.337] I> Active slot suffix: _b
[0009.341] I> add_boot_slot_suffix: slot_suffix = _b
[0009.345] I> Linux Cmdline: console=ttyTCU0,115200 video=tegrafb no_console_suspend=1 earlycon=tegra_comb_uart,mmio32,0x0c168000 gpt tegra_fbmem=0x800000@0xa069c000 lut_mem=0 
[0009.375] I> Updated bootarg info to DTB
[0009.379] W> MAC addr invalid!
[0009.382] E> Failed to get WIFI MAC address
[0009.386] W> MAC addr invalid!
[0009.389] E> Failed to get Bluetooth MAC address
[0009.393] I> eeprom_get_mac_addr: MAC (type: 2): 00:04:4b:c7:bb:2a
[0009.400] W> "plugin-manager" doesn't exist, creating
[0009.404] I> Adding /chosen/plugin-manager/cvm
[0009.408] W> "chip-id" doesn't exist, creating
[0009.413] I> Adding /chosen/plugin-manager/chip-id
[0009.417] W> "configs" doesn't exist, creating
[0009.422] I> Adding /chosen/plugin-manager/configs
[0009.426] W> "ids" doesn't exist, creating
[0009.430] I> Adding /chosen/plugin-manager/ids
[0009.435] W> "odm-data" doesn't exist, creating
[0009.439] I> Adding /chosen/plugin-manager/odm-data
[0009.446] W> "memory" doesn't exist, creating
[0009.448] I> [0] START: 0x80000000, END: 0xac000000
[0009.452] I> [1] START: 0xac004000, END: 0xf09d0000
[0009.457] I> [2] START: 0xf09dc000, END: 0xf09e0000
[0009.462] I> dram_block larger than 80000000
[0009.466] I> [3] START: 0x100000000, END: 0x480000000
[0009.471] I> added [base:0x80000000, size:0x2c000000] to /memory
[0009.477] I> added [base:0xac200000, size:0x44600000] to /memory
[0009.483] I> added [base:0x100000000, size:0x380000000] to /memory
[0009.490] I> Updated memory info to DTB
[0009.493] E> add_disp_param: failed to get display params for du=1
[0009.499] W> "reset" doesn't exist, creating
[0009.503] I> NVG: Logical CPU: 0; MPIDR: 0x80000000
[0009.507] I> NVG: Logical CPU: 1; MPIDR: 0x80000001
[0009.512] I> NVG: Logical CPU: 2; MPIDR: 0x80000100
[0009.517] I> NVG: Logical CPU: 3; MPIDR: 0x80000101
[0009.521] I> NVG: Logical CPU: 4; MPIDR: 0x80000200
[0009.526] I> NVG: Logical CPU: 5; MPIDR: 0x80000201
[0009.531] I> NVG: Logical CPU: 6; MPIDR: 0x80000300
[0009.536] I> NVG: Logical CPU: 7; MPIDR: 0x80000301
[0009.541] W> "misc-data" doesn't exist, creating
[0009.545] I> Boot-device: eMMC
[0009.548] I> Add boot-sdmmc to plugin-manager/misc-data
[0009.553] I> Add storage-sdmmc to plugin-manager/misc-data
[0009.558] W> Unknown storage device
[0009.562] I> Add serial number:[REDACTED] as DT property
[0009.568] I> Plugin-manager override starting
[0009.571] I> node /plugin-manager/fragement-tegra-wdt-en matches
[0009.579] I> node /plugin-manager/fragement-soft-wdt matches
[0009.586] I> node /plugin-manager/fragment-pcie-c5-rp matches
[0009.591] I> node /plugin-manager/fragment-tegra-ufs-lane10 matches
[0009.603] I> Disable plugin-manager status in FDT
[0009.603] I> Plugin-manager override finished successfully
[0009.604] I> tegrabl_load_kernel_and_dtb: Done
[0009.653] I> Kernel EP: 0x80080000, DTB: 0x90000000
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.140-tegra (buildbrain@mobile-u64-3357) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Lin0
[    0.000000] Boot CPU: AArch64 Processor [4e0f0040]
[    0.000000] OF: fdt:memory scan node memory, reg size 48,
[    0.000000] OF: fdt: - 80000000 ,  2c000000
[    0.000000] OF: fdt: - ac200000 ,  44600000
[    0.000000] OF: fdt: - 100000000 ,  380000000
[    0.000000] earlycon: tegra_comb_uart0 at MMIO32 0x000000000c168000 (options '')
[    0.000000] bootconsole [tegra_comb_uart0] enabled
[    0.000000] Found tegra_fbmem: 00800000@a069c000
[    0.000000] Found lut_mem: 00002008@a0697000
��WARNING: pll_d3 has no dyn ramp
��bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

Then, I realized i snipped off the bash prompt at the end, so I rebooted again. Realizing i specified the wrong TTY, i powered off (causing an unclean unmount of teh NVMe), and retried… and it booted.

[0009.638] I> Kernel EP: 0x80080000, DTB: 0x90000000
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.140-tegra (buildbrain@mobile-u64-3357) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Lin0
[    0.000000] Boot CPU: AArch64 Processor [4e0f0040]
[    0.000000] OF: fdt:memory scan node memory, reg size 48,
[    0.000000] OF: fdt: - 80000000 ,  2c000000
[    0.000000] OF: fdt: - ac200000 ,  44600000
[    0.000000] OF: fdt: - 100000000 ,  380000000
[    0.000000] earlycon: tegra_comb_uart0 at MMIO32 0x000000000c168000 (options '')
[    0.000000] bootconsole [tegra_comb_uart0] enabled
[    0.000000] Found tegra_fbmem: 00800000@a069c000
[    0.000000] Found lut_mem: 00002008@a0697000
��WARNING: pll_d3 has no dyn ramp
��[    6.147453] cgroup: cgroup2: unknown option "nsdelegate"
[    7.672332] EXT4-fs (nvme0n1p6): warning: mounting unchecked fs, running e2fsck is recommended
[   12.343943] using random self ethernet address
[   12.344094] using random host ethernet address
[   12.354405] random: crng init done
[   12.354525] random: 7 urandom warning(s) missed due to ratelimiting
[   12.782666] using random self ethernet address
[   12.782670] using random host ethernet address

Ubuntu 18.04.4 LTS hostname ttyTCU0

So no clue what happened there. It works, now. The only difference seems to be the filesystem was not cleanly unmounted, and yet that fixed it? Waiting for me at login was an update for the Coral modules.

 $ sudo apt update && sudo apt dist-upgrade -y --autoremove
[sudo] password for username: 
Hit:1 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease
Hit:2 http://ports.ubuntu.com/ubuntu-ports bionic InRelease              
Hit:3 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease      
Hit:4 http://ports.ubuntu.com/ubuntu-ports bionic-backports InRelease   
Hit:5 https://repo.download.nvidia.com/jetson/ffmpeg main InRelease     
Hit:6 https://repo.download.nvidia.com/jetson/common r32.4 InRelease
Hit:7 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease           
Hit:8 https://repo.download.nvidia.com/jetson/t194 r32.4 InRelease             
Reading package lists... Done                                                  
Building dependency tree       
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  gasket-dkms
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 46.2 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 https://packages.cloud.google.com/apt coral-edgetpu-stable/main arm64 gasket-dkms all 1.0-12 [46.2 kB]
Fetched 46.2 kB in 0s (112 kB/s)     
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 200449 files and directories currently installed.)
Preparing to unpack .../gasket-dkms_1.0-12_all.deb ...

-------- Uninstall Beginning --------
Module:  gasket
Version: 1.0
Kernel:  4.15.0-109-generic (aarch64)
-------------------------------------

Status: Before uninstall, this module version was ACTIVE on this kernel.

gasket.ko:
 - Uninstallation
   - Deleting from: /lib/modules/4.15.0-109-generic/updates/dkms/
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.


apex.ko:
 - Uninstallation
   - Deleting from: /lib/modules/4.15.0-109-generic/updates/dkms/
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.

depmod...

Warning: Unable to find an initial ram disk that I know how to handle.
Will not try to make an initrd.

DKMS: uninstall completed.

-------- Uninstall Beginning --------
Module:  gasket
Version: 1.0
Kernel:  4.9.140-tegra (aarch64)
-------------------------------------

Status: Before uninstall, this module version was ACTIVE on this kernel.

gasket.ko:
 - Uninstallation
   - Deleting from: /lib/modules/4.9.140-tegra/updates/dkms/
 - Original module
   - No original module was found for this module on this kernel.
   - Use the dkms install command to reinstall any previous module version.


Progress: [  0%] [..........................................................] 
 - Uninstallation
Done.
Unpacking gasket-dkms (1.0-12) over (1.0-11) ...............................] 
Setting up gasket-dkms (1.0-12) ...
Loading new gasket-1.0 DKMS files...
It is likely that 4.9.140-tegra belongs to a chroot's host
Building for 4.15.0-108-generic, 4.15.0-109-generic and 4.9.140-tegra
Module build for kernel 4.15.0-108-generic was skipped since the
kernel headers for this kernel does not seem to be installed.
Building initial module for 4.15.0-109-generic
Done.

gasket:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.15.0-109-generic/updates/dkms/

apex.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.15.0-109-generic/updates/dkms/

depmod...

Warning: Unable to find an initial ram disk that I know how to handle.
Will not try to make an initrd.

DKMS: install completed.
Building initial module for 4.9.140-tegra
Done.

gasket:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.9.140-tegra/updates/dkms/

apex.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.9.140-tegra/updates/dkms/

depmod...

Backing up initrd.img-4.9.140-tegra to /boot/initrd.img-4.9.140-tegra.old-dkms
Making new initrd.img-4.9.140-tegra
(If next boot fails, revert to initrd.img-4.9.140-tegra.old-dkms image)
update-initramfs.....

DKMS: install completed.

and rebooting… and it works. I assume I can put my Coral back in now that a module is rebuilt. Any insight as to what might have gone wrong? I’m unsure whether to blame the DKMS, the Coral modules, or the NVMe configuration here.

Edit: yeah, it boots with the coral inserted and the modules for it are loaded.
Edit2: I didnt’ put 2 and 2 together immeidatley, but somebody reported this an issue on my GitHub the other day, so I don’t think it’s NVMe or Coral related per the note at the bottom of this post it may be a bad kernel package that did it. I still have no earthly idea why it’s booting now.
Edit: per post directly below, it might be NVMe related, actually.

16

I am the person that reported the same issue on your github and yes it was booting on nvme , I rebooted it maybe 100 times but it never fixed itself (with and without nvme ssd, and with different other micro sd card). I Sent it back to vendor and got reimbursed.

17

Interesting. Mine wasn’t booting from NVMe, but /usr and a few other things were mounted. Also, in my case this happened on an AGX, not an NX. I’m just glad it fixed itself for me. Sorry it didn’t in your case. Hope you will still give Jetson another try at some point. I’ve had really good luck apart from this.

18

This is purely speculation, but consider the following. Any journal replay occurs at the moment of mounting, and the moment of mounting for the rootfs is likely prior to the Linux kernel loading and using the ext4 rootfs. It doesn’t mean the kernel can’t do a remount (the init would have to be set up to do this, and if you have an initrd, then both the initrd init and the full system init might be involved, depending on timing) and run the journal then.

Next consider that the drivers for ext4 in boot stages prior to Linux running do not accept all of the options which work with the Linux kernel itself. In the past neither the 64bit option nor the metadata_csum option were allowed in boot stages. I’m not positive, but the 64bit option might work in earlier boot stages now, but I’m “mostly” confident that the metadata_csum is still not allowed in early boot stages. It turns out that metadata_csum is related to the journal (after all, it is “meta” data checksum).

If for some reason the journal needed to be played upon mount, and the mount was prior to the Linux kernel being loaded, then the driver for replay of the journal would be the one in the boot stages. If metadata_csum was used, but boot stages could still mount ext4, then the only way to mount would be to ignore the journal and mount in such a ways as to risk corruption.

I’m pretty sure (via “dumpe2fs | egrep -i '(64bit|metadata_csum)'”) that the newer releases include the 64-bit options now, but part of this depends on the host PC itself. Ubuntu 16.04 would not have used those options by default, but I think 18.04 does (?..I am using Fedora). So it makes sense that probably earlier boot stages can now use those options for mount (but not necessarily for journal replay).

Can anyone from NVIDIA say if recent releases support ext4 64bit and metadata_csum in earlier boot stages, and whether those earlier stages are set up to do a journal replay in the case of a journal showing unclean shutdown?

NOTE: Try manually creating the rootfs partition’s ext4 some time with these options:
sudo mkfs.ext4 -O ^metadata_csum,^64bit /dev/whichever_it_is
…then see if you can purposely do an unclean mount, followed by logging boot to see if journal replay works or not.

EDIT: I probably have this wrong because I just realized you did not use rootfs on the NVMe. Non-rootfs probably mounts only after earlier boot stages run.

19

It turns out it’s unrelated to NVMe @linuxdev . There’s was a bad bootloader package pushed as an update that did it. This is happening to people running purely from SD card. The solution is to reboot seven times. Seven shall be the number of the reboot and the number of the reboot shall be seven. Six shalt thou not count, neither shalt thou count four, excepting that thou then proceedeth to five. Eighth is right out.

20

Oddly, a bootloader package error would be no different than not supporting journals with metadata_csum. Now that you know the true name of the evil package though, I’m sure it will be easier to slay than is metadata_csum. :P