Trouble using self-built u-boot in R28.1

I’m having trouble getting a Tx1 DevKit to boot using a u-boot that I have built. Specific notes are below but the TL;DR; question is: what is the new process in R28.1 to deploy u-boot after building it?

Background

I have a working build of a custom tx1-based system with custom rootfs, custom kernel build and custom uboot build currently based on L4T R24.2.1. I’m trying to update this to R28.1 and just “swapping in” R28.1 leads to a non-booting system. Using a devkit I’ve narrowed it down to something different about the bootloader deployement.

R28.1 Boot Flow

The documentation for l4T r28.1 says:

U-Boot is the default boot loader for NVIDIA® Tegra® Linux Driver Package.
For more information, see U-Boot Customization.

(i.e. from file:///tmp/nvidia-docs/nvl4t_docs/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fl4t_bootflow_tx1.html%23wwconnect_header)

This does appear to be the case as I see the following in the serial terminal output while booting:

U-Boot 2016.07-g0ce7ca2 (Jul 20 2017 - 00:37:03 -0700)

However the file that is actually flashed to the EBT partition is called cboot.bin and comes from Linux_for_Tegra/bootloader/t210ref/cboot.bin

This is different than previous BSPs where u-boot-dtb (ELF file) was patched with a TBOOT header and then deployed to the EBT partition (as filename u-boot-dtb.bin`).

We currently build u-boot ourselves as we have applied a couple of patches to the source which aren’t in the nvidia distributed version. In older releases (R24.1) we would build u-boot from source, then run gen-tboot-image.py on the file to get the image that was flashed to the EBT partition. When we replicate that same process with the R28.1 uboot and a stock L4T driver package and rootfs, we are unable to boot.

We are building from the nvidia branch l4t/l4t-r28.1 (tag tegra-l4t-r28.1) on git://nv-tegra.nvidia.com/3rdparty/u-boot.git currently at commit:

commit 0ce7ca286491e97a34d741bd92a57f2dcdc3033c (tag: tegra-l4t-r28.1, nvidia/l4t/l4t-r28.1, l4t/l4t-r28.1)
Author: JC Kuo <jckuo@nvidia.com>
Date:   Fri Apr 14 17:54:21 2017 +0800

    T210: do not enable PLLE and UPHY PLL HW PWRSEQ

    This commit removes the programming sequence that enables PLLE
    and UPHY PLL hardware power sequencers. Per TRM, boot software
    should enable PLLE and UPHY PLLs in software controlled power-on
    state and should power down PLL before jumping into kernel or
    the next stage boot software.

    Bug 1889735

    Change-Id: I41b563d613b502abe902393b92ce311720d59ec0
    Signed-off-by: JC Kuo <jckuo@nvidia.com>
    Reviewed-on: http://git-master/r/1462905
    (cherry picked from commit f6ce7fbb731949fb94187b735ee76d30ef6256ad on dev-other-v2016.07)
    Reviewed-on: http://git-master/r/1484083
    Reviewed-by: Stephen Warren <swarren@nvidia.com>
    GVS: Gerrit_Virtual_Submit
    Reviewed-by: Tom Warren <twarren@nvidia.com>
    Tested-by: Tom Warren <twarren@nvidia.com>

There must be something different about the uboot configuration and/or boot flow and the method used to generate the bootloader partition image that has changed between 24.1 and 28.1.

Steps to reproduce this problem:

  1. Start with a fresh 28.1 Driver pack
  2. Extract the sample root filesystem into rootfs
  3. Run sudo ./apply_binaries.sh

WORKS:
4. flash with sudo ./flash.sh jetson-tx1 mmcblk0p1

WORKS:
4. flash with sudo ./flash.sh -r -L bootloader/t210ref/cboot.bin jetson-tx1 mmcblk0p1

(should be the same as above)

FAILS at flash step:
4. sudo ./flash.sh -L u-boot-dtb.bin jetson-tx1 mmcblk0p1

Error: missing bootloader (/home/skydio/aircam/build/images/LinuxForTegra_R28.1.0_arm64_tx1/u-boot-dtb.bin).

FAILS to boot:
4. sudo ./flash.sh -L bootloader/t210ref/p2371-2180/u-boot-dtb.bin jetson-tx1 mmcblk0p1

[0000.403] Performing RAM repair
[0000.406] Updating A64 Warmreset Address to 0xa00002e9
[0000.434] Bootloader DTB Load Address: 0x83000000
[0000.462] Kernel DTB Load Address: 0x83100000
[0000.467] Bootloader is not valid
[0000.470] Error in NvTbootLoadBinary: 0x14 !
[0000.474] GPT: Partition NOT found !
[0000.477] Find Partition via GPT Failed
[0000.481] Find Partition via PT Failed
[0000.485] function NvTbootGetBinaryOffsets: 0x1 error
[0000.489] Error in NvTbootLoadBinary: 0x1 !
[0000.493] Error is 1

FAILS to boot:
4. flash with sudo ./flash.sh -r -L /path/to/my/build/of/u-boot-dtb.bin jetson-tx1 mmcblk0p1

[0000.403] Performing RAM repair
[0000.406] Updating A64 Warmreset Address to 0xa00002e9
[0000.434] Bootloader DTB Load Address: 0x83000000
[0000.462] Kernel DTB Load Address: 0x83100000
[0000.467] Bootloader is not valid
[0000.470] Error in NvTbootLoadBinary: 0x14 !
[0000.474] GPT: Partition NOT found !
[0000.478] Find Partition via GPT Failed
[0000.481] Find Partition via PT Failed
[0000.485] function NvTbootGetBinaryOffsets: 0x1 error
[0000.490] Error in NvTbootLoadBinary: 0x1 !
[0000.494] Error is 1

It might be too early to tell for sure but it appears to be a documentation bug. The actual boot flow for tx1 appears to match the flow documented in the tx2 page: cboot boots uboot boots linux.

In other words uboot is in the file called boot.img, written to partition LNX, and constructed using the command

./mkbootimg --kernel /path/to/uboot.img --ramdisk initrd --board ${target_rootdev} --output ${localbootfile} --cmdline "${cmdline}" > /dev/null 2>&1;

(around line 1076 of flash.sh)

If I change the file bootloader/t210ref/p2371-2180/u-boot.bin to the u-boot.bin that I build, then the system boots and the U-Boot line is

U-Boot 2016.07-gd35df3d (Sep 20 2017 - 09:18:25 -0700)

(which is different than the default flash).

hello josh_sky,

we should use below commands to update u-boot partition for R28.1/TX1, please confirm.

$ sudo ./flash.sh -k LNX jetson-tx1 mmcblk0p1

Hi JerryChang,

Almost. I think that since we are using a custom uboot build what you mean is.

sudo ./flash.sh -k LNX -K /path/to/my/u-boot.bin jetson-tx1 mmcblk0p1

Note that this seems to contradict the documentation which says:

In reality it seems that this partition contains U-Boot on T210 and T186 systems both.

hello josh_sky,

thanks for point out, we had revise this in the latest user guide.