Bootable SD Card > 16Gb for TK1?

imbs2 · February 15, 2017, 5:16pm

I am bringing up a new embedded board based on the Tegra TK1. It has a standard eMMC with 16Gb, but to conserve board space there is NO USB port for the flash cable. It DOES have a micro-SD slot… The plan is to boot from SD and install our custom L4T image from that. I have a Tegra-bootable micro SD that is 64Gb, but the standard nVidia flash/nvflash tools are hard-coded for 16Gb devices so only 16Gb of the SD card are being used. This SD card is flashed/prepared/tested on a ‘standard’ Jetson TK1 dev board, not the new board yet, until the SD card boot/prep works on a standard TK1.

What I want to do is expand the root (APP) partition on my bootable SD card to utilize the entire 64Gb available (less what the boot and hidden partitions require). Then I can put a compressed copy of our L4T image in the root partition and install it to eMMC with a script after booting from the SD card.

So far I have not had success. I have tried the following:

I can mount the SD card on a normal Ubuntu Linux machine and see the contents of the APP partition. The idea was to use gparted to expand the APP partition, but the hidden partitions come after the APP partition, are of unknown type, so gparted does not provide an option to move them.
Following on from 1) above, I thought I would just create an additional partition in the 48Gb of free space after the hidden partitions. Nope. gparted complains that there are too many primary partitions so I can’t create a new one.
New approach. Modify the jetson-tk1.conf file to indicate the device and APP partition sizes and use flash/nvflash. I increased the EMMCSIZE= from 15766388736 to 47223668736 and the ROOTFSSIZE= from 15032385536 to 46489665536. The difference in both cases is +30Gb. When I run sudo ./flash.sh jetson-tk1 mmcblk1p1 it builds a system image, reporting the following:

*** GPT Parameters ***
Device Sector Size ------- 512
device size -------------- 47223668736
bootpart size ------------ 8388608
userpart size ------------ 47215280128
Erase Block Size --------- 2097152
FS Buffer size ----------- 4096
Partition Config file ---- flash.cfg
Visible partition flag --- GP1
Primary GPT output ------- PPT->ppt.img
Secondary GPT output ----- GPT->gpt.img
Target device name ------- none

*** PARTITION LAYOUT(20 partitions) ***
[     BCT] BH            0        16383       8.0MiB 
[     PPT] UH            0         4095       2.0MiB ppt.img
[      PT] UH         4096         8191       2.0MiB 
[     EBT] UH         8192        16383       4.0MiB u-boot.bin
[     LNX] UH        16384        49151      16.0MiB 
[     SOS] UH        49152        61439       6.0MiB 
[     NVC] UH        61440        65535       2.0MiB 
[     MPB] UH        65536        77823       6.0MiB 
[     MBP] UH        77824        90111       6.0MiB 
[     GP1] UH        90112        94207       2.0MiB 
[     APP] UV        94208     90894335   44336.0MiB 
[     DTB] UV     90894336     90902527       4.0MiB tegra124-jetson_tk1-pm375-000-c00-00.dtb
[     EFI] UV     90902528     91033599      64.0MiB 
[     USP] UV     91033600     91041791       4.0MiB 
[     TP1] UV     91041792     91049983       4.0MiB 
[     TP2] UV     91049984     91058175       4.0MiB 
[     TP3] UV     91058176     91066367       4.0MiB 
[     WB0] UV     91066368     91070463       2.0MiB 
[     UDA] UV     91070464     92213247     558.0MiB 
[     GPT] UH     92213248     92217343       2.0MiB gpt.img
copying flasher(/opt/TK1/Linux_for_Tegra_tk1/bootloader/ardbeg/fastboot.bin)... done.
Existing flashapp(/opt/TK1/Linux_for_Tegra_tk1/bootloader/nvflash) reused.
*** Flashing target device started. ***
./nvflash  --bct PM375_Hynix_2GB_H5TC4G63AFR_H5TC4G63CFR_RDA_924MHz.cfg --setbct --configfile flash.cfg  --create --bl fastboot.bin --odmdata 0x6009C000 --go
Nvflash 4.13.0000 started
BR_CID: 0x34001001740de103180000000e050480
rcm version 0X400001
Skipping BoardID read at miniloader level
System Information:
   chip name: unknown
   chip id: 0x40 major: 1 minor: 1
   chip sku: 0x0
   chip uid: 0x00000001740de103180000000e050480
   macrovision: disabled
   hdcp: disabled
   jtag: disabled
   sbk burned: false
   board id: 0
   warranty fuse: 0
   dk burned: false
   boot device: emmc
   operating mode: 3
   device config strap: 0
   device config fuse: 0
   sdram config strap: 0

RCM communication completed
BCT sent successfully
sending file: tegra124-jetson_tk1-pm375-000-c00-00.dtb
- 59661/59661 bytes sent
tegra124-jetson_tk1-pm375-000-c00-00.dtb sent successfully
odm data: 0x6009c000
downloading bootloader -- load address: 0x83d88000 entry point: 0x83d88000
sending file: fastboot.bin
- 594363/594363 bytes sent
fastboot.bin sent successfully
waiting for bootloader to initialize
bootloader downloaded successfully
ML execution and Cpu Handover took 2 Secs
Partition backup took 0 Secs
setting device: 2 3
deleting device partitions
creating partition: BCT
creating partition: PPT
creating partition: PT
creating partition: EBT
creating partition: LNX
creating partition: SOS
creating partition: NVC
creating partition: MPB
creating partition: MBP
creating partition: GP1
creating partition: APP
creating partition: DTB
creating partition: EFI
creating partition: USP
creating partition: TP1
creating partition: TP2
creating partition: TP3
creating partition: WB0
creating partition: UDA
creating partition: GPT
failed executing command 17 NvError 0x120002
command failure/warning: create failed (bad data)
bootloader status: failed to create the partition (code: 10) message: nverror:0x9 (0x9) CreatePartition 1812 flags: 0

Failed flashing ardbeg.

That obviously didn’t work and then the eMMC wouldn’t boot, even with the SD card removed, so it appears it tried to flash the eMMC (mmcblk0) instead of the SD card (mmcblk1) and messed up the eMMC image. Very minor setback: I successfully reflashed the eMMC, so the TK1 is okay again.

How can I accomplish creating a bootable 64Gb SD card with ~62Gb APP partition??

linuxdev · February 15, 2017, 6:09pm

On the eMMC, have you tried booting to this, and then editing the extlinux.conf for a new “root=”? You can keep eMMC for use of “/boot” and just rename root to mmcblk1p1 instead of mmcblk0p1. If you do that then no other partitions need to be on the SD card other than just rootfs…the other hidden partitions remain on eMMC.

imbs2 · February 15, 2017, 7:09pm

I boot from the SD card the very first time the board is brought up. The eMMC has not been initialized yet and there is no USB recovery port, so I MUST boot from SD the first time and install to the eMMC from there.

In production there will be no SD card inserted in the board. The SD slot is strictly for bringing up a new board, which comes from the board manufacturer with an unformatted eMMC. Once an image has been successfully installed into the eMMC, the SD card is no longer needed and is removed, and the Tegra will boot from eMMC forever after that (unless it needs a new image).

The challenge is to be able to boot this first time from a bootable SD card that has a proper Tegra rootfs + boot PLUS an additional ~16Gb payload that is then installed on the eMMC from the SD card’s rootfs.

linuxdev · February 15, 2017, 8:40pm

That’s an interesting limitation. It would be best if you could do as you wanted by specifying over 16GB when flashing to the SD card on the host…I do not know what causes the error in your attempt to exceed 16GB (someone here may know why it is assumed by the flash software that the SD card is only 16GB and how to avoid that issue).

It would get quite complicated, but there may be ways to use dd between host and SD, then between SD and eMMC to accomplish what the flash software does. This would requiring having a complete bit-for-bit image for all of what you want on the eMMC. Individual partitions are not hard to do this with, especially for the rootfs…but the metadata at the start of eMMC is a different story. Hopefully you won’t need to go that route.

What it comes down to is that your larger SD card does not need to be partitioned for dd to read or write from it if you know the starting offset and keep track of that manually (dd can write bits to unpartitioned space, and then later read those bits for writing elsewhere). Keep in mind though that partitions are not the only part of the eMMC image…there is metadata at the start, and this too is needed for dd to do the complete job. If you have that metadata (basically GPT scheme) and binary images of each partition, then dd will do everything you want. The flash software does this as well, but I’m uncertain where the image would be for that metadata part (the flash software does not use gdisk over USB, it simply writes binary data…some of that data is computed rather than existing as static bits, and this is the part which is difficult).

imbs2 · February 15, 2017, 9:13pm

if you know the starting offset and keep track of that manually (dd can write bits to unpartitioned space
Eureka!! That would work. Thanks!!

I’d still like to be able to use the entire 64Gb in the rootfs when booting from SD, but you’ve provided a solution I can make work.