In what follows I am assuming R28.1. Earlier releases had much in common, but you may see differences with regard to how USB speed is configured. At the end of this I also have a question which NVIDIA will need to answer.
Just FYI, USB3.0 is supported, though not in U-Boot…and USB3.1 is not supported, even in Linux…so USB3.0 would be a speed cap if you were not booting from the drive. Since you are booting from the drive you’ll have to verify it works with USB2 speeds. “lsusb -t” lists speeds of devices where “480M” implies USB2, and “5000M” implies USB3. Other than lower speed I do not believe there is any downside to storage over USB. Some USB3 devices need their full speed and do not have a USB2 compatibility mode, but hard drives do not have this limitation (some USB3 high resolution cameras do not provide a USB2 mode).
If you connect your drive and it shows as 5000M after a normal boot, then expect to lose the drive as a root partition until the port is forced back to USB2 mode.
USB3 in U-Boot would be difficult.
For SD card boot you would not need to worry about any of those restrictions.
Do you have serial console? This will make life much less painful while experimenting. U-Boot has a selection menu where you can pick which boot entry you want and have test entries such that a failure means you just boot the original entry instead…without serial console many failed tests would require a flash to boot again.
Some Information About Boot Targets…
When you flash a Jetson there are many hidden partitions going into the eMMC. Regardless of where your boot configuration is found (extlinux.conf), and regardless of where your root partition goes, the hidden partitions will remain on eMMC.
mmcblk0p1 is the first partition of eMMC, mmcblk1p1 is the first partition of SD card, sda1 is the first partition of the first hard drive. These are possible choices for rootfs, and also for where extlinux.conf is searched for. It is not mandatory that extlinux.conf and rootfs be on the same partition. I recommend keeping it simple and doing a normal flash to eMMC, and then editing extlinux.conf so that after configuration is read Linux will use the USB drive for rootfs.
Think closely about this: If you name mmcblk0p1 as a flash target, then both rootfs and extlinux.conf go on eMMC…but an edit to this extlinux.conf can cause the Linux kernel to boot to the USB drive. You can add a second entry in extlinux.conf (versus editing the existing entry), and suddenly serial console can be used to pick which one you want to boot to. It’s like having a built in rescue system. You can have a separate entry for eMMC, SD card, and sda1…all with the edit of a file and no flash required.
If you choose to flash to mmcblk1p1 (SD card) instead of mmcblk0p1 (eMMC), then the rootfs will still go to this alternate device as expected…what you won’t necessarily expect is that extlinux.conf must now be on the SD card or other media…eMMC will no longer have an extlinux.conf which matters (and in fact everything on the “/boot” of any file system other than the one used as the flash target will be ignored). If you remove the SD card or other media which had extlinux.conf, then the system will no longer be able to find configuration and will no longer boot. You can play around with the environment in U-Boot and have failover behavior, but this probably isn’t what you want. Just flash to mmcblk0p1 and then edit extlinux.conf for the simplest life. Make your rootfs on sda1 or mmcblk1p1 at a later time. If you want to you can make one of these external media a default entry, and then use serial console only when you want to try something new.
So for example this might be your “/boot/extlinux/extlinux.conf” on eMMC:
TIMEOUT 30
DEFAULT primary
MENU TITLE p2771-0000 eMMC boot options
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4
LABEL sdcard
MENU LABEL sdcard mmcblk1p1
LINUX /boot/Image
APPEND ${cbootargs} root=/dev/mmcblk1p1 rw rootwait rootfstype=ext4
LABEL sda1
MENU LABEL sda1
LINUX /boot/Image
APPEND ${cbootargs} root=/dev/sda1 rw rootwait rootfstype=ext4
If you interrupt boot early on you get to the U-Boot command line…this is too early…simply type “boot” to continue…then very quickly you get a kernel selection menu…hit any key (quickly) to stop boot from continuing. Enter the number of the entry. In the above you’d be able to pick “2” for SD card, or “3” for SATA. Beware that whatever key you hit to stop boot might still be there, so you might need to hit the backspace before entering the number.
Everything else is about creating the rootfs or setting up USB. If you flashed to mmcblk0p1, then you can do all of this at a later date.
About External Rootfs…
A typical procedure is to create an ext4 partition on the first partition of the USB drive. I did this by extracting Tegra_Linux_Sample-Root-Filesystem_R28.1.0_aarch64.tbz2 there (it was ext4 formatted and extraction done with sudo), then using the R28.1 apply_binaries.sh on it while it was mounted on “/mnt” of my host:
sudo ./apply_binaries.sh -r /mnt
This puts everything there except for “/boot” content…if you flashed to mmcblk0p1, then you don’t need “/boot” content.
You could also clone an existing install and copy to this partition from the clone.
USB Settings…
Without USB mode edits you will get part way through boot, and then sda1 will fail. This is an artifact of switching modes…you need sda1 to get the rootfs, and you need to lose the rootfs to get a new USB mode…a proverbial “chicken and the egg” dilemma.
This is somewhat more problematic on R28.1 since some of the settings were taken out of the kernel command line and instead put in the device tree. Device tree itself is somewhat more complicated because as of R28.1 the dtb isn’t provided in a file accessed by name in extlinux.conf…this is now in partitions (which as mentioned before will be on eMMC).
Mostly people ask how to make sure USB3 is enabled, but you need the inverse. Would someone at NVIDIA comment on what part of the device tree to change to cause the USB3 port to run only at USB2 mode? Without this booting on a USB device will fail.