Hi All,
The NVIDIA Jetson TX2 Module datasheet mentions the Boot sources for the board as Internal eMMC and USB(Reset recovery).
Is there a way in which the Jetson TX2 board can be made to boot from an SD Card which has the boot image? Any config change needed to be done for this?
Please advise.
Thanks
M
The easiest wayâŠthe way which I prefer since it leaves the eMMC available for rescueâŠis to chain load. On the eMMC â/boot/extlinux/extlinux.confâ the APPEND key/value pair offers the command line the Linux kernel uses. In that the âroot=â determines where Linux will find its root partition (prior to that U-Boot determines where to look for extlinux.conf separately from where Linux will look).
âmmcblk0p1â is the first partition of eMMC; âmmcblk1p1â is the first partition of the SD card. If âroot=mmcblk1p1â, then Linux will look at SD card instead of eMMC (U-Boot will still require eMMC first partition to have extlinux.conf, and editing files in â/boot/â of SD will have no effect).
When manually flashing on command line you would normally use a command such as this:
sudo ./flash.sh -S 29318MiB jetson-tx2 mmcblk0p1
âŠif you instead name âmmcblk1p1â instead of âmmcblk0p1â, then U-Boot will look for extlinux.conf on SD card instead of eMMC. Note that there are differences in boot with a missing SD card when extlinux.conf is on eMMC versus when it is on the SD card, which is why I prefer to flash normally and then edit âroot=â (versus flashing to mmcblk1p1). In this latter scheme of flashing to mmcblk1p1 any edits to extlinux.conf on the eMMC will be ignoredâŠthe inverse of changing only the âroot=â.
During a manual command line flash you normally unpack the sample rootfs, and then run âsudo ./apply_binaries.shâ. If you use the âsudo ./apply_binaries.sh -r /some/where/elseâ, then the binaries go to the alternate location. You might use your PC to mount an SD card there, unpack sample rootfs onto it, and the apply_binaries.shâŠthis will give you a complete file system on the SD card which can then be used directly on the Jetson. Be sure the SD card is using ext4 on the first partition (GPT partitions are used, e.g., from gdisk). If for some reason your host uses 64-bit ext4 extensions you may need to mkfs.ext4 using options to disable those extensions.
If you use a serial console you can make multiple boot entries and pick which one you want at boot time. You can also drop into the U-Boot environment, run âhelpâ to see commands, and examine some of the variables which are boot macros used to determine boot device and order. Typing âbootâ would continue to a normal boot.
you may also use the kangalows jetsonhacks examples: Run Jetson TX1 from SD Card - JetsonHacks
Install Samsung SSD on NVIDIA Jetson TX2 - JetsonHacks
As I understand it the Jetsonhacks example is only about the Root filesystem, and it isnât about loading the kernel or anything else. All that will still be loaded from the eMMC.
Outside of flashing directly from a host machine I donât see anyway of accomplishing doing a complete boot from an SD card (u-boot, kernel, rootfs). Unless doing a drive image from the eMMC to the SD card works, but it doesnât seem like it would.
Iâm a little baffled why it isnât pretty standard to boot completely from the SD card where the eMMC is left alone as recovery/test. At least for development purposes because the Jetpack versions constantly evolves. That way the new jetpack version can be tested before committing to it.
It would also allow people to test out complete SD images that someone else creates. Like where I work we shipped TX2âs to multiple sites to have people test things out with. I should be able to save a compressed drive image on a network somewhere that any site with the TX2 can test out whatâs on the drive image. Without having any concern about whatâs on the eMMC. As everything is booted from the SD card.
When your flash target is mmcblk0p1 extlinux (and â/bootâ files) are read from eMMC. When the flash target is mmcblk1p1, then extlinux (and â/bootâ files) are read from SD card. U-Boot binary and some of its environment are stored in hidden partitions of eMMC regardless of root partition location.
To see partitions try this:
sudo gdisk -l /dev/mmcblk0
# OR:
sudo blkid
There is a possibility of cloning or writing individual partitions. The main caveat is that your partition data must fit in the size of the current partition size if writingâŠI donât think there is any guarantee the same size partition will be valid across releases. Second, not all U-Boot or hidden partition content will be compatible across releases. So clone and restore within a single release version (such as R28.1) should be fine, but mixing releases (such as R27.1, R24.2.1, and R28.1) will not necessarily work.
On a prior release for the TK1 there was an alias for the entire eMMC, âallâ. It would be nice if newer releases were to add this ability back in (subtle feature request hint! :). However, each individual image concatenated in the correct order is the same as âallâ. When cloning on a TK1 with âallâ you can use dd at the right offsets and get a subset of the image which corresponds exactly to cloning just that one partitionâŠyou can go back and forth between concatenating individual partitions or taking subsets of multiple partitions and be guaranteed of exactly the same results.
If you want to put all hidden partitions on SD card I donât believe this is possible. You could of course put those partitions on SD card, and while booted to SD card, use those partition for dd replication onto eMMCâŠthose partitions though would not be active until you reboot since the SD card hidden partitions would basically be just data and not participate in the boot process.
Testing a root partition (APP partition) from SD card is easy, just change the extlinux.conf on eMMC after flash to mmcblk0p1 to âroot=/dev/mmcblk1p1â, or if using SD card boot partition, then changing to this on the SD card version of extlinux.conf (the extlinux.conf on SD card could be named during flash, yet have it point at eMMC instead of itselfâŠthe corollary of being on eMMC and pointing at SD card is interchangeable).
Additional note: U-Boot environment variables can be set for a search order. One can say search first for extlinux.conf on eMMC, then other devices; or you could alter the variables to search first on SD card, and if not there, only then boot to eMMC version. Much of the boot behavior is customizable from the serial console U-Boot shell.
Here is what I did.
I followed the jetson hacks guide to create the rootfs on the SD card. This didnât really give me what I wanted because as soon as I stuck the SD card into a new TX2 it didnât boot. Which was expected, but I did it just to verify.
So then I flashed from the host to the TX2 (in recovery mode) to mmcblk1p1. That didnât actually seem to flash to the SD card at all. In fact that same rootfs that was on there was still there, along with no other partitions. I did this before I saw your comment, and your comment explains this.
To be perfectly clear is it possible to boot COMPLETELY from the SD card? Where I can actually delete the eMMC?
Is this R28.1 (check with âhead -n 1 /etc/nv_tegra_releaseâ)? Does the SD card have â/boot/extlinux/extlinux.confâ, and if so, what is its content?
The answer to COMPLETELY booting from SD is no, it cannot. The boot loader and environment remain on eMMC regardless of which media the boot loader looks to for further configuration. Flashing to mmcblk1p1 changes extlinux.conf and whether it points to eMMCâs version of extlinux.conf or whether it points to the SD cardâs extlinux.conf. Boot loader binary remains on eMMC. Only the root partition of eMMC can be deleted.
Correct, itâs R28.1 and the extlinux.conf in the SD card was already setup to boot off the SD.
Thanks for the clarification on the booting from the SD. That confirms what this document says in that you canât put u-boot anywhere, but in the eMMC.
Iâm not sure why they made that decision because if the eMMC ever becomes corrupt there is no way to boot the unit with exception to the USB recovery mode. Thatâs fine on the development system, but not production systems.
So did you sucessfully boot Jetson TX2 R28.1 from external SD card om the end?
I am trying to use flash.sh file but it doesnât workâŠ
Flash to alternate boot targets works, but beware youâll need to set up the rootfs on the SD card separately. Flash to eMMC puts the rootfs in place, but flash to a non-eMMC target simply places the boot loader setup such that it will look at the SD card when the time is right. No rootfs on the SD card implies it canât boot. Have you looked to see if the first partition of this device has an ext4 mountable partition on it?
Yes, actually I am trying to flash it onto an alternative usb drive. And the first partition is ext4.
But how to setup rootfs on my external drive?
And I have folders required (/bootloader, /kernel, /rootfs, /nv_tegra) in the repository: JetPack3.1/64_TX1/Linux_for_Tegra_64_tx1
total 132K
drwxr-xr-x 6 yerong yerong 4.0K Dec 24 00:45 ./
drwxrwxr-x 3 yerong yerong 4.0K Dec 23 23:55 ../
-rwxr-xr-x 1 yerong yerong 18K Jul 20 02:45 apply_binaries.sh*
drwxr-xr-x 3 yerong yerong 4.0K Dec 24 01:40 bootloader/
-rw-r--r-- 1 yerong yerong 2.7K Jul 20 02:45 e2220-1170.conf
-rwxr-xr-x 1 yerong yerong 2.5K Jul 20 02:45 elf-get-entry.py*
-rwxr-xr-x 1 yerong yerong 51K Jul 20 02:45 flash.sh*
lrwxrwxrwx 1 yerong yerong 22 Jul 20 02:45 jetson-tx1.conf -> p2371-2180-devkit.conf
drwxr-xr-x 3 yerong yerong 4.0K Jul 20 02:46 kernel/
drwxr-xr-x 3 yerong yerong 4.0K Jul 20 02:46 nv_tegra/
-rw-r--r-- 1 yerong yerong 2.9K Jul 20 02:45 p2371-0000.conf
-rw-r--r-- 1 yerong yerong 2.8K Jul 20 02:45 p2371-2180.conf
-rw-r--r-- 1 yerong yerong 2.9K Jul 20 02:45 p2371-2180-devkit-24x7.conf
-rw-r--r-- 1 yerong yerong 2.9K Jul 20 02:45 p2371-2180-devkit.conf
drwxr-xr-x 21 yerong yerong 4.0K Dec 23 23:57 rootfs/
-rwxr-xr-x 1 yerong yerong 9.3K Jul 20 02:45 source_sync.sh*
Those directories are on the host. The content which is in rootfs needs to be copied into the root of the SD cardâs first partition. Had this been to eMMC, then flash would have done that copy for you.
Normally rootfs is constructed by unpacking the sample rootfs into it as root, and then using the apply_binaries.sh script to copy the hardware-accelerated version of libraries into this as root. Lastly, the flash.sh script itself adds content to the ârootfs/boot/â directory depending on what your flash options were. If that content was copied there, then all you need to do is recursively copy rootfs (with sudo) into that first partition.
Do you have file ârootfs/boot/extlinux/extlinux.confâ? If so, then it likely has a âroot=/dev/sda1â entry for the case of having flashed to target sda1. Had there never been a flash, then extlinux.conf wouldnât be there; had flash been to target mmcblk0p1, then it would have âroot=/dev/mmcblk0p1â.
NOTE: Many distributions do not use 64-bit ext4 extensions. Ubuntu 14.04 and 16.04 are in this category. If you use one of the more recent distributions, and if those extensions are enabled, then U-Boot will not be able to see the file system even if you did everything correctly. If your host file â/etc/mke2fs.confâ shows â64bitâ or âmetadata_csumâ, then you know the default ext4 wonât be valid for your Jetson due to U-Boot. You can still mkfs.ext4 from the host, but youâd need to pass options â-O ^metadata_csum,^64bitâ to mkfs.ext4 to tell it to not use those options.
I am wondering what are the steps for flashing the OS directly to sda1 on the tx1 side?
My rootfs/boot/extlinux/extlinux.conf shows âmmcblk0p1â:
TIMEOUT 30
DEFAULT primary
MENU TITLE p2371-2180 eMMC boot options
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait
When I run the command:
sudo ./flash.sh jetson-tx1 mmcblk0p1
I can directly flash the OS to eMMC as normal installation.
Yes, you are right, if I run the command:
sudo ./flash.sh jetson-tx1 mmcblk0p1
on the desktop side in the repository rootfs, the /boot/extlinux/extlinux.conf reads
TIMEOUT 30
DEFAULT primary
MENU TITLE p2371-2180 eMMC boot options
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait
and if I run the command
sudo ./flash.sh jetson-tx1 sda1
extlinux.conf in rootfs(on desktop side) reads
TIMEOUT 30
DEFAULT primary
MENU TITLE p2371-2180 USB boot options
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} root=/dev/sda1 rw rootwait
Here is what I did: I firstly copied the system.img.raw to the thumb drive on the desktop side.
Then I run the flash.sh with the thumb drive pluged on the Jetson side.
sudo ./flash.sh jetson-tx1 sda1
And it still doesnât work saying that Jetson cannot configure graphic driver properly.
As a summary:
[First try]
1.Copy system.img.raw to the thumb drive on the desktop;
2.Flash OS to eMMC
3.Edit extlinux.conf on Jetson eMMC to sda1
Gives output
random: nonblocking pool is initialized
cannot find fireware....retry after 1 second
Direct firemware load for tergra21x_xusb_firemware failed with error -2
Falling back to user helper
cannot find fireware....retry after 1 second
Direct firemware load for tergra21x_xusb_firemware failed with error -2
Falling back to user helper
cannot find fireware....retry after 1 second
Direct firemware load for tergra21x_xusb_firemware failed with error -2
Falling back to user helper
cannot find fireware....retry after 1 second
Direct firemware load for tergra21x_xusb_firemware failed with error -2
Falling back to user helper
[Second try]
1.Copy system.img.raw to the thumb drive on the desktop side;
2.On the desktop side, run flash.sh with thumb drive inserted on Jetson (and with the micro-USB connecting Jetson and desktop);
sudo ./flash.sh jetson-tx1 sda1
Gives pop-out error
Cannot configure graphics driver properly and Jetson asked me to do the configuration manually.
External Media
And after I âOKâ, Jetson asked me âwhat would you like to do?â
External Media
Then I could not press arrow keys or press the enter key but press ESC and the low graphics error repeats over and over again.
[/quote]
One thing I have forgotten to checkâŠif the USB port used for the flash drive changes between USB2 mode and USB3 mode during boot (the controller, not the device), then that USB port cannot be used as a rootfs device. U-Boot only understands USB2, and the full-sized USB port will switch to USB3 mode after boot, but the micro-USB port will stay USB2. Are you using the micro-USB port during boot? If not, then try that out.
Iâm not sure what those GUI displays are from, but it would seem to be a case of missing the correct driversâŠwhich in turn could be an install issue, or a rootfs setup issue, or perhaps losing the USB port during boot (which implies a loss of reading drivers from disk).
I think I succeed in booting R24.1 (JetPack 2.3) from my thumb drive previously. I hope this indicates that my USB port and thumb drive should work on Jetson (TX1) board.
I believe by default R24.1 has both USB ports in USB2 modeâŠthis would cause continuity of service for any rootfs device over USB and work well. Newer L4T may have a port which is USB2 in U-Boot and then switch to USB3 under LinuxâŠthis will break boot of a rootfs device over that USB port since it momentarily drops access to the device until it re-enumerates.
I found this is possibly happening in my case:
I tried to put /usr/local in my thumb drive and editing fstab as
UUID=178f9428-9292-4024-8011-93fafb90f17c /usr/local ext4 noauto,user,exec,owner,exec 0 0
And reboot the system and the thumb drive does not mount automatically.
The /usr/local looks as this at the beginning
nvidia@tegra-ubuntu:/usr/local$ ll
total 8
drwxr-xr-x 2 root root 4096 Dec 27 07:46 ./
drwxr-xr-x 11 root root 4096 May 3 2016 ../
It was after I mounted the thumb drive manually through the Disk Utility, the /usr/local looks normal:
nvidia@tegra-ubuntu:/usr/local$ ll
total 59388
drwx------ 16 nvidia nvidia 4096 Dec 27 07:45 ./
drwxr-xr-x 11 root root 4096 May 3 2016 ../
drwxr-xr-x 2 root root 4096 Apr 20 2016 bin/
lrwxrwxrwx 1 root root 8 Dec 27 07:40 cuda -> cuda-8.0/
drwxr-xr-x 11 root root 4096 Dec 27 07:40 cuda-8.0/
drwxr-xr-x 2 root root 4096 Apr 20 2016 etc/
drwxr-xr-x 2 root root 4096 Apr 20 2016 games/
-rw-rw-r-- 1 nvidia nvidia 1595408 Nov 6 2016 get-pip.py
drwxr-xr-x 2 root root 4096 Apr 20 2016 include/
drwxr-xr-x 4 root root 4096 May 3 2016 lib/
drwx------ 2 root root 16384 May 6 2016 lost+found/
lrwxrwxrwx 1 root root 9 Apr 20 2016 man -> share/man/
-rw-rw-r-- 1 nvidia nvidia 1335034 Dec 26 06:58 numba-0.36.2.tar.gz
drwxrwxr-x 2 nvidia nvidia 4096 Dec 26 07:20 openssl/
drwxrwxr-x 22 nvidia nvidia 4096 Dec 26 07:53 openssl-master/
-rw-rw-r-- 1 nvidia nvidia 17070797 Dec 26 07:16 openssl-master.zip
drwxrwxr-x 16 nvidia nvidia 4096 Dec 26 07:59 Python-3.5.1/
-rw-rw-r-- 1 nvidia nvidia 20143759 Dec 26 07:01 Python-3.5.1.tgz
-rw-rw-r-- 1 nvidia nvidia 20566643 Dec 26 06:50 Python-3.5.2.tgz
drwxr-xr-x 2 root root 4096 Apr 20 2016 sbin/
drwxr-xr-x 8 root root 4096 May 3 2016 share/
drwxr-xr-x 2 root root 4096 Apr 20 2016 src/
drwx------ 4 nvidia nvidia 4096 Dec 26 07:35 .Trash-1001/
Do you know how to fix the port issue here?
The ânoautoâ in fstab says to not auto-mount. Basically youâd still need to âsudo mount /usr/localâ, and if that UUID is present, then it will mount.
If you say to always mount, and if the SD card isnât there, then it will hang and never finish booting. Since you are specifying by UUID, then this is the only SD card which will satisfy the requirement unless you manually force this UUID onto other SD cards. If for whatever reason you forget this UUID and lose the SD card, then your system will likely be unbootable beyond the point of where it mounts file systems (youâd have to test to be sureâŠbut test on one you can afford to lose, e.g., after cloning).
Iâm not sure where you would do it, but udev should be able to see a disk with this exact UUID and auto mount on â/usr/localâ when the disk is detected as present. You might look around in â/etc/udev/rules.d/â. Note that the system places its default rules in â/lib/udev/rules.d/â and that if the same file name is seen in â/etc/udev/rules.d/â, then the â/etcâ version is used. You could put a symbolic link in the â/etcâ version pointing at the â/libâ version and it would essentially have no effectâŠor you could copy a file from the â/libâ version, put it in the â/etcâ area, and then edit this fileâŠthe rule would be changed until you remove the â/etcâ version. Itâs a way to test without messing up system files.