Process for flashing tx1 modules

Three questions/issues related to flashing TX1 modules efficiently and reliably:

  1. Does nvidia recommend using the flash.sh and tegra_flash.py scripts that included from the driver package in a production environment? If so is there any documentation on these scripts beyond what is provided in the driver package tar file from https://developer.nvidia.com/embedded/linux-tegra.

  2. Our process currently uses flash.sh which calls tegra_flash.py from the “driver package” tarfile. We’ve only dug into this script a little bit to see all of what it’s doing to flash, but one of the things appears to be creating a system.img file from the rootfs root filesystem. This appears to be a rather lengthy process. We’d like to understand better what this process is. We expect to have some per-vehicle configuration that we want (e.g. hostname) so we will likely want to automate this. If I can better understand the rootfs -> system.img process we might be able to speed it up, rather than repeating some of the steps for every flash.

  3. Is there a recommended way to update the root filesystem with additional ubuntu packages. The driver package documentation recommends using “rootstock” but it appears that rootstock is deprecated and is no longer distributed by canonical with ubuntu. I have tried the following two things, but neither works as I expected:

a. chroot into the the root filesystem under arm emulation with qemu-arm-static (binding things like /proc/, /dev/, etc into the chroot), run apt-get install to get the packages we want, exit the chroot, run flash.sh.

b. Flash a devkit module with r23.2 unmodified. Login, then use apt-get to install additional packages. Create a backup of the root filesystem with:
tar xf Tegra210_Linux_R23.2.0_armhf.tbz2 --directory /mnt/data0/jetson_tx1/bsp_r23.2/
copy that file to my hostmachine and extract it into the rootfs dir:
sudo tar xpf backup.tar.gz --directory Linux_for_Tegra/rootfs/
then run flash.sh

Neither (a) nor (b) appears to work well. In (a) the X server does not start, there is a popup on boot indicating that the system is in low-graphics-mode and clicking through the popup to get a graphical session just gets me a black screen. In (b) the system boots but freezes at the lightdm login screen and the colors are funky (bluish/purplish) as if the display driver froze.

I can imagine that (b) might be unreliable due to package config scripts querying the system to make decisions, but (a) I would have thought would be completely consistent. If the only thing that is messed up is the X configuration then that is fine, we don’t really want to run X on our machines anyway, but I’m figuring that if X is messed up some other packages are likely to be messed up as well.

One alternative that I haven’t been able to try is to pull off a direct copy of the system.img. I don’t know if that is possible, using perhaps tegrarcm. If that is possible please let me know, though if we proceed with that route we will still want a way to alter the rootfs with per-vehicle configuration files.

Although these commands are for a Jetson TK1, this cloning article will probably be of use (information about a clone image is still correct on a JTX1):
http://elinux.org/Jetson/Cloning

For instructions on clone of a Jetson TX1:
https://devtalk.nvidia.com/default/topic/898999/jetson-tx1/tx1-r23-1-new-flash-structure-how-to-clone-/post/4784149/#4784149

The use of flash.sh from the driver package is the correct thing to do. The option you will be interested in is the “-r” option to reuse the image.

An important detail is that a clone is a “raw” image, but the final image normally used in flash is essentially a compressed version of the raw image (it is the “sparse” image where empty file system space is essentially stripped out and NULL byte run length noted). Historically, flash.sh used only the raw image, but sparse image was later added to decrease time to flash (the USB port was the slowest part of the flash)…flash should still work directly using a raw image, but command line tools can take a cloned raw image and create a sparse image from the raw clone image if desired.

The raw image can be loopback mounted:

sudo mount -o loop my_image.img.raw /mnt
# ...make edits or customize, e.g., copy files or use source code control software...

If you take your reference JTX1 and do all of the package updates on this and basically take this Jetson as close as possible to your end product state, and then clone, you can loopback mount and edit this clone on a host, followed by clone back into other Jetsons set to “reuse” the edited clone. I do this for not only backup and restore, but also between two Jetson TK1s such that ssh keys and network information remain constant and custom to the individual JTK1. I only have one JTX1, so I do not edit/clone/restore, but cloning so far as I have tested works the same on the JTX1.

It may be of use to know that the normal flash process builds the raw image from a combination of the sample rootfs and “/boot” related setup determined by your command line and content of the driver package “bootloader” subdirectory. Most of sample rootfs is copied verbatim if you exclude boot-related files (e.g., the zImage). As an example, I have different network configurations and ssh keys for different Jetsons, and it is possible to simply copy those files into a sample rootfs before flash to have those changes propagate. Normally though I would reuse a clone image and make edits to that instead. A large advantage of clone-edit-reuse is that there are files I do not know how to locate to do various things…especially setting up the GUI desktop. Should I just edit a sample rootfs I do not know how to customize the desktop settings…using a clone I simply set up the desktop then clone, and it propagates to future restores.

You also have the option to loopback mount a cloned raw image to the sample rootfs location and use the actual clone as your rootfs to generate a new image. Be warned though that if you do this and want to keep a pristine clone you’ll need to use a copy of the clone…sample rootfs is edited by flash.sh. If you want to see the differences between a cloned image and what flash-produces post-flash, you could compare a direct clone to how the clone appears after using it as a loopback mounted sample rootfs.

It is highly unlikely to matter for your case, but be aware that any loopback mount of an image (even read-only) will slightly alter the image file…this is because the journalling system keeps track of mount counts…the data on the file system would be constant, but the journal meta data changes.

Re: journalling: a read-only loopback mount will not affect the underlying filesystem. For extra assurance, you can user losetup with the -r option.

The current flashing mechanism used for tx1 can be used in production-- their rootfs image goes into the rootfs directory, ‘apply_binaries’ adds the NVIDIA binaries to the sample root file system, and the flash.sh command creates the image and flashes it. The details of the process can be found in ‘quick start guide’.

Subsequent flashing can reduce time substantially by preserving the created root file system image with the ‘-R’ option to flash.sh (read the script for details); but this does not address unique contents you may wish to flash-- that would currently require regeneration of the system.img.

It is possible to use current mechanism for flashing, then use the mass storage device support in U-Boot to reboot and expose the root file system as a USB mass storage device for modification by a flashing station. Please check the U-Boot documentation for details on MSD support.

Hello, Jijikos:
This can be resolved by different methods.
Here’s one suggestion.

  1. Flash Jetson, boot-up the board, install everything needed.
  2. Prepare another SATA HDD as a second root-fs device. Extract the original root-FS into this device and make sure that Jetson can boot from this device.
  3. Change the boot configuration to boot the Jetson from SATA HDD. Run ‘dd’ to get raw image from ‘/dev/mmcblk0p1’.
  4. That raw image can be used for further flash.

br
Chenjian