Creating a custom rootfs

The documentation for L4T recommends this page for creating a custom ubuntu rootfs, which describes using rootstock. It seems that the rootstock tool is deprecated. We have found that using multistrap is a relatively easy alternative that works.

Here is the process we are using to create a rootfs using multistrap. We create the rootfs on an sdcard, move it to an existing devkit to chroot in and run dpkg --configure. Then move the sdcard back to the host system to tar-it up and deploy.

Note that multistrap distributed with ubuntu 14.04 is broken. The fix is simple and is in comment #6. The patch is at the bottom of this post.

On the host system:

# Mount an sdcard (already formatted with something like ext2 or ext4)
sudo mkdir /mnt/sdcard
sudo mount /dev/sdaX /mnt/sdcard

# Run multistrap to generate the rootfs
sudo multistrap --file /path/to/multistrap.conf

# Tweak the filesystem to assist a few package installs
# Create a symlink for insserv
sudo ln -s ../usr/lib/insserv/insserv /mnt/sdcard/sbin/insserv

# Create a symlink for awk
sudo ln -s mawk /mnt/sdcard/usr/bin/awk

# Unmount and eject the sdcard
sudo umount /mnt/sdcard

On an existing tx1 devkit

# Insert and mount the sdcard
sudo mkdir /mnt/sdcard
sudo mount /dev/mmcblk1p1 /mnt/sdcard

# Copy a passwd and group file onto the filesystem so we don't get weird errors
# and warnings. You don't really need this step, and you probably want to
# strip down the passwd and group file to suit your needs.

# NOTE: The name of the file is 'passwd'. There seems to be a forum bug that
# prevents this post if I use 'passwd' in the line below:
sudo cp /etc/passw /mnt/sdcard/etc/
sudo cp /etc/group /mnt/sdcard/etc/

# If you are installing java, you'll need /proc mounted inside the chroot.
sudo mount -t proc /proc /mnt/sdcard/proc

# Configure packages. Note the little hack to make dash happy.
sudo chroot /mnt/sdcard /var/lib/dpkg/info/dash.preinst install
sudo chroot /mnt/sdcard dpkg --configure -a

#If you installed any packages that configure a daemon, they may have started
#inside the chroot. Find out their pid with `lsof /mnt/sdcard` and kill them.

#Unmount and eject sdcard
sudo umount /mnt/sdcard/proc
sudo umount /mnt/sdcard

Back on the host system

# Insert and mount the sdcard again. You can mount the sdcard directly to
# Linux_for_Tegra/rootfs and run flash.sh to test the root filesystem. Or you
# can create a tarfile for deployment in a fashion similar to jetpack.
sudo mount /dev/sdaX /mnt/sdcard
cd /mnt/sdcard
sudo tar cpzf /path/to/custom_rootfs.tar.gz *

Here is an example multistrap config file. Note that multistrap will not
install /etc/apt/sources.list so you’ll probably want to deploy one of those.
See the one in the sample rootfs for a sample.

[General]
# arch and directory can be specified on the command line.
arch=armhf
directory=/mnt/sdcard
# same as --tidy-up option if set to true
cleanup=true
# retain the sources outside the rootfs for distribution
# specify a directory to which all the .debs can be moved.
# or override with the --source-dir option.
# retainsources=/path/to/dep/cache
# same as --no-auth option if set to true
# keyring packages listed in each debootstrap will
# still be installed.
noauth=true
# extract all downloaded archives
unpack=true
# the order of sections is no longer important.
# debootstrap determines which repository is used to
# calculate the list of Priority: required packages
debootstrap=Trusty
# the order of sections is no longer important.
# aptsources is a list of sections to be listed
# in the /etc/apt/sources.list.d/multistrap.sources.list
# of the target.
aptsources=Trusty,TrustyUpdates
#configscript=config.sh
#setupscript=setup.sh

# Any packages installed from trusty
[Trusty]
packages=apt at acpid isc-dhcp-client nano openssh-server net-tools python rsyslog sudo tmux udev ubuntu-keyring
source=http://www.ports.ubuntu.com/ubuntu-ports
keyring=ubuntu-keyring
suite=trusty
components=main universe multiverse
omitdebsrc=true

# Any packages installed from trusty-updates
[TrustyUpdates]
packages=clang-format-3.6
source=http://www.ports.ubuntu.com/ubuntu-ports
keyring=ubuntu-keyring
suite=trusty-updates
components=main universe multiverse
omitdebsrc=true

Here is the diff to fix multistrap on ubuntu 14.04:

--- /usr/sbin/multistrap    2014-02-21 23:27:42.000000000 -0800
+++ /tmp/multistrap 2016-05-16 08:29:44.666249427 -0700
@@ -986,7 +992,7 @@
    }
    # reinstall set
    foreach my $reinst (sort @reinstall) {
-       system ("$str $env chroot $dir apt-get --reinstall -y $forceyes install $reinst");
+       system ("$str $env chroot $dir apt-get --reinstall -y install $reinst");
    }
    &run_native_hooks_end(sort @{$hooks{'N'}}) if (defined $hooks{'N'});
    return $retval;

We’ve also patched multistrap to support the use of apt-cacher to speed up the
process of building a rootfs. We run apt-cacher on a machine in our office and
have added the ‘–apt-cacher’ command line option to multistrap.

--- /usr/sbin/multistrap    2014-02-21 23:27:42.000000000 -0800
+++ /tmp/multistrap 2016-05-16 08:29:44.666249427 -0700
@@ -36,7 +36,7 @@
  $explicit_suite $allow_recommends %omitdebsrc @dsclist @sectoutput
  %flatfile %important $addimportant @debconf $hookdir %hooks
  $warn_count $use_shortcut @foreignarches $olddpkg $ignorenative
- %foreignpkgs $markauto $default_release /;
+ %foreignpkgs $markauto $default_release $apt_cacher /;
 
 setlocale(LC_MESSAGES, "");
 textdomain("multistrap");
@@ -74,6 +74,8 @@
        $noauth++;
    } elsif (/^(--dry-run|--simulate)$/) {
        $dryrun++;
+   } elsif (/^(--apt-cacher)$/) {
+       $apt_cacher = shift(@ARGV);
    } else {
        die "$progname: "._g("Unknown option")." $_.\n";
    }
@@ -343,6 +345,10 @@
 $config_str .= " -o Dir::Etc::Parts=${dir}${etcdir}apt.conf.d/";
 $config_str .= " -o Dir::Etc::PreferencesParts=${dir}${etcdir}preferences.d/";
 $config_str .= " -o APT::Default-Release=$default_release";
+if (defined $apt_cacher) {
+  $config_str .= " -o APT::http::Proxy=${apt_cacher}";
+  $config_str .= " -o APT::https::Proxy=false";
+}
 #  if (not defined $preffile);
 if (defined $deflist) {
    $sourcesname = "sources.list.d/multistrap.sources.list";

Thank for sharing this info.

We also created a wiki page to guide to generating the sample root filesystem, please see the further information:
[url]http://elinux.org/Jetson/TX1_Sample_Root_Filesystem[/url]

Hello,

I tried to generate an Ubuntu root filesystem by following the tutorial on elinux.org.
Unfortunately, the link to the ubuntu-core arm64 archive is broken. I searched the internet for the ubuntu-core arm64 archive, but I couldn’t find it. Do you actually use another ubuntu archive or rootfs generation method to create the L4T sample root fs?

Thank you in advance
Christian

Any hint on how to get the ubuntu-core arm64 archive?

I built Ubuntu Base images for 16.04 on the TX2 should also work on the TX1. I created a page on eLinux.org here detailng the steps and posted some example quickstart images over on github:

I’m surprised more people are not using Ubuntu Base or building their own filesystems from scratch using Debootstrap to strip out all the Desktop guff in the nVidia Sample filesystem.

More info on ubuntu base here Base - Ubuntu Wiki (used to be called ubuntu core).

Excellent work! Nice to see this experience sharing and posted on wiki.

Hi

I am using multistrap to create a rootfs from scratch, After to create the file system, it has been configured using NVidia scripts from Tegra210_Linux_R28.1.0_aarch64.tbz2, The steps applied are:

  1. Create filesystem (Xenial) using multistrap
  2. Install binaries (apply_binaries.sh)
  3. Flash filesystem (./flash.sh)

When TX1 board up I am getting this error when is applied the automatic login.

. . .
[   15.116587] dhd_prot_ioctl : bus is down. we have nothing to do
[   15.116749] sdhci-tegra sdhci-tegra.1: Tuning already done, restoring the best tap value : 44
[   15.117810] wifi_platform_set_power = 0

Ubuntu 16.04 LTS localhost ttyS0

localhost login: ubuntu (automatic login)

login: PAM Fa
Ubuntu 16.04 LTS localhost ttyS0

localhost login: ubuntu (automatic login)

login: PAM F
Ubuntu 16.04 LTS localhost ttyS0

localhost login: ubuntu (automatic login)

login: PAM Failure, a
Ubuntu 16.04 LTS localhost ttyS0

localhost login: ubuntu (automatic login)

login: PAM Failu
Ubuntu 16.04 LTS localhost ttyS0

localhost login: ubuntu (automatic login)

Do you know what could be the cause of this error (login: PAM Failure)?

Yeah, there is a file that is written as part of apply_binaries.sh. It’s at etc/systemd/system/serial-getty@ttyS0.service.d/autologin.conf. This file sets up the terminal server on ttyS0 to automatically login as the user ubuntu. If you used multistrap to create your rootfs you probably don’t have such a user. You’ll have to add the ubuntu user to /etc/passwd, /etc/group, and /etc/shadow. You could also copy those three files from the jetpack rootfs to your multistrap rootfs. Or, you could change the getty@ttyS0.service.d/autologin.conf to login as a different user.

josh6uk9d thanks for the fast response.

I already had tried to fix the problem copying these files from Nvidia filesystem (JetPack-L4T-3.1/64_TX1/Linux_for_Tegra_64_tx1/rootfs) to multistrap filesystem

/etc/passwd*
/etc/shadow
/etc/group
/etc/pam.d/*

but after the change I still having the same problem.

Also I have applied your suggestion, to change the file /etc/systemd/system/serial-getty@ttyS0.service.d/autologin.conf to login as a different user.

This was the original content

[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin ubuntu --keep-baud 115200,38400,9600 %I $TERM

Then I edited the user information to use ‘nvidia’ instead of ‘ubuntu’

[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin nvidia --keep-baud 115200,38400,9600 %I $TERM

But this change doesn’t solve the problem either.

If you have another suggestion please let me know. I appreciate your help.

Thanks.

I suggest that you remove the file autologin.conf in it’s entirety. If I recall correctly that should leave you with a login prompt instead of trying to login automatically. Once you get that working, you can start building on top of that.

Hi,

When the rootfs is created, it does contain the files /etc/passwd, /etc/shadow, /etc/group
I have remove the file ‘autologin.conf’ but in this case I don’t know what is the user that I should use. I have tried with some common users names but it doesn’t work:

Ubuntu 16.04 LTS localhost ttyS0

localhost login: 
Ubuntu 16.04 LTS localhost ttyS0

localhost login: root
log
Ubuntu 16.04 LTS localhost ttyS0

localhost login: localhost
log
Ubuntu 16.04 LTS localhost ttyS0

Also I have tried adding the files /etc/passwd, /etc/shadow, /etc/group from Nvidia rootfs and removing the file autologin.conf but in this case I am getting the ‘PAM Failure’ when I have tried to loggin using ubuntu or nvidia users

Ubuntu 16.04 LTS localhost ttyS0

localhost login: ubuntu
login: PAM Failure, abortin
Ubuntu 16.04 LTS localhost ttyS0

localhost login: nvidia
login: PAM Failu
Ubuntu 16.04 LTS localhost ttyS0

localhost login: 
Ubuntu 16.04 LTS localhost ttyS0

localhost login:

I have seen that tools like rootstock has a parameter to define an user (–login, password)
but I have not found something equivalent for multistrap.

I am wondering, How to loggin multistrap files system?

In Ubuntu root can never log in without special steps.

Some related files are needed in addition to passwd, group, and shadow. Not all of these are technically required (the “-” versions), but check again after adding the rootfs files from “/etc/”:

group-
<b>gshadow</b>
gshadow-
shadow-

User “ubuntu” would be a good test login.

I flashed Tegra_Linux_Sample-Root-Filesystem_Debootstrap_cmdline_aarch64.tbz2 to TX2 Demo Board,
But when I use the command “mount /dev/sda1 /mnt” to access udisk,
it give me following message:
mount:unknow filesystem type ‘vfat’

VFAT is an older Windows filesystem type, typically found on preformatted SD cards. You’ll want to use type ext4 unless you have some unusual kind of system. However, it isn’t unusual to want to use VFAT on an SD card even if the rootfs itself is ext4. In this latter case you would need the kernel to have support for VFAT, but first you’ll want to make sure the flashed filesystem is correct before you try to add VFAT support.

For the PC used for flashing the system, I am guessing your disk storage was on removable media and not a native Linux installation. If you examine the PC, then what do you see from “df -H -T /wherever/you/flashed/from/Linux_for_Tegra” (or replace “/wherever/you/flashed/from/Linux_for_Tegra” with “~/nvidia/nvidia_sdk”)? If VFAT shows up, and not ext4, then this is why.

Hi newmanross,

I just flashed Tegra_Linux_Sample-Root-Filesystem_Debootstrap_lxde_aarch64.tbz2 to TX2 Eval Board(Based Jetpack3.3 R28.2.1).
I just unzip Tegra_Linux_Sample-Root-Filesystem_Debootstrap_lxde_aarch64.tbz2 onto Linux_for_Tegra/rootfs,
then run the ./apply_binaries.sh to copy nvidia related file onto rootfs.
then run flash.sh jetson-TX2 mmcblk0p1.
It is working fine and boot to LXDE desktop.
Just get one issues, I don't see wlan0 wifi device by "ifconfig"
Can you let me know how to enable built-in Wifi Module?

Also, How to check if the wifi driver is running correctly? 

Thanks,
Martin