Booting from NFS without flashing/recovery mode

Hi,

I want to make my Jetson AGX Xavier boot from NFS always, without having to switch to recovery mode every time (as i understand, that’s the standard way to NFS-boot the device, according to this page: https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/flashing.html).

Is there a standard way to go about this?

The options i’m thinking of so far are:

  1. Modifying the extlinux.conf to append nfs kernel command line arguments. (will CBoot support this?)
  2. Modifying the relevant DTBs to append the kernel command line arguments (it seems some people managed to do this here: Best Way to Change Boot Arguments on Xavier?).
  3. Building an initrd image that initializes the network device, mounts the nfs tree on /mnt, and switches rootfs into it (similarly to how meta-tegra’s tegra-minimal-initramfs switches root, except that the dest root is a network mount, and not the flash device).

I haven’t tried any of these options yet, so i’m not sure if it’s possible.
Has anyone done it before, or can add useful info if i were to try one of these options?

Thanks.
Eliran

If you just want to know whether cboot can read the kernel cmdline from extlinux.conf, the answer is yes.

  1. Modifying the extlinux.conf to append nfs kernel command line arguments. (will CBoot support this?)

Hi @WayneWWW, thanks for your reply!

Should CBoot also read extlinux.conf when booting from TFTP without NFS? (That is, when running the initrd from RAM)
It isn’t clear to me whether it should, according to the Nvidia boot flow guide.

I placed /boot/extlinux/extlinux.conf in the initrd, and booted from TFTPBoot, without NFS - just to check whether it will boot with the extra command line arguments specified in extlinux.conf.

It doesn’t look like CBoot tries to read extlinux.conf in this case (there’s no indication for that in the serial log), and also, when looking at the kernel command line arguments, they look unaltered.
I’ve attached the serial log of the boot (also printed the kernel arguments, and the /boot/extlinux/extlinux.conf from while running the initrd to make sure that it exists).
screenlog.0.txt (92.0 KB)

Thanks.

Hi,

You said “without having to switch to recovery mode every time”.
Actually, you have such behavior because you added “–rcm-boot” during flash, so each reboot would erase the content on RAM and it would fallback to emmc again.

You can try to use below commands to flash and the rootfs will be loaded from NFS server.
$ sudo ./flash.sh -N host_IP:/rootfs_location jetson-xavier eth0

Please note that the kernel is still on emmc for this setup only rootfs is coming from network.

Thanks @WayneWWW. That sounds like a good solution, I wasn’t aware of it.

Although I ended up with a solution of booting an initrd from TFTP boot, which mounts the NFS share, and executes ‘switch_root’ into the NFS mount in its init process.

This way, I can switch between NFS and emmc boot simply by enabling or disabling the TFTP server respectively.

Would you like t o share your setup steps for booting initrd from TFTP?

Other users may get benefit from it. Thanks.

Sure.

I’ll just mention that I’m using a Yocto distribution based on the meta-tegra layer, instead of Nvidia’s L4T Ubuntu. I’m using it both for the NFS-shared rootfs, and the Initramfs that boots from TFTP and switches root into that NFS share.

So, if someone were to try using an NFS share with a different OS, It should be theoretically possible, however i haven’t tried to switch_rootfs from a Yocto Initramfs to a non-Yocto OS, so it might require some small adjustments, depending on the OS on the NFS share.

So for the TFTPboot, i just configured the Xavier’s boot priority to ‘net’ (build the boot priority DTB according to this page , and write the DTB to the CPUBL-CFG partition according to this page).

Then, to prepare the Initramfs that switches into NFS, i built small Yocto image based on meta-tegra’s tegra-minimal-initramfs. Its init script was modified to this:

#!/bin/sh
# Mount the virtual filesystems
PATH=/sbin:/bin:/usr/sbin:/usr/bin
mount -t proc proc /proc
mount -t devtmpfs none /dev
mount -t sysfs sysfs /sys

# Activate network device
LOCAL_IP_ADDRESS=192.168.13.6
NFS_Host_IP_ADDRESS=192.168.13.10
echo "Activating network device with address ${LOCAL_IP_ADDRESS}..."
ifconfig eth0 $LOCAL_IP_ADDRESS up

# Wait for NFS server to be up
echo "Validating network connectivity by Pinging NFS host (${NFS_Host_IP_ADDRESS}) until an answer is received..."                                                                 
ping -W1 -c1 $NFS_Host_IP_ADDRESS
while [ $? -ne 0 ]; do ping -W1 -c1 $NFS_Host_IP_ADDRESS; done

# Mount NFS
echo "NFS host is pingable. Trying to mount the NFS share..."
rootdev="${NFS_Host_IP_ADDRESS}:/mnt/r"
opt="vers=3,nolock"
echo "Mounting ${rootdev}..."
mount -t nfs -o "$opt" "${rootdev}" /mnt || exec sh

# Switch rootfs
echo "Switching to rootfs on ${rootdev}..."
mount --move /sys  /mnt/sys
mount --move /proc /mnt/proc
mount --move /dev  /mnt/dev
exec switch_root /mnt /sbin/init

The image artifact is called tegra-minimal-initramfs-jetson-xavier.cpio.gz.cboot. It has to be signed, as follows (using the tegraflash.py script that is shipped with Nvidia’s SDK):

python tegraflash.py --chip 0x19 --cmd "sign tegra-minimal-initramfs-jetson-xavier.cpio.gz.cboot" --skipuid

Then it has to be placed in the TFTP directory, under the name boot.img, together with the ‘tegra194-p2888-0001-p2822-0000.dtb’ which is also shipped with Nvidia’s utils (both files are required for the CBoot to perform TFTPBoot).

And finally, for the NFS share, I used a Yocto’s core-image with the meta-tegra layer (to build a directory tree rootfs as an artifact, in addition to the tegraflash.zip artifact, add

IMAGE_TYPES += " rootfs.tar.gz"

to the local.conf file). But then again, any rootfs would work, assuming that it can be booted into from initramfs with the switch_root command.

With the above solution, I can switch between flash boot and NFS boot by toggling the TFTP server; That is, if the TFTP server is offline, then CBoot will fail to TFTPBoot and will default to the local boot.

2 Likes