overlayfs for Jetson

Hello,

I tried to make root filesystem to read-only uinsg overlayfsroot,
but I could not make it works.

[Procedure]

  1. sudo apt install overlayroot
  2. echo ‘overlayroot=“tmpfs”’ | sudo tee -a /etc/overlayroot.local.conf
  3. sudo update-initramfs -c -k $(uname -r)
  4. reboot

As I check overlayfs for Jetson the following link, but no one can solve.
https://devtalk.nvidia.com/default/topic/1002630/jetson-tx1/tx1-using-overlayroot-for-read-only-filesystem/

Is threre any one who is able to use overlayfs successfully?

Our product is IoT product and
End-user may suddenly pick up power cable from Jetson Nano.
In that case, file system is broken and never boot up in worst case.
So we want to use read-only filesystem for Jetson Nano.

Best regards,

hello bdaddr,

your procedure looks you should create a custom initramfs,
suggest you check Topic 1032951, and also Using the initial RAM disk (initrd) for reference.

BTW,
just for your reference, there’s another discussion thread, linuxquestions.org talking about how to setup a readonly rootfs without an initramfs.
thanks

1 Like

bdaddr, if you succeed with this, could you share the steps required to make OverlayFS work on Jetson Nano?

I have spent a few hours now trying two approaches based on the linked posts and some other topics that I found in the forum, but to no avail so far:


I am tapping in the dark at this point…

I should add some context: First I tired to modify the l4t_initrd.img file inside Linux_for_Tegra/bootloader on the Host OS, and flash it back to the Nano. I wasn’t quite sure which partition I need to flash for this to be updated, after inspecting flash.xml I though EBT is the one:

sudo ./flash.sh -r -k EBT jetson-nano-qspi-sd mmcblk0p1

This worked, and I have successfully flashed other things to the Nano in the past, e.g. an updated BMP logo for startup. But it doesn’t look like my modified l4t-initrd.img is actually loaded.

I then also tried the normal Ubuntu approach, where I modify the initrd directly on the Nano, using update-initramfs -u. But this too didn’t seem to actually do anything.

Just after I wrote the above, I’ve figured it out myself, thanks to some hints in this repo https://github.com/JetsonHacksNano/rootOnUSB! So here it goes:

I found that the scripts from this project here, developed for the Raspberry Pi, works perfectly for the Nano: https://github.com/JasperE84/root-ro

The only thing that’s different is the installation procedure, so don’t run the install.sh script on the Nano:

sudo -i # root shell

# Change a few things in /etc/initramfs-tools so that overlayfs is used on startup to mount a ready-only root
echo overlay >> /etc/initramfs-tools/modules
wget https://raw.githubusercontent.com/JasperE84/root-ro/master/etc/initramfs-tools/hooks/root-ro -O /etc/initramfs-tools/hooks/root-ro
wget https://raw.githubusercontent.com/JasperE84/root-ro/master/etc/initramfs-tools/scripts/init-bottom/root-ro -O /etc/initramfs-tools/scripts/init-bottom/root-ro
chmod +x /etc/initramfs-tools/hooks/root-ro
chmod +x /etc/initramfs-tools/scripts/init-bottom/root-ro

# Update the initrd file
update-initramfs -u

# Now, the file /boot/initrd.img-4.9.140-tegra contains the above modifications.
# There is also the symbolic link in /boot/initrd.img pointing to it, but this doesn't
# actually get loaded on startup. By default /boot/initrd (without extension) is loaded.
# So change /boot/extlinux/extlinux.conf so that it actually gets loaded:
# Search for `INITRD /boot/initrd` and replace it with `INITRD /boot/initrd.img`:
if ! grep -q "INITRD /boot/initrd.img" /boot/extlinux/extlinux.conf; then sed -i "s/INITRD \/boot\/initrd/INITRD \/boot\/initrd.img/" /boot/extlinux/extlinux.conf; fi

# Before rebooting, also add the helper scripts from JasperE84/root-ro:
wget https://raw.githubusercontent.com/JasperE84/root-ro/master/reboot-to-readonly-mode.sh -O /root/reboot-ro
wget https://raw.githubusercontent.com/JasperE84/root-ro/master/reboot-to-writable-mode.sh -O /root/reboot-rw
chmod +x /root/reboot-ro
chmod +x /root/reboot-rw

Now you should be able to run sudo /root/reboot-ro to boot into RO mode, and sudo /root/reboot-rw to boot back into RW mode.

4 Likes

Hi,

The root-ro partition is rw by default on the Jetson, if you want to set it in read-only, you just need to add the following:

sudo sed -i ‘s/root=\/dev\/mmcblk0p1 rw/root=\/dev\/mmcblk0p1 ro/g’ /boot/extlinux/extlinux.conf

This doesn’t work for me. After having applied the changes the Nano boots continuously.

Perhaps try using my forked repo instead, which comes with an install-nano.sh script:

Ah, thank you very much. This will be my next attempt. Will let you know.

Hmm. No. The same situation. I must be missing something. What I did so far:

  1. Flash a fresh copy of the latest Nano Development Kit Image from the Nvidia site. JetPack SDK | NVIDIA Developer

  2. Perform a full installation. Finally issue “sudo apt update && sudo apt upgrade” at console level

  3. Clone your git, run the install-nano script

Result:

neil@jetson:~/root-ro$ sudo ./install-nano.sh 
dphys-swapfile is not installed, assuming we dont need to disable swap
Setting up maintenance scripts in /root...
Setting up initramfs-tools scripts...
Adding "overlay" to /etc/initramfs-tools/modules
Updating initramfs...
update-initramfs: Generating /boot/initrd.img-4.9.201-tegra
Warning: couldn't identify filesystem type for fsck hook, ignoring.
I: The initramfs will attempt to resume from /dev/zram3
I: (UUID=2804e605-4890-494d-8dbd-5f3af75759f3)
I: Set the RESUME variable to override this.
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_EGL.conf: No such file or directory
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_GL.conf: No such file or directory
Changing INITRD in /boot/extlinux/extlinux.conf
Removing the random seed file
Please restart your Jetson Nano now to boot into read-only mode
neil@jetson:~/root-ro$ 
  1. After reboot the Jetson is going into an endless reboot loop.

It is possible that this isn’t working with the latest version of JetPack… I was using JetPack 4.3 when I worked with this.

I could make an attempt with an earlier version. You also did use a pre-canned image? I would have a 4.4 image, could try that first

Good, tried it from a JP 4.4 installation. Same result. Going for 4.3 now

neil@jetson:~/root-ro$ sudo ./install-nano.sh

[sudo] password for neil:

dphys-swapfile is not installed, assuming we dont need to disable swap

Setting up maintenance scripts in /root...

Setting up initramfs-tools scripts...

Adding "overlay" to /etc/initramfs-tools/modules

Updating initramfs...

update-initramfs: Generating /boot/initrd.img-4.9.140-tegra

Warning: couldn't identify filesystem type for fsck hook, ignoring.

I: The initramfs will attempt to resume from /dev/zram3

I: (UUID=ab07f45d-daf6-4b8e-bebf-fe9b8e0e1800)

I: Set the RESUME variable to override this.

/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_EGL.conf: No such file or directory

/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_GL.conf: No such file or directory

Changing INITRD in /boot/extlinux/extlinux.conf

Removing the random seed file

Please restart your Jetson Nano now to boot into read-only mode

neil@jetson:~/root-ro$

Too bad. Jetson Download Center | NVIDIA Developer has only JP 4.4.1 as oldest :(

EDIT: Wait, found something Jetpack 4.3 Archive | NVIDIA Developer Jetpack 4.3 Archive | NVIDIA Developer

EDIT 2: OK, I can confirm it works with 4.3. At least it doesn’t endlessly reboot.

But correct me if I’m wrong: Shouldn’t I NOT be able to create a file in the root filesystem after rebooting RO?

I’m just asking because I’m able to create files after reboot. Wasn’t expecting this

@juerg Just to refresh my question: Shouldn’t a “RO” booted system refuse to create files? At least this is my understanding of read-only.

The OverlayFS approach creates a in-memory overlay over the read-only FS that holds all the modifications and acts as if the underlying FS is writable. Anything you write, modify or delete is persisted into memory only, and will be gone / restored after a reboot. This allows Linux to function normally without the need for any other modifications. But it could mean you’re running out of memory when the system is up for longer periods of time.

OK I was aware of this. But does that explain, that I found my newly created file again after a complete power cycle of the box? Does that make sense to you?

No that should not happen. Then the OverlayFS is not working as it should. Did you run the script to enable it?

sudo /root/reboot-ro

Yes, did that exactly after the initial setup (on a JP 4.3 image). Me was told, that the rootfs is already ro.

I will try again from scratch later the day.

Thanks

OK, checked several times now with the fresh JP 4.3 image: An attempt to set the system into permanent R/O ends up in “Is always in RO mode” message. Creating a file then. Shutting the system down. Remove power supply and all cables attached. Wait a minute.

After restart the file is there again :(

Are you sure your R/O mode wasn’t just an illusion?

Maybe it has an impact, this is what I’m seeing at reboot for a second:

Regards