overlayfs for Jetson

Hi,

The root-ro partition is rw by default on the Jetson, if you want to set it in read-only, you just need to add the following:

sudo sed -i ‘s/root=\/dev\/mmcblk0p1 rw/root=\/dev\/mmcblk0p1 ro/g’ /boot/extlinux/extlinux.conf

1 Like

This doesn’t work for me. After having applied the changes the Nano boots continuously.

Perhaps try using my forked repo instead, which comes with an install-nano.sh script:

Ah, thank you very much. This will be my next attempt. Will let you know.

Hmm. No. The same situation. I must be missing something. What I did so far:

  1. Flash a fresh copy of the latest Nano Development Kit Image from the Nvidia site. https://developer.nvidia.com/embedded/jetpack#install

  2. Perform a full installation. Finally issue “sudo apt update && sudo apt upgrade” at console level

  3. Clone your git, run the install-nano script

Result:

neil@jetson:~/root-ro$ sudo ./install-nano.sh 
dphys-swapfile is not installed, assuming we dont need to disable swap
Setting up maintenance scripts in /root...
Setting up initramfs-tools scripts...
Adding "overlay" to /etc/initramfs-tools/modules
Updating initramfs...
update-initramfs: Generating /boot/initrd.img-4.9.201-tegra
Warning: couldn't identify filesystem type for fsck hook, ignoring.
I: The initramfs will attempt to resume from /dev/zram3
I: (UUID=2804e605-4890-494d-8dbd-5f3af75759f3)
I: Set the RESUME variable to override this.
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_EGL.conf: No such file or directory
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_GL.conf: No such file or directory
Changing INITRD in /boot/extlinux/extlinux.conf
Removing the random seed file
Please restart your Jetson Nano now to boot into read-only mode
neil@jetson:~/root-ro$ 
  1. After reboot the Jetson is going into an endless reboot loop.

It is possible that this isn’t working with the latest version of JetPack… I was using JetPack 4.3 when I worked with this.

I could make an attempt with an earlier version. You also did use a pre-canned image? I would have a 4.4 image, could try that first

Good, tried it from a JP 4.4 installation. Same result. Going for 4.3 now

neil@jetson:~/root-ro$ sudo ./install-nano.sh

[sudo] password for neil:

dphys-swapfile is not installed, assuming we dont need to disable swap

Setting up maintenance scripts in /root...

Setting up initramfs-tools scripts...

Adding "overlay" to /etc/initramfs-tools/modules

Updating initramfs...

update-initramfs: Generating /boot/initrd.img-4.9.140-tegra

Warning: couldn't identify filesystem type for fsck hook, ignoring.

I: The initramfs will attempt to resume from /dev/zram3

I: (UUID=ab07f45d-daf6-4b8e-bebf-fe9b8e0e1800)

I: Set the RESUME variable to override this.

/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_EGL.conf: No such file or directory

/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_GL.conf: No such file or directory

Changing INITRD in /boot/extlinux/extlinux.conf

Removing the random seed file

Please restart your Jetson Nano now to boot into read-only mode

neil@jetson:~/root-ro$

Too bad. Jetson Download Center | NVIDIA Developer has only JP 4.4.1 as oldest :(

EDIT: Wait, found something https://developer.nvidia.com/jetpack-43-archive https://developer.nvidia.com/jetpack-43-archive

EDIT 2: OK, I can confirm it works with 4.3. At least it doesn’t endlessly reboot.

But correct me if I’m wrong: Shouldn’t I NOT be able to create a file in the root filesystem after rebooting RO?

I’m just asking because I’m able to create files after reboot. Wasn’t expecting this

@juerg Just to refresh my question: Shouldn’t a “RO” booted system refuse to create files? At least this is my understanding of read-only.

The OverlayFS approach creates a in-memory overlay over the read-only FS that holds all the modifications and acts as if the underlying FS is writable. Anything you write, modify or delete is persisted into memory only, and will be gone / restored after a reboot. This allows Linux to function normally without the need for any other modifications. But it could mean you’re running out of memory when the system is up for longer periods of time.

OK I was aware of this. But does that explain, that I found my newly created file again after a complete power cycle of the box? Does that make sense to you?

No that should not happen. Then the OverlayFS is not working as it should. Did you run the script to enable it?

sudo /root/reboot-ro

Yes, did that exactly after the initial setup (on a JP 4.3 image). Me was told, that the rootfs is already ro.

I will try again from scratch later the day.

Thanks

OK, checked several times now with the fresh JP 4.3 image: An attempt to set the system into permanent R/O ends up in “Is always in RO mode” message. Creating a file then. Shutting the system down. Remove power supply and all cables attached. Wait a minute.

After restart the file is there again :(

Are you sure your R/O mode wasn’t just an illusion?

Maybe it has an impact, this is what I’m seeing at reboot for a second:

Regards

i tried overlayroot on JetPack 4.5.1 by this step ( overlayfs for Jetson - #5 by juerg ) . but after reading extlinux.conf, startup fails and reboot.

the last line before rebooting(by serial console):

[    0.018254] bootconsole [uart8250] disabled

messages after reading extlinux.conf:

U-Boot 2020.04-g6b630d64fd (Feb 19 2021 - 08:37:46 -0800)

SoC: tegra210
Model: NVIDIA Jetson Nano Developer Kit
Board: NVIDIA P3450-0000
DRAM:  4 GiB
MMC:   sdhci@700b0000: 1, sdhci@700b0600: 0
Loading Environment from SPI Flash... SF: Detected mx25u3235f with page size 256 Bytes, erase size 4 KiB, total 4 MiB
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  2  1  0 
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
843 bytes read in 26 ms (31.3 KiB/s)
1:	primary kernel
Retrieving file: /boot/initrd.img
20042806 bytes read in 901 ms (21.2 MiB/s)
Retrieving file: /boot/Image
34338824 bytes read in 1505 ms (21.8 MiB/s)
append: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1  earlycon=uart8250,mmio32,0x70006000  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 root=/dev/mmcblk0p1 ro rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 
## Flattened Device Tree blob at 83100000
   Booting using the fdt blob at 0x83100000
ERROR: reserving fdt memory region failed (addr=0 size=0)
ERROR: reserving fdt memory region failed (addr=0 size=0)
   Using Device Tree in place at 0000000083100000, end 000000008317e68d
copying carveout for /host1x@50000000/dc@54200000...
copying carveout for /host1x@50000000/dc@54240000...

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.201-tegra (buildbrain@mobile-u64-5294-d8000) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #1 SMP PREEMPT Fri Feb 19 08:40:32 PST 2021
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 32,
[    0.000000] OF: fdt: - 80000000 ,  7ee00000
[    0.000000] OF: fdt: - 100000000 ,  7f200000
[    0.000000] Found tegra_fbmem: 00800000@92ca9000
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000070006000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node iram-carveout, compatible id nvidia,iram-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fac00000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 24 pages/cpu s57560 r8192 d32552 u98304
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1023544
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1  earlycon=uart8250,mmio32,0x70006000  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 root=/dev/mmcblk0p1 ro rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 98304 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 29256(89%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Memory: 3552524K/4159488K available (15294K kernel code, 2942K rwdata, 6664K rodata, 8576K init, 609K bss, 131828K reserved, 475136K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f70000   ( 15296 KB)
[    0.000000]     .rodata : 0xffffff8008f70000 - 0xffffff8009600000   (  6720 KB)
[    0.000000]       .init : 0xffffff8009600000 - 0xffffff8009e60000   (  8576 KB)
[    0.000000]       .data : 0xffffff8009e60000 - 0xffffff800a13f808   (  2943 KB)
[    0.000000]        .bss : 0xffffff800a13f808 - 0xffffff800a1d7cbc   (   610 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf03fc8000   (    63 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0ff200000   (  4082 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller
[    0.000000] t210 clock and reset probe
[    0.000000] tegra-pmc: get_secure_pmc_setting: done secure_pmc=1
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000006] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.010098] Console: colour dummy device 80x25
[    0.014746] console [tty0] enabled
[    0.018254] bootconsole [uart8250] disabled
[0000.159] [L4T TegraBoot] (version 00.00.2018.01-l4t-e82258de)
(...snip...)

Welcome to the club. This stuff is not working at all…

But you are right, attached a comparison of the boot of an unchanged JP4.5 and the patched version. The arrow shows the reboot. I have no idea why this doesn’t work, but it doesn’t work :((((

1 Like

Has anyone been able to make this work with the latest JP4.5.1? I’ve tried all the proposed solutions and they all end up in an infinite boot loop.

The same here, I tried literally all (I know about). Bu I have a very long thread with the very helpful @linuxdev. If he agrees, I will try again step by step. I hope this will finally lead to something. I invite you to participate.

Thread is here SD card damage protection - #40 by linuxdev