overlayfs for Jetson

i tried overlayroot on JetPack 4.5.1 by this step ( overlayfs for Jetson - #5 by juerg ) . but after reading extlinux.conf, startup fails and reboot.

the last line before rebooting(by serial console):

[    0.018254] bootconsole [uart8250] disabled

messages after reading extlinux.conf:

U-Boot 2020.04-g6b630d64fd (Feb 19 2021 - 08:37:46 -0800)

SoC: tegra210
Model: NVIDIA Jetson Nano Developer Kit
Board: NVIDIA P3450-0000
DRAM:  4 GiB
MMC:   sdhci@700b0000: 1, sdhci@700b0600: 0
Loading Environment from SPI Flash... SF: Detected mx25u3235f with page size 256 Bytes, erase size 4 KiB, total 4 MiB
*** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  2  1  0 
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
843 bytes read in 26 ms (31.3 KiB/s)
1:	primary kernel
Retrieving file: /boot/initrd.img
20042806 bytes read in 901 ms (21.2 MiB/s)
Retrieving file: /boot/Image
34338824 bytes read in 1505 ms (21.8 MiB/s)
append: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1  earlycon=uart8250,mmio32,0x70006000  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 root=/dev/mmcblk0p1 ro rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 
## Flattened Device Tree blob at 83100000
   Booting using the fdt blob at 0x83100000
ERROR: reserving fdt memory region failed (addr=0 size=0)
ERROR: reserving fdt memory region failed (addr=0 size=0)
   Using Device Tree in place at 0000000083100000, end 000000008317e68d
copying carveout for /host1x@50000000/dc@54200000...
copying carveout for /host1x@50000000/dc@54240000...

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.201-tegra (buildbrain@mobile-u64-5294-d8000) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #1 SMP PREEMPT Fri Feb 19 08:40:32 PST 2021
[    0.000000] Boot CPU: AArch64 Processor [411fd071]
[    0.000000] OF: fdt:memory scan node memory@80000000, reg size 32,
[    0.000000] OF: fdt: - 80000000 ,  7ee00000
[    0.000000] OF: fdt: - 100000000 ,  7f200000
[    0.000000] Found tegra_fbmem: 00800000@92ca9000
[    0.000000] earlycon: uart8250 at MMIO32 0x0000000070006000 (options '')
[    0.000000] bootconsole [uart8250] enabled
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb0_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: fdt:Reserved memory: failed to reserve memory for node 'fb1_carveout': base 0x0000000000000000, size 0 MiB
[    0.000000] OF: reserved mem: initialized node vpr-carveout, compatible id nvidia,vpr-carveout
[    0.000000] OF: reserved mem: initialized node iram-carveout, compatible id nvidia,iram-carveout
[    0.000000] OF: reserved mem: initialized node ramoops_carveout, compatible id nvidia,ramoops
[    0.000000] cma: Reserved 64 MiB at 0x00000000fac00000
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.0 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.1
[    0.000000] percpu: Embedded 24 pages/cpu s57560 r8192 d32552 u98304
[    0.000000] CPU features: enabling workaround for ARM erratum 832075
[    0.000000] Speculative Store Bypass Disable mitigation not required
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1023544
[    0.000000] Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1  earlycon=uart8250,mmio32,0x70006000  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 root=/dev/mmcblk0p1 ro rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 
[    0.000000] log_buf_len individual max cpu contribution: 32768 bytes
[    0.000000] log_buf_len total cpu_extra contributions: 98304 bytes
[    0.000000] log_buf_len min size: 32768 bytes
[    0.000000] log_buf_len: 131072 bytes
[    0.000000] early log buf free: 29256(89%)
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.000000] Memory: 3552524K/4159488K available (15294K kernel code, 2942K rwdata, 6664K rodata, 8576K init, 609K bss, 131828K reserved, 475136K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000]       .text : 0xffffff8008080000 - 0xffffff8008f70000   ( 15296 KB)
[    0.000000]     .rodata : 0xffffff8008f70000 - 0xffffff8009600000   (  6720 KB)
[    0.000000]       .init : 0xffffff8009600000 - 0xffffff8009e60000   (  8576 KB)
[    0.000000]       .data : 0xffffff8009e60000 - 0xffffff800a13f808   (  2943 KB)
[    0.000000]        .bss : 0xffffff800a13f808 - 0xffffff800a1d7cbc   (   610 KB)
[    0.000000]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000]               0xffffffbf00000000 - 0xffffffbf03fc8000   (    63 MB actual)
[    0.000000]     memory  : 0xffffffc000000000 - 0xffffffc0ff200000   (  4082 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
[    0.000000] NR_IRQS:64 nr_irqs:64 0
[    0.000000] /interrupt-controller@60004000: 192 interrupts forwarded to /interrupt-controller
[    0.000000] t210 clock and reset probe
[    0.000000] tegra-pmc: get_secure_pmc_setting: done secure_pmc=1
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] clk_cbus_recalc_rate: no gbus parent
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000006] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.010098] Console: colour dummy device 80x25
[    0.014746] console [tty0] enabled
[    0.018254] bootconsole [uart8250] disabled
[0000.159] [L4T TegraBoot] (version 00.00.2018.01-l4t-e82258de)
(...snip...)

Welcome to the club. This stuff is not working at all…

But you are right, attached a comparison of the boot of an unchanged JP4.5 and the patched version. The arrow shows the reboot. I have no idea why this doesn’t work, but it doesn’t work :((((

1 Like

Has anyone been able to make this work with the latest JP4.5.1? I’ve tried all the proposed solutions and they all end up in an infinite boot loop.

The same here, I tried literally all (I know about). Bu I have a very long thread with the very helpful @linuxdev. If he agrees, I will try again step by step. I hope this will finally lead to something. I invite you to participate.

Thread is here SD card damage protection - #40 by linuxdev

I just want to point out a possible issue by explaining something about the boot process. This is not an answer, but it is useful to know this when working with any new filesystem which has boot issues (and OverlayFS is a new filesystem, and replaces or adds to ext4).

During a normal boot the bootloader will basically use the Linux kernel to overwrite itself. When this happens the Linux kernel must be able to read whatever filesystem it is sitting on, and the default is ext4. The bootloader software itself can also read ext4. Once the kernel loads the kernel modules can load…and these are normally read from ext4.

If you use a filesystem type the kernel does not understand, then the kernel will panic and fail. If you go to append some other filesystem layer on top of the ext4 (such as OverlayFS), then the kernel will use this if and only if it understands that addition.

If for some reason your kernel has support for that additional filesystem layer, but the driver for this is from a loadable module where the module is not yet loaded, then that module must be on some filesystem type which the kernel understands without loading any modules. Loading a module to understand how to load the module does not work.

The “adapter” to these situations where the kernel must load a module in order to load a previously unknown addition to the filesystem is the initial ramdisk (the “initrd”). An initrd is a very minimal entire operating system environment with a Linux kernel and other init files and kernel modules. This initrd runs entirely in RAM and the kernel will always understand how to read the filesystem when purely in RAM. If that kernel loads the additional filesystem support module from RAM, then pivots the filesystem from the RAM initrd to the actual hard disk, the problem is solved. The kernel ends up with the ability to load new modules because it already had that module loaded from the initrd prior to the pivot.

I don’t know for sure, but I suspect that OverlayFS support needs additions to the initrd. Notice in “/boot/extlinux/extlinux.conf” the line which has “INITRD /boot/initrd”. That file, “/boot/initrd”, is a self-contained miniature operating system which has no purpose other then to run the kernel with a few needed loadable kernel modules, and then pivot to the hard drive. If for some reason the kernel is starting on ext4, and then trying to add OverlayFS on top of this, then success can only occur if the OverlayFS is understood. Failure implies remaining on ext4. Should the attempt to load the OverlayFS kernel module content succeed in the initrd prior to pivoting over to the hard drive, then instructions for transitioning to OverlayFS would work. At least that is my theory.

There are ways to examine the existing initrd, make modifications, so on, but the methods for customizing it are not automatic. Some learning curve is involved. FYI, if you use serial console, then extlinux.conf can leave the original boot content unmodified, plus offer a chance to boot to an alternate configuration. Presumably the unmodified version is for safety in case experimenting goes wrong, while the second boot entry would name an alternate INITRD, e.g., something like:
INITRD /boot/initrd-overlayfs

I am sorry I cannot simply offer changes to “just make it work”. Experimentation and feedback is needed.

1 Like

So after reading all of the threads related to overlayfs on here I still haven’t been able to make this thing work. The whole boot process seems like such a mess on jetson nano.
I have a question though. After changing " /boot/extlinux/extlinux.conf" I noticed the following line inside the boot logs:

Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,4 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 gpt tegra_fbmem=0x800000@0x92ca9000 is_hdmi_initialised=1 earlycon=uart8250,mmio32,0x70006000 root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 root=/dev/mmcblk0p1 ro rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

Some arguments are repeated multiple times (e.g. root). My question is that will the values that I defined inside “extlinux.conf” override the cboot arguments (I assume that’s where they are coming from)? or do they conflict with each other? Should I change the cboot arguments too?

“Typically”, if an argument to the kernel is repeated, then only the last occurrence is counted. What goes on prior to the Linux kernel running can be a different story.

The boot process has changed more than anything else as L4T releases come out. Different releases may work a bit differently, but device tree is very likely to take part in earlier boot stages before Linux loads. The device tree itself may be installed in multiple ways. Device tree content might be edited by those earlier stages, and then as the actual Linux kernel boots, that edited content is what the extlinux.conf line content “APPEND ${cbootargs}” is. “${cbootargs}” is device tree content which expands to become a command line.

  • What do you see from:
    cat /proc/device-tree/chosen/bootargs
  • What is your “APPEND” line for the booted kernel in “/boot/extlinux/extlinux.conf”?
  • Does your “/boot/extlinux.conf” have an “FDT” line? If so, what is that line? (if not, then your tree is in a partition instead of being in a file)
1 Like

I’ve adjusted Jetson Nano & JetPack 4.5.1 to enable overlayroot.
There may still be some problems, but give it a try.

$ mount
/dev/mmcblk0p1 on /media/root-ro type ext4 (ro,relatime,data=ordered)
proc on /media/root-ro/proc type proc (rw,relatime)
sysfs on /media/root-ro/sys type sysfs (rw,relatime)
none on /media/root-ro/dev type devtmpfs (rw,relatime,size=1780108k,nr_inodes=445027,mode=755)
tmpfs-root on /media/root-rw type tmpfs (rw,relatime)
overlayroot on / type overlay (rw,relatime,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,size=1780108k,nr_inodes=445027,mode=755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/debug type cgroup (rw,nosuid,nodev,noexec,relatime,debug)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=26,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime)
sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
tmpfs on /run/user/120 type tmpfs (rw,nosuid,nodev,relatime,size=405108k,mode=700,uid=120,gid=124)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=405108k,mode=700,uid=1000,gid=1000)
3 Likes

I tried to achieve the same behavior as overlayfs on my Jetson Nano (dev kit) by making the uSD card read-only (Using tools such as arm-sdtool). It looks like it works. However after clearing the ReadOnly bit (Of the uSD card) to make a few changes, i get a few boot errors on next boot. (Ext4… bla bla bla). Finally the Nano comes up with the ‘Ubuntu’ background instead of the Nvidia background and the ‘toolbar’ on the left is missing. The shortcuts on the dashboard work. Shutting it down is impossible (Missing power buttons and the ‘shutdown’ command reports shutdown is scheduled, but it never shuts down).

Shutting the power off, transferring the image to the uSD card again and the system is up and running again.

Any suggestions how to avoid/solve such problems.

I tried the same with the latest available image (jetson-nano-jp451-sd-card-image.zip, the above used version was: 441). It is getting better now. i.e. It looks like the system starts normally. The background is normal and all buttons/bars are available again. The ReadOnly feature on the uSD card works as expected.
However, we still see some warnings/errors in the bootlog:
[ 2.630580] systemd-journald[2007]: Received request to flush runtime journal from PID 1
[ 2.742359] EXT4-fs error (device mmcblk0p1): htree_dirblock_to_tree:1004: inode #914219: block 3679131: comm systemd-tmpfile: bad entry in directory: rec_len is smaller than minimal - offset=0, inode=0, rec_len=0, name_len=0, size=4096
[ 2.768534] EXT4-fs warning (device mmcblk0p1): ext4_empty_dir:2698: inode #914219: comm systemd-tmpfile: directory missing ‘.’ and/or ‘…’
[ 2.781274] EXT4-fs warning (device mmcblk0p1): ext4_rmdir:2963: inode #914219: comm systemd-tmpfile: empty directory ‘systemd-private-45e6473d88134b75a18bcb23ff19a8b1-haveged.service-91zJ3k’ has too many links (3)
[ 2.804209] EXT4-fs error (device mmcblk0p1): htree_dirblock_to_tree:1004: inode #914198: block 3678577: comm systemd-tmpfile: bad entry in directory: rec_len is smaller than minimal - offset=0, inode=0, rec_len=0, name_len=0, size=4096
[ 2.827869] EXT4-fs warning (device mmcblk0p1): ext4_empty_dir:2698: inode #914198: comm systemd-tmpfile: directory missing ‘.’ and/or ‘…’
[ 2.842880] EXT4-fs error (device mmcblk0p1): htree_dirblock_to_tree:1004: inode #914202: block 3678578: comm systemd-tmpfile: bad entry in directory: rec_len is smaller than minimal - offset=0, inode=0, rec_len=0, name_len=0, size=4096
[ 2.866773] EXT4-fs warning (device mmcblk0p1): ext4_empty_dir:2698: inode #914202: comm systemd-tmpfile: directory missing ‘.’ and/or ‘…’
[ 2.888502] EXT4-fs error (device mmcblk0p1): htree_dirblock_to_tree:1004: inode #914197: block 3678576: comm systemd-tmpfile: bad entry in directory: rec_len is smaller than minimal - offset=0, inode=0, rec_len=0, name_len=0, size=4096
[ 2.913113] EXT4-fs warning (device mmcblk0p1): ext4_empty_dir:2698: inode #914197: comm systemd-tmpfile: directory missing ‘.’ and/or ‘…’
[ 2.927471] EXT4-fs warning (device mmcblk0p1): ext4_empty_dir:2698: inode #914196: comm systemd-tmpfile: directory missing ‘.’ and/or ‘…’
[ 2.948778] EXT4-fs error (device mmcblk0p1): htree_dirblock_to_tree:1004: inode #914221: block 3679133: comm systemd-tmpfile: bad entry in directory: rec_len is smaller than minimal - offset=0, inode=0, rec_len=0, name_len=0, size=4096
[ 2.973294] EXT4-fs warning (device mmcblk0p1): ext4_empty_dir:2698: inode #914221: comm systemd-tmpfile: directory missing ‘.’ and/or ‘…’
[ 2.986228] EXT4-fs warning (device mmcblk0p1): ext4_rmdir:2963: inode #914221: comm systemd-tmpfile: empty directory ‘systemd-private-45e6473d88134b75a18bcb23ff19a8b1-haveged.service-gsmCom’ has too many links (3)
[ 3.448690] using random self ethernet address
[ 3.453229] using random host ethernet address
[ 3.833362] random: crng init done
[ 3.836793] random: 7 urandom warning(s) missed due to ratelimiting
[ 4.277877] Mass Storage Function, version: 2009/09/11
[ 4.277883] LUN: removable file: (no medium)
[ 4.303052] using random self ethernet address
[ 4.313236] using random host ethernet address
[ 4.369098] EXT4-fs error (device mmcblk0p1): mb_free_blocks:1471: group 112, block 3678460:freeing already freed block (bit 8444); block bitmap corrupt.
[ 4.369114] EXT4-fs error (device mmcblk0p1): ext4_mb_generate_buddy:759: group 112, block bitmap and bg descriptor inconsistent: 23712 vs 23713 free clusters
[ 4.369123] EXT4-fs error (device mmcblk0p1): mb_free_blocks:1471: group 114, block 3736064:freeing already freed block (bit 512); block bitmap corrupt.
[ 4.369137] EXT4-fs error (device mmcblk0p1): ext4_mb_generate_buddy:759: group 114, block bitmap and bg descriptor inconsistent: 31887 vs 31888 free clusters
[ 4.379475] rndis0: HOST MAC 3a:5d:b9:c4:8a:fc

I don’t know if these warnings/error cause ‘long term’ problems.
Still hoping someone may give me a hint how to avoid/solve these warnings/errors.

Thank you negachov

This worked perfectly with JetPack 4.5

1 Like

I have started looking into why my approach doesn’t work anymore with JP 4.5 and later (JP 4.6 is also broken). It turns out that even without my changes applied, using the default /boot/initrd.img image that is produced by update-initramfs doesn’t work, the boot-time crash happens regardless of wether the overlayfs module is loaded or not.

This looks like a flaw in the more recent JP distributions, it means that none of the standardized ways to modify the initrd will work, and other projects that rely on this will fail too.

@jerrychang, @kayccc is there a chance that somebody at NVidia could look into this?

It should be very easy to reproduce: Create a fresh SD-card from the default image, then switch from boot/initrd to /boot/initrd.img in boot/extlinux/extlinux.conf and restart. You’ll be caught in an endless boot loop, and the crash happens exactly where @foreverneilyoung has observed it and posted a screenshot of.

@negachov’s approach works because it manually patches the working boot/initrd file to included the overlayfs additions and load them (thanks for the work on that!), but wouldn’t it be better to actually get the included initrd.img file working?

Full ack. Doesn’t work. Should work :)

1 Like

hello juerg,

please initial another new discussion thread to track overlayfs support with the latest JetPack release.
thanks

Hello,

I’m new in this forum. I’m using the Jetson Nano with JetPack 4.6 (L4T 32.6.1 version) and I tried to follow the same procedure of @bdaddr to implement the read-only overlayfs.

Additionally, I read the Topic 1032951 and the guide Using the initial RAM disk [initrd] suggested by @JerryChang but I was not able to find a working solution.

Has anyone managed to find a solution with this JetPack version?

Thanks

@JerryChang I’ve reported the issue in a separate thread. Please note that this isn’t related to OverlayFS and the problem is much larger, /boot/initrd.img appears to be broken on JP 4.5 and newer:

Actually, I think it’s already JP 4.4 that broke support for /boot/initrd.img & update-initramfs

I’ve managed to find the reason why the /boot/initrd.img files didn’t work anymore on JP 4.4. and newer, along with a simple fix for it, see here: /boot/initrd.img as generated by update-initramfs broken on JetPack 4.6 - #20 by juerg

I’ve already updated my root-ro repo accordingly, if you run install-nano.sh as root on Jetson Nano, it should work again!

1 Like