F2FS linux root

Hello, is it possible to use F2FS as the linux root filesystem? Browsing u-boot’s source code I can’t see support for F2FS, so an option would be replacing U-boot with some other bootloader that does support it. Another option would be to use APP as a /boot partition and have an extra F2FS partition mounted as / but I don’t know if this is supported by the Jetson boot process or how.

Any advice would be welcome!

no experience with this. Maybe other users can share their idea.

Ok, after not seeing my family and friends or doing anything else in my free time for 4 days in a row I think I’ve got it. Here are my notes in case it helps someone else. I’m specifically targeting Waveshare’s JetRacer AI PRO image but the idea should work the same for NVidia images:

We have two problems here: 1) U-boot doesn’t support F2FS, so the /boot directory where the kernel and extlinux config stuff lives must still be in an ext4 partition, but since it’s barely containing /boot it can be as short as 1GB and the rest of the contents of the old APP will go in a new partition (/dev/mmcblk0p15) named “ROOT” (for example). So we will destroy the APP partition and recreate it as two new partitions, a new 1GB ext4 APP partition with just /boot in and an F2FS partition with everything else. Problem number 2) is that the kernel itself doesn’t support F2FS either. For some reason NVidia thought that it would be funny to have Libreoffice but not F2FS support, so we need to recompile the kernel like we are in 1993. We are going to need Waveshare image which is based on Jetpack 4.5, L4T 32.5.0 and kernel 4.9.201 and I’m sticking to that exact version.

To compile the kernel you need ubuntu 18.04. Since I have 20.04, I used docker to pull a bare 18.04 image but you can skip this if you have 18.04 or use a 18.04 VM:

docker pull ubuntu:18.04
docker run --name jetracer4.9.201 -v /space:/space -it ubuntu:18.04

If you exit for some reason, you can come back in with:

docker start jetracer4.9.201
docker exec -it jetracer4.9.201 bash

Inside your 18.04 ubuntu, do:

useradd -G sudo -s /bin/bash -m javi
passwd javi # give it a password
apt update
apt install gnupg2 apt-utils xxd kmod sudo wget tzdata
apt install /space/JETRACER/sdkmanager_1.6.1-8175_amd64.deb
su - javi
sdkmanager --cli install  --logintype devzone --product Jetson --host --targetos Linux --version 4.6 --target JETSON_NANO_TARGETS --flash skip 

This will install sdkmanager, which I previously downloaded in /space/JETRACER and note that I shared my host machine’s /space dir as /space inside the docker container.

Authenticate, accept all licenses, install all host stuff, if there is something that requires interactive configuracion CTRL-C, install it with apt and relaunch sdkmanager since sdkmanager won’t let you interact with dpkg.

Finally, skip installing SDK components on your Jetson Nano.

After this, we’ll be mostly following directions from this great blog: Compiling Custom Kernel Modules on the Jetson Nano | Kevin's Blog

wget http://releases.linaro.org/components/toolchain/binaries/7.3-2018.05/aarch64-linux-gnu/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
tar xf gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
export CROSS_COMPILE=$(pwd)/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
export LOCALVERSION=-f2fs

wget https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/sources/T210/public_sources.tbz2
tar -xjf public_sources.tbz2
cd Linux_for_Tegra/source/public
tar -xjf kernel_src.tbz2
cd kernel/kernel-4.9

sudo apt install build-essential bc libncurses5 libncurses5-dev
TEGRA_KERNEL_OUT=$HOME/t4l-kernel
mkdir -p $TEGRA_KERNEL_OUT

make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig
make ARCH=arm64 O=$TEGRA_KERNEL_OUT menuconfig

Go to File SYstems → enable F2FS filesystem support. Make sure you make it <*>, not <M>, ie. baked in support and not module support since module support would also require to modify initrd and this is already tedious as it is.

To compile and pack the kernel + modules:

make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j32 # 32=num of threads

mkdir ~/to_tar
make ARCH=arm64 O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=~/to_tar/
mkdir -p ~/to_tar/boot
cp ~/t4l-kernel/arch/arm64/boot/Image ~/to_tar/boot/
cp ~/t4l-kernel/arch/arm64/boot/dts/* ~/to_tar/boot/
cd ~/to_tar
tar czf /space/JETRACER/kernel-4.9.201-f2fs.tgz .

This fixes problem NUMBER 2, ie. we now have a kernel that supports F2FS. Now to the partition reshuffle:

Download waveshare jetracer pro ai image from: JetRacer Pro AI Kit - Waveshare Wiki outside of the docker (I put it in a directory named /space/JETRACER):

unzip jetracer_pro_ws.zip
sudo losetup -Pf jetracer_pro_ws.img
mkdir APP ROOT OLDAPP
sudo mount /dev/loop0p1 APP
sudo mksquashfs APP jetracer_pro_ws.squashfs -comp lz4
sudo umount APP

sudo gdisk /dev/loop0

Follow this to destroy and recreate the partitions. I’m giving the root partition 30GB so that later on I don’t have to reflash the whole SD.

Command (? for help): d
Partition number (1-14): 1

Command (? for help): n
Partition number (1-128, default 1): 1
First sector (34-123596766, default = 28672) or {+-}size{KMGTP}:
Last sector (28672-123596766, default = 123596766) or {+-}size{KMGTP}: +1G
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Command (? for help): c
Partition number (1-14): 1
Enter name: APP

Command (? for help): n
Partition number (15-128, default 15):
First sector (34-123596766, default = 2125824) or {+-}size{KMGTP}:
Last sector (2125824-123596766, default = 123596766) or {+-}size{KMGTP}: +30G
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Command (? for help): c
Partition number (1-15): 15
Enter name: ROOT

Finally, exit with w and confirm.

I’m backing up the original contents of APP into an squashfs because it keeps all permissions, owners, hard links etc and will be using rsync later on to split into the /boot and / partitions.

sudo partprobe /dev/loop0
sudo mkfs.ext4 /dev/loop0p1
sudo mkfs.f2fs /dev/loop0p15
sudo mount -o loop jetracer_pro_ws.squashfs OLDAPP
sudo mount /dev/loop0p1 APP
sudo mount /dev/loop0p15 ROOT
sudo rsync -aAHx --info=progress2 --delete OLDAPP/boot APP
sudo rsync -aAHx --info=progress2 --delete --exclude boot OLDAPP/ ROOT/

Replace kernel (save previous kernel and dtbs). In the end we’ll have /boot/4.9.201-tegra with the original kernel + dtbs and /boot/4.9.201-f2fs with our F2FS kernel, and by symlinking one or the other directory’s contents we can switch kernels.

cd APP/boot
sudo mkdir 4.9.201-tegra 4.9.201-f2fs
sudo mv Image tegra* 4.9.201-tegra
cd ..
sudo tar xf /space/JETRACER/kernel-4.9.201-f2fs.tgz ./boot
cd boot
sudo mv Image tegra* 4.9.201-f2fs
sudo ln -s 4.9.201-f2fs/* .

Now:

sudo vi extlinux/extlinux.conf

And replace:

APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

with:

APPEND ${cbootargs} quiet root=/dev/mmcblk0p15 rw rootwait rootfstype=f2fs console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0

Unpack the modules in the new root fs:

cd ../../ROOT
sudo tar xvf /space/JETRACER/kernel-4.9.201-f2fs.tgz ./lib/modules
cd etc
sudo vi fstab

Replace:

/dev/root            /                     ext4           defaults                                     0 1

with:

/dev/root            /                     f2fs           defaults,noatime                                     0 1

Then clean up:

cd ../../
sudo umount ROOT
sudo umount APP
sudo gdisk /dev/loop0 # p to print the partition table, it will be useful soon
sudo losetup -D

And finally flash the SD card (replace /dev/sdc with your SD)

sudo dd if=jetracer_pro_ws.img of=/dev/sdc count=31758 bs=1M status=progress

I limited to 31758 GB because that’s where the the new ROOT partition ends. To find the number I used gdisk /dev/loop0 earlier, printed the partition table with p and found the last used sector of the ROOT (15th) partition plus 1, times 512 (sector size) and divided by (1024 * 1024) to find that 31758 number which is the used portion of the SD in GB and it doesn’t make sense to flash beyond it.

Put the card in the jetson, boot, rejoice:

jetson@jetson-desktop:~$ mount |grep f2fs
/dev/mmcblk0p15 on / type f2fs (rw,noatime,background_gc=on,no_heap,user_xattr,inline_xattr,acl,inline_data,inline_dentry,flush_merge,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,fsync_mode=posix)

TODO: script to auto-expand the F2FS on boot.