I would assume if the correct cboot args were set in the extlinux.conf file it would work. however I am having difficulty finding documentation on cboot. also would btrfs-progs need to be added to the initramfs? Thanks!
Also at some point I also want to use a luks encrypted root partition can that be done as well?
My board is an Xavier NX dev-kit with an nvme ssd
The short answer is yes to btrfs but unsure about luks.
cboot doesn’t apply to the root filesystem UNLESS the root filesystem is also the boot filesystem. cboot’s job is to locate the boot partition which must be ext-something and is usually named APP. On startup, cboot reads the [/boot]/extlinux/extlinux.conf file. From the conf file, it gets the file names of the selected kernel image, dtb, and initramfs which all must be in the same partition. It then loads the 3 files into memory and then starts the kernel. It’s the kernel/systemd’s job to locate and mount the root partition.
Now here’s the tricky part…
Depending on the version of Linux for Tegra you’re using, you could have several different partition layouts. Since you’re talking about btrfs and luks, I’ll assume you know what you’re doing. I’ll also assume you have a running NX that is booted from the sdcard. The question is… Is your current boot partition also your root partition? I.E. In your root filesystem, is /boot just a directory or is it actually a mount point for it’s own directory? Also, have you flashed the NX with the latest L4T that moves all the bootloader stuff from the ~18 partitions on the sdcard to the onboard QSPI-NOR flash? Depending on your answers, I can give you more detailed instructions.
Regardless, here are some things you’ll have know/do…
You’ll still need an sdcard in the devkit module. This is because cboot doesn’t yet support booting from NVMe.
btrfs isn’t a compiled-in module so you’ll need to both install btrfs-progs and rebuild the initramfs to include btrfs and nvme support. You should also add rootfstype=btrfs to your kernel command line in extlinux.conf. Of course, you should add an appropriate root entry as well. Something like root=/dev/nvme0n1p3 or whatever. BTW, you should always keep an extlinux.conf entry that boots solely to the sdcard so you have a backup.
It’s hard to give you further instructions without answers to the questions above but here’s the partition layout I use on my NXs with NVMe…
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 223.6G 0 disk
├─nvme0n1p1 259:1 0 2.8G 0 part /mnt/backup_boot
├─nvme0n1p2 259:2 0 3.7G 0 part [SWAP]
└─nvme0n1p3 259:3 0 215.2G 0 part /
mmcblk0 179:0 0 59.6G 0 disk
├─mmcblk0p1 179:1 0 3.7G 0 part /boot
├─mmcblk0p2 179:2 0 4G 0 part
└─mmcblk0p3 179:3 0 51.9G 0 part /mnt/sdcard_root
So the “live” (non-swap) partitions are nvme0n1p3 for “/” and and mmcblk0p1 for “/boot”. nvme0n1p3 is formatted as btrfs (or f2fs for some of my NXs) and mmcblk0p1 is formatted as ext4. cboot only cares about mmcblk0p1.
BTW, getting into the habit of keeping the boot and root filesystems separate (like desktops) is a good thing. There is UEFI firmware in the roadmap which will allow you to use a standard distribution filesystem image which uses EFI and Grub. In fact, there’s a very early UEFI firmware available for download in the Downloads area but it’s not yet ready for general use.
Oh yeah… luks. The reason I was unsure about it is that I’m not sure if the pre-built 4.9 kernel has all the parts built in. If it does, you’re good to go. If not, you’ll have to rebuild the kernel yourself.
1 Like
Thanks! currently I have the root and home mounts on separate partitions and just have the sd card extlinux.conf pointed to the partuuid of root. however I am slightly confused about the exact boot system. do I even need a boot partition on the nvme if its using the sd card?
Technically no. I always partition with boot, swap and root partitions since I’ve been doing it that way for a few decades. I use the nvme boot partition just to hold a backup of the sdcard boot partition because I have fat fingers and bad typing skills. :) It’s also a prep for the UEFI firmware
If you really wanted to, you could actually create a small EFI partition at the front of the nvme just to hold space for moving to UEFI later. Here’s my layout on the NX I’m testing the UEFI firmware on…
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 238.5G 0 disk
├─nvme0n1p1 259:1 0 600M 0 part /boot/efi
├─nvme0n1p2 259:2 0 1G 0 part /boot
├─nvme0n1p3 259:3 0 5.4G 0 part [SWAP]
└─nvme0n1p4 259:4 0 204.9G 0 part /
No sdcard at all.
Now that looks like a plan. It also looks a lot more familiar lol. Glad they’re making the move to UEFI. Really appreciate the help!
EXACTLY!!
Don’t hesitate to post if you run into issues.
You wouldn’t happen to have a copy of what your extlinux.conf and fstab look like would you? I’m still having difficulty transitioning to btrfs. Also I would like to have root and home on separate subvolumes, I think I found some info on how to configure that but I’m not sure.
Oops, I didn’t see this yesterday, sorry.
Here’s extlinux.conf
TIMEOUT 30
DEFAULT nvme
MENU TITLE L4T boot options
LABEL nvme
MENU LABEL nvme
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} text root=/dev/nvme0n1p4 rootfstype=btrfs rootwait rw
LABEL sdcard
MENU LABEL sdcard
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} text root=/dev/mmcblk0p2 rootfstype=ext4 rootwait rw
There’s nothing really special about it and you can replace root=<devname> with root=UUID=<uuid> or root=PARTUUID=<partuuid> if you like. You generally don’t need to specify the subvolume for the root filesystem because it’s usually the default subvolume. If you needed to specify a specific subvolume you could add rootflags=subvol=/path/to/subvolume or rootflags=subvolid=objectid to the kernel command line. I wouldn’t recommend that though.
One thing to note though… The paths for LINUX and INITRD are relative to the root of the first or APP partition on the SDcard. The default SDcard image has one APP partition that contains the entire root filesystem, including /boot. That’s why the LINUX and INITRD paths are prefixed with /boot. If your SDcard has separate partitions for the boot and root filesystems, you’ll need to remove the /boot prefixes. This only applies to the SDcard, not the NVMe.
Here’s fstab
# /etc/fstab: static file system information.
#
# These are the filesystems that are always mounted on boot, you can
# override any of these by copying the appropriate line from this file into
# /etc/fstab and tweaking it as you see fit. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/root / btrfs defaults,noatime 0 1
/dev/mmcblk0p1 /boot ext4 defaults,noatime 0 0
Notice that the root filesystem is specified as /dev/root. This is a virtual device that corresponds to the root filesystem mounted by systemd, which it got from the kernel command line. You could use the same device as yo used in the root= kernel parameter but just using /dev/root makes things easier since you don’t have to keep track of root filesystem devices in two places.
/dev/mmcblk0p1 is the boot partition on the SD card. It’s where cboot read extlinux.conf and the images from.
Making /home a btrfs subvolume is as easy as creating a new subvolume /home-temp, copying or moving the contents of /home to /home-temp, deleting /home, and renaming /home-temp to /home. You don’t need any fstab entry for that.
For some reason when ever I boot the board I keep getting the error “unknown file system type btrfs”. But I am able to mount it with the system booted from just the sd card and I made sure to install btrfs-progs. Do I need to rebuild the initramfs somehow?
:)
Annnnnd, could you possibly point me in the direction of how to go about doing that?
And sorry I must have missed that part while reading the previous reply
Heh, no worries. :)
I use dracut to manage initramfses for both Ubuntu and Fedora but it isn’t installed by default in these Ubuntu images. So I think for these images, look in /etc/initramfs-tools. You should be able to add modules to the “modules” file and run update-initramfs. Make a copy of the original one first though.
That did the trick! I just installed dracut to make it easier and we’re up and running. Thanks!!
Also this piece of info may have solved an infuriating problem I’ve been working on for a long time involving Linux on my MacBook Pro.
One more question, i’m having trouble finding consistent documentation on this. When I create btrfs subvolumes should I for example create @ @home @snapshots or @ @/home @/snapshots ?
Talking about btrfs subvolume/snapshot strategy is like talking about religion or politics. Whatever you say, you can’t win. For this reason, I tend to stick with whatever is recommended here… btrfs Wiki
Ah so that’s why all the places I looked on Google were radiating rage. I think I should go on one of those forms and make the suggestion that you should really keep the partition table for a drive on a separate floppy disk, and write it as one block of raw data.
Good luck with that :)