Attempts to set up EcryptFS and FSCrypt failed. And failed and failed

Summary

I’ve just spent the past few days trying to get a file directory encryption scheme working on the Jetson Nano. While this is trivial on desktop Ubuntu 18.04 (both EcryptFS and Fscrypt set up and work perfectly), attempting this on the Jetson required kernel rebuilding and all kinds of challenging-to-the-new-comer operations! Here are the relevant areas required:

  • Jetson HW
  • Jetson Setup & Installation
  • Linux Kernel
  • Linux Kernel Recompilation
  • Linux Kernel Security
  • EXT4 Filesystem
  • PAM
  • ecryptfs
  • fscrypt

Sadly, none of these are areas of great familiarity for me, let alone expertise, so this was hard work. First up was ecryptfs. The kernel needed changing for this, but ultimately there were kernel errors no matter what I did. Then reading that ecryptfs was no longer maintained, I turned my attention to fscrypt. Same long procedure and same sad outcome.

Below are the steps to install fscrypt by the book, and the results taken along the way, including, the ultimate kernel errors on the Jetson Nano.

The Kernel Error is [ 547.987897] tegra-se 70012000.se: invalid key size

If someone who has some more expertise could say what’s going wrong and ideally help get this working that would be amazing. Failing that some definitive statements about what does and doesn’t work would save the next person some time and let me grieve and move on.

Resources

Following: https://tlbdk.github.io/ubuntu/2018/10/22/fscrypt.html

Fscrypt GitHub

Kernel Config

Kernel flags are described in the GitHub project

  • For ext4, the kernel must be v4.1 or later, and the kernel configuration must have either CONFIG_FS_ENCRYPTION=y (for kernels v5.1+) or CONFIG_EXT4_ENCRYPTION=y or =m (for older kernels). Also, the filesystem must have the encrypt feature flag enabled; see here for how to enable it.

CONFIG_EXT4_ENCRYPTION=Y

  • selects FS_ENCRYPTION

CONFIG_FS_ENCRYPTION=Y

  • selects CRYPTO, CRYPTO_AES, CRYPTO_CBC, CRYPTO_ECB, CRYPTO_XTS, CRYPTO_CTS, CRYPTO_CTR, KEYS

Examining the source, (keyinfo.c) there’s a reference to a new crypto system option - SPECK

Weirdly, SPECK is not an option in later kernels (4.15)

CONFIG_CRYPTO_SPECK=Y

  • selects CRYPT_ALGAPI

Interesting configs which may be related

When the above failed (eg. [ 547.987897] tegra-se 70012000.se: invalid key size), additionally adding

CONFIG_ENCRYPTED_KEYS=Y

CONFIG_BIG_KEYS=Y

Kconfig Record in the fs/crypt directory

config FS_ENCRYPTION
	tristate "FS Encryption (Per-file encryption)"
	select CRYPTO
	select CRYPTO_AES
	select CRYPTO_CBC
	select CRYPTO_ECB
	select CRYPTO_XTS
	select CRYPTO_CTS
	select CRYPTO_CTR
	select KEYS
	help
	  Enable encryption of files and directories.  This
	  feature is similar to ecryptfs, but it is more memory
	  efficient since it avoids caching the encrypted and
	  decrypted pages in the page cache.

The source reveals no other undeclared dependencies to the causal observer.

Kernel is built, Jetson flashed.

Post Install Kernel Check

This is on the newly flashed Nano

$ zgrep -h ENCRYPTION /proc/config.gz
CONFIG_EXT4_ENCRYPTION=y
CONFIG_EXT4_FS_ENCRYPTION=y
CONFIG_FS_ENCRYPTION=y
$ zgrep -h KEYS /proc/config.gz
CONFIG_BIG_KEYS=y
CONFIG_ENCRYPTED_KEYS=y
$ zgrep -h SPECK /proc/config.gz
CONFIG_CRYPTO_SPECK=y
# CONFIG_CRYPTO_SPECK_NEON is not set

Fscrypt Setup

First - Pam Setup

Create

/usr/share/pam-configs/keyinit-fix

And insert

Name: keyinit fix
Default: yes
Priority: 0
Session-Type: Additional
Session:
	optional	pam_keyinit.so force revoke
sudo pam-auth-update

Reboot

Then, enable the file system for encryption

sudo tune2fs -O encrypt /dev/device

(No errors, No kernel errors)

Grab User Code

$ sudo apt-get install fscrypt libpam-fscrypt

(No errors, no kernel errors)

Set fscrypt up on the partition

$ sudo fscrypt setup

[No Errors]

$ fscrypt status
filesystems supporting encryption: 1
filesystems with fscrypt metadata: 0

MOUNTPOINT  DEVICE          FILESYSTEM  ENCRYPTION  FSCRYPT
/           /dev/mmcblk0p1  ext4        supported   No
$ sudo fscrypt setup /
Metadata directories created at "/.fscrypt".
Filesystem "/" (/dev/mmcblk0p1) ready for use with ext4 encryption.

(No Kernel errors)

fscrypt is now enabled. Status request now reports fscrypt is now fully active.

$ fscrypt status
filesystems supporting encryption: 1
filesystems with fscrypt metadata: 1

MOUNTPOINT  DEVICE          FILESYSTEM  ENCRYPTION  FSCRYPT
/           /dev/mmcblk0p1  ext4        supported   Yes

(Status matches my desktop at the same point in the process)

$ fscrypt status
filesystems supporting encryption: 1
filesystems with fscrypt metadata: 1

MOUNTPOINT                      DEVICE       FILESYSTEM  ENCRYPTION     FSCRYPT
/                               /dev/sda2    ext4        supported      Yes

Use

Finally, we attempt to create a folder

$ cd Documents
$ mkdir secret
$ fscrypt encrypt secret
Should we create a new protector? [y/N] y
Your data can be protected with one of the following sources:
1 - Your login passphrase (pam_passphrase)
2 - A custom passphrase (custom_passphrase)
3 - A raw 256-bit key (raw_key)
Enter the source number for the new protector [2 - custom_passphrase]: secret
Enter the source number for the new protector [2 - custom_passphrase]: 2
Enter a name for the new protector: secret
Enter custom passphrase for protector "secret":
Confirm passphrase:
"secret" is now encrypted, unlocked, and ready for use.

(no dmesg errors or warnings)

All is well. It has all gone so smoothly.

$ cat > things
secret things
$ cp things secret/

OH NO!

cp: cannot create regular file 'secret/things': Required key not available
$ dmesg
...
[  547.982991] tegra-se 70012000.se: invalid key size
[  547.987897] tegra-se 70012000.se: invalid key size
[  547.992849] tegra-se 70012000.se: invalid key size
[  547.997757] tegra-se 70012000.se: invalid key size
[  548.002631] tegra-se 70012000.se: invalid key size

HELP!

[Edit - for clarity to emphasize that fscrypt is failing in the kernel]
[Edit - to remove some distracting output]

This is just spot checking and some things to beware of. No particular order.

On the Jetson, when it comes to knowing about kernel features, you can get a list of the running kernel’s config as it was compiled:
zcat /proc/config.gz

You can search for a particular item in the config via this example for ‘CRYPTO’:
zcat /proc/config.gz | grep CRYPTO

You can “page” this to make it easier to scroll around:

# All:
zcat /proc/config.gz | less
# Just that tag:
zcat /proc/config.gz | grep CRYPTO | less

Within “less” you can search with a “/” and the tag, e.g., “/AES” for “AES”. To search again for that tag just tap the “n” key, or for previous match, shift-n.

When you see that kernel config tag is “=y”, then the feature is always present, and is integrated directly into the kernel Image. If you see “=m”, then the feature is present as a module, and should automatically load upon demand. At times you can manually load a module if setup is not complete.

The result of the command “uname -r” is a combination of the base version of the kernel, plus the “CONFIG_LOCALVERSION” during the compile. Within the “/proc/config.gz” the “CONFIG_LOCALVERSION” is left blank and is the only feature which keeps config.gz from being a 100% exact match to what that kernel was compiled with. If your CONFIG_LOCALVERSION="-tegra" during the build, then the command for a “4.9.140” kernel would be “4.9.140-tegra”.

Modules are searched for at:
/lib/modules/$(uname -r)/kernel/

If you “cd /lib/modules/(uname -r)/kernel/", then you will change to that directory because the "(uname -r)” is a macro expansion of the actual command. Modules not in some subdirectory of this will not be found.

You need user space software to perform some setup of filesystems. I have no experience with the encrypted system you are working on, but most likely much of this would allow setup and encryption via those tools even if the kernel features are not enabled (not always, but much of the time).

To actually use one of those features it is very likely that you need that kernel feature. I do not know for sure which CONFIG items you need, but if you compare your “/proc/config.gz” to what documents say, then you can report here which symbols are missing and we can go from there to add those symbols in (FYI, “symbol” and “feature” and “config” are pretty much the same thing in this context).

Some details will change depending on which L4T release is involved. To find this use “head -n 1 /etc/nv_tegra_release”.

You should always copy your original “/proc/config.gz” somewhere as a reference before you work on a kernel. This is the key to starting with what you already have, or reproducing exactly what you had if something goes wrong. You may see references to “make tegra_defconfig”, and for the most part these will match on a fresh system…but this is not guaranteed, and if you really want an exact match (and you do), then start with “config.gz”.

A “config.gz” file can be copied somewhere (the original is not a real file…it is a read-only simulated file produced by the kernel), and decompressed with “gunzip config.gz”. If you were to then edit this file and make CONFIG_LOCALVERSION like this, then you have a 100% match (assuming “uname -r” has suffix “-tegra”):

CONFIG_LOCALVERSION="-tegra"

Placing a copy of this as name “.config” in the correct part of kernel source code prior to a compile or menu config editor puts you way ahead of someone trying to configure from scratch.

Note that if you compile a feature as a module, then you only need a copy of the file to the right place under “/lib/modules/$(uname -r)/kernel/” to complete install. If you compile as an integrated feature, then things require a different file copy. Should module install go wrong, it isn’t a big deal. Should a kernel integrated Image file go wrong, then the system will be unbootable at times, and there are precautions to take. Not everything can be built as a module, and not everything can be built integrated, but if you can build as a module, then life is simplified for you.

Even if you will not install a new kernel Image I advise compiling this at least once to look for errors just as a sanity check. Then compile the modules. For modules which are already installed and for cases where you only add configuration to a new Image which ends up with the same “uname -r”, then you are probably ok to only add new modules and reuse old modules.

Note that this is not something you edit directly:

Even if you are an expert I would normally advise using a menu editor application which is what reads a Kconfig. Most of these require package “libncurses5-dev”, which you get from “sudo apt-get install libncurses5-dev”. Then, at the top level of the kernel source, you can use any of the many possible editors, and this will read the existing “.config” if there is one. Examples, and my favorite is “nconfig” due to symbol search:

make nconfig
make menuconfig
make xconfig

Note that it is a bad idea to compile directly within your kernel source. You can clean the source first with “sudo make mrproper”. Then create some temporary area with no content, e.g., “mkdir ~/kernel_stuff”, copy the “.config” to “~/kernel_stuff/”, and from the source of the kernel refer to the other location for all output:

cd /where/ever/it/is/kernel
make O=~/kernel_stuff nconfig

If you start by finding out which kernel features (config items) are needed, and compare it to the running system’s “/proc/config.gz”, the we can give more specific information.

Thanks for the detailed reply, LinuxDev. I pretty much got through all that, set flags, compiled kernels, downloaded software, run it. I wrote about it in detail! I think I have all of that down. Specifically:

  • zcat / zgrep for examining kernel - check
  • less and / - check
  • uname-r - yup
  • CONFIG_LOCALVERSION - I did not bother with this. I should have.
  • module locations - check (no new modules - all compiled in)
  • user space modules - check (fscrypt)
  • suggestion about building modules - check - although I wanted everything in the kernel (and I am 10 or so successful kernel rebuilds and installs on the Nano now)
  • compile kernel and modules - check
  • .config not edited directly - check - using make menuconfig
  • not build in the source tree - yup - using the file layout of the sdkmanager. Building separately.

OK. So all that done as documented. The kernel errors I noted are the result of that (previous two days work!)

The original post has been slightly edited to make it more clear that the process is ready for some expert eyes.

Hi,
We support secureboot on Jetson platforms. Please check the docuement

Unfortunately, we don’t have implementation/reference of encrypting rootfs. May need other users to share experience.