Enabling disk encryption doesn't boot the device // Jetson Xavier NX // P3668-0001

Hello there,

Last few months we are trying to enable disk encryption with no luck. We really need your help to figure out the issues. We have the worst experience ever with your documentation.

We are using Jetson Xavier NX production board (P3668-0001). Last week we successfully enabled the secure boot in our device. We followed following steps to achieve it.

  1. Install dependencies
sudo apt-get update 
sudo apt-get install qemu-user-static libxml2-utils cryptsetup libftdi-dev openssh-server
  1. Setup L4T
mkdir ~/nvidia
cd nvidia
#download l4t
wget https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/t186/tegra186_linux_r32.5.1_aarch64.tbz2
# download rootfs 
wget https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/t186/tegra_linux_sample-root-filesystem_r32.5.1_aarch64.tbz2
# download secureboot packages 
wget https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/T186/secureboot_R32.5.0_aarch64.tbz2
tar xf tegra186_linux_r32.5.1_aarch64.tbz2
tar xvjf secureboot_R32.5.0_aarch64.tbz2
cd ./Linux_for_Tegra/rootfs/
sudo tar xpf ../../tegra_linux_sample-root-filesystem_r32.5.1_aarch64.tbz2
cd ..
sudo ./apply_binaries.sh
  1. Generate keys using openssl
    For example, openssl genrsa -out rsa_priv.pem 3072

  2. Fuse device

sudo BOARDID=3668 BOARDSKU=0001 FAB=100 BOARDREV=H.0 ./odmfuse.sh -p -i 0x19 -k $XAVIER_KEYS/rsa_priv.pem --KEK2 $XAVIER_KEYS/KEK2 -S $XAVIER_KEYS/SBK jetson-xavier-nx-devkit-emmc
  1. Flash device
sudo BOARDID=3668 BOARDSKU=0001 BOARDREV=H.0 ./flash.sh --no-flash --sign -u $XAVIER_KEYS/rsa_priv.pem -v $XAVIER_KEYS/SBK jetson-xavier-nx-devkit-emmc mmcblk0p1 
cd bootloader
sudo bash ./flashcmd.txt

The above steps successfully enabled the secureboot option and flash the device. Now when we try to enable disk encryption, the device is not booting.

We followed the following steps on the same device.

  1. Used keys (generated using openssl)
➜  ~ cat $XAVIER_KEYS/KEK2 
3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5
➜  ~ cat $XAVIER_KEYS/SBK 
0xcxxxxxx3 0xdxxxxxx5 0x2xxxxxx1 0x4xxxxxx9
➜  ~ cat $XAVIER_KEYS/iv_hex_file
bad66eb4484983684b992fe54a648bb8
➜  ~ cat $XAVIER_KEYS/usr_eks.key 
bxxxxxx5dxxxxxx2dxxxxxx0cxxxxxx2
➜  ~ cat $XAVIER_KEYS/usr_flash.key 
0xbxxxxxx5 0xdxxxxxx2 0xdxxxxxx0 0xcxxxxxx2
➜  ~ cat $XAVIER_KEYS/ekb.key 
dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1
  1. Generate EKS blob and replace with the existing eks.img
➜  ~ python3 gen_ekb.py -kek2_key $XAVIER_KEYS/KEK2 -fv $XAVIER_KEYS/iv_hex_file -in_sym_key $XAVIER_KEYS/usr_eks.key -in_sym_key2 $XAVIER_KEYS/ekb.key -out eks.img

➜  ~ cp eks.img ../../../../../../../../Linux_for_Tegra/bootloader/
  1. Update partition table on bootloader/t186ref/cfg/flash_l4t_t194_spi_emmc_p3668.xml
       <partition name="APP" type="data">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 104857600 </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
            <align_boundary> 4096 </align_boundary>
            <unique_guid> APPUUID </unique_guid>
            <filename> system_boot.img </filename>
            <description> **Required.** Contains the boot partition. This partition must be defined
              after `primary_GPT` so that it can be accessed as the fixed known special device
              `/dev/mmcblk0p1`. </description>
        </partition>
        <partition name="APP_ENC" type="data" encrypted="true">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> APP_ENC_SIZE </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 0x8 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
            <align_boundary> 4096 </align_boundary>
            <unique_guid> APP_ENC_UUID </unique_guid>
            <filename> system_root_encrypted.img </filename>
            <description> **Required.** Contains the encrypted root partition("/"). </description>
        </partition>
  1. Update board config file jetson-xavier-nx-devkit-emmc.conf
source "${LDK_DIR}/p3668.conf.common";
disk_enc_enable=1;
EMMC_CFG=flash_l4t_t194_spi_emmc_p3668.xml;
EMMCSIZE=17179869184;
  1. Run flash script
➜  ~ sudo ROOTFS_ENC=1 BOARDID=3668 BOARDSKU=0001 BOARDREV=H.0 ./flash.sh --sign -u $XAVIER_KEYS/rsa_priv.pem -v $XAVIER_KEYS/SBK -i $XAVIER_KEYS/ekb.key --user_key $XAVIER_KEYS/usr_flash.key jetson-xavier-nx-devkit-emmc mmcblk0p1

There is no error on flashing device. But device cannot boot after flashing. Always showing a blank screen with nvidia logo.

Here is the console logs of flash command:
Console log of flash with disk encryption command.txt (115.5 KB)

Here are the serial port boot logs:
Boot logs from serial port.txt (25.8 KB)

Please help us to figure out the long existing issue. Thanks in advance.

1 Like

hello AkterHossain,

is it possible to upgrade the release version? there’s an issue to enable disk encryption with Xavier series on r32.5.1.
we had some changes update, would you please OTA the target for moving to the r32.6.1 (or, the later release) to include the bug fixes.
please see-also this thread for reference, Topic 205243.
thanks

hi @JerryChang,
Thank you for your reply.

Unfortunately upgrading from 32.5.1 is not possible since we are using AWS Sagemaker which is not compatible with later versions.

We are trying cboot patch now. Do you think it will work?

Hi @JerryChang,
We tried Cboot patch and still device is not booting.
We used ( L4T R28.3.2 GCC 4.8.5 Tool Chain for 64-bit BSP. Cboot builds successfully and we replaced to L4T/bootloader/cboot_t194.bin . Do you think we are missing anything else?

We can’t comment, as we mentioned previously, there’s an issue to enable disk encryption with Xavier series on r32.5.1, we had some changes update at later version, please OTA the target for moving to the r32.6.1 (or, the later release) to include the bug fixes. Thanks

Hi @kayccc ,
Yesterday we tried out 32.6.1. Flashing shows successful but still the device is not booting/discoverable. Our device is already fused using 32.5.1. Do you think it might cause the issue? I am sharing the latest steps below:

# Prepare host machine (Ubuntu 18.04_x64)
sudo apt-get update
sudo apt-get install qemu-user-static libxml2-utils cryptsetup libftdi-dev openssh-server

# Install L4T 32.6.1
mkdir ~/nvidia
cd nvidia
tar xf Jetson_Linux_R32.6.1_aarch64.tbz2
tar xvjf secureboot_R32.6.1_aarch64.tbz2
cd ./Linux_for_Tegra/rootfs/
sudo tar xpf ../../Tegra_Linux_Sample-Root-Filesystem_R32.6.1_aarch64.tbz2
cd ..
sudo ./apply_binaries.sh

# !!! IMPORTANT !!! We already used L4T 32.5.1 to fuse the device successfully
sudo BOARDID=3668 BOARDSKU=0001 FAB=100 BOARDREV=H.0 ./odmfuse.sh -p -i 0x19
  \ -k $XAVIER_KEYS/rsa_priv.pem 
  \ --KEK2 $XAVIER_KEYS/KEK2 
  \ -S $XAVIER_KEYS/SBK jetson-xavier-nx-devkit-emmc
  
# Flash with encryption enabled
sudo ROOTFS_ENC=1 BOARDID=3668 BOARDSKU=0001 BOARDREV=H.0 ./flash.sh --sign 
  \ -u $XAVIER_KEYS/rsa_priv.pem 
  \ -v $XAVIER_KEYS/SBK 
  \ --user_key $XAVIER_KEYS/usr_flash.key 
  \ jetson-xavier-nx-devkit-emmc mmcblk0p1

hello AkterHossain,

you should also have ‑i options to specifies the key (i.e. ekb.key) to be used for disk encryption.

Hi @JerryChang

Yes I did. Please refer my original post description.

hello AkterHossain,

the user_key is specified in eks.img.
Documentation: https://tegra-sw-opengrok.nvidia.com/source/xref/rel-32_l4t/3rdparty/trusty/app/nvidia-sample/hwkey-agent/CA_sample/tool/gen_ekb/example.sh to generate eks.img, the sym.key is the user_key.
Trusty retrieves user_key from eks.img, and loads the key into keyslot_14 for decryption.
when flashing, please use --use_key options to specify the user_key.

hence,
could you please double check you’re using correct keys.
thanks

hello AkterHossain,

may I also know which Ubuntu version it is,
for example, ubuntu-20.04 use new version of cryptsetup (v2.2.0), so the default luks type is luks2 instead luks1.
please execute $ dpkg -l | grep crypt to confirm the cryptsetup version.
thanks

Hi @JerryChang,

We are using ubuntu 18.04.6.

➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

While flashing our device with enabling disk encryption, we got something like cryptsetup is not found. So we installed it manually running sudo apt install cryptsetup. I think it installed latest version. Here is the output:

➜  ~ dpkg -l | grep crypt
ii  cryptsetup                                 2:2.0.2-1ubuntu1.2                              amd64        disk encryption support - startup scripts
ii  cryptsetup-bin                             2:2.0.2-1ubuntu1.2                              amd64        disk encryption support - command line tools
ii  gcr                                        3.28.0-1                                        amd64        GNOME crypto services (daemon and tools)
ii  gpg-agent                                  2.2.4-1ubuntu1.4                                amd64        GNU privacy guard - cryptographic agent
ii  libblockdev-crypto2:amd64                  2.16-2                                          amd64        Crypto plugin for libblockdev
ii  libcryptsetup12:amd64                      2:2.0.2-1ubuntu1.2                              amd64        disk encryption support - shared library
ii  libgcrypt20:amd64                          1.8.1-4ubuntu1.3                                amd64        LGPL Crypto library - runtime library
ii  libhcrypto4-heimdal:amd64                  7.5.0+dfsg-1                                    amd64        Heimdal Kerberos - crypto library
ii  libhogweed4:amd64                          3.4.1-0ubuntu0.18.04.1                          amd64        low level cryptographic library (public-key cryptos)
ii  libk5crypto3:amd64                         1.16-2ubuntu0.2                                 amd64        MIT Kerberos runtime libraries - Crypto Library
ii  libnettle6:amd64                           3.4.1-0ubuntu0.18.04.1                          amd64        low level cryptographic library (symmetric and one-way cryptos)
ii  libsodium23:amd64                          1.0.16-2                                        amd64        Network communication, cryptography and signaturing library
ii  libvolume-key1                             0.3.9-4                                         amd64        Library for manipulating storage encryption keys and passphrases
ii  openssl                                    1.1.1-1ubuntu2.1~18.04.15                       amd64        Secure Sockets Layer toolkit - cryptographic utility
ii  python3-asn1crypto                         0.24.0-1                                        all          Fast ASN.1 parser and serializer (Python 3)
ii  python3-crypto                             2.6.1-8ubuntu2                                  amd64        cryptographic algorithms and protocols for Python 3
ii  python3-cryptography                       2.1.4-1ubuntu1.4                                amd64        Python library exposing cryptographic recipes and primitives (Python 3)

What do you think now? Is it okay or we need to do something?

hello AkterHossain,

could you please also check the UDA partition’s size for disk encryption configuration,
it’s the flash configuration file, flash_l4t_t194_spi_emmc_p3668_enc_rootfs_ab.xml.
please modify the size as following,

        <partition name="UDA" type="data" encrypted="true">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 122880000 </size>

please also review the disk_encryption_helper.func,
please check it’s using luks1 to maintain backward compatibility,
for example,

	# Add the LUKS header
	eval ${GEN_LUKS_PASS_CMD} | ${CRYPTSETUP_BIN} \
		--type luks1 \
		-c aes-cbc-essiv:sha256 \
		-s 128 \
		--uuid "${__rootfsuuid}" \
		luksFormat \
		${loop_dev};
	chkerr "Add LUKS header on ${__localsysfile} failed.";

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.