Unexpected Behavior when deriving keys from fuses inside of OP-TEE

eth4 · September 21, 2022, 8:10am

Hi,

I am experiencing unexpected behavior on the Xavier NX module with L4T R35.1 while developing for OP-TEE. Specifically I am trying to use the OP-TEE PTA jetson_user_key_pta.c to derive a root key from KEK256 as described in the developer documentation. I have added BctKEKKeySelect = 1; to the tegra194-br-bct-qspi.cfg to allow accessing of KEK0 and KEK1 as a single KEK256 register.

I modified jetson_user_key_pta.c to print out the derived key inside hwkey_derivation_process immediately after the call to tegra_se_nist_sp_800_108_cmac_kdf. I have not modified the original call:

        rc = tegra_se_nist_sp_800_108_cmac_kdf(SE_AES_KEYSLOT_KEK256,
                                               TEGRA_SE_KEY_256_SIZE,
                                               "Derived 256-bit root key",
                                               "256-bit key",
                                               TEGRA_SE_KEY_256_SIZE,
                                               demo_256_rk);

I then burned fuses on a development Xavier NX module with the following values:

PublicKeyHash: 0000000000000000000000000000000000000000000000000000000000000000
SecureBootKey: 00000000000000000000000000000000
Kek0: 41414141414141414141414141414141
Kek1: 42424242424242424242424242424242
Kek2: 00000000000000000000000000000000
Kek256: 4141414141414141414141414141414142424242424242424242424242424242
BootSecurityInfo: 00000000
JtagDisable: 00000000
SecurityMode: 00000000
SwReserved: 00000000
DebugAuthentication: 00000000
OdmId: 0000000000000000
OdmLock: 00000000
ReservedOdm0: 00000000
ReservedOdm1: 00000000
ReservedOdm2: 00000000
ReservedOdm3: 00000000
ReservedOdm4: 00000000
ReservedOdm5: 00000000
ReservedOdm6: 00000000
ReservedOdm7: 00000000
ReservedOdm8: 00000000
ReservedOdm9: 00000000
ReservedOdm10: 00000000
ReservedOdm11: 00000000

Upon the first boot of the module after flashing I get the following derived key:

M/TC: hwkey_derivation_process: Successfully derived the 256 root key!
______________________________________________________________
M/TC: 31
M/TC: ee
M/TC: da
M/TC: 84
M/TC: 61
M/TC: e8
M/TC: ba
M/TC: b5
M/TC: 5f
M/TC: 10
M/TC: f5
M/TC: 73
M/TC: 4c
M/TC: b1
M/TC: 27
M/TC: ad
M/TC: a0
M/TC: 1
M/TC: 3a
M/TC: ed
M/TC: 95
M/TC: ac
M/TC: a5
M/TC: e9
M/TC: e7
M/TC: ed
M/TC: c7
M/TC: 0
M/TC: cf
M/TC: 73
M/TC: e2
M/TC: 3e
M/TC: ______________________________________________________________

However every subsequent boot results in a completely different key printed to the console, for example:


M/TC: hwkey_derivation_process: Successfully derived the 256 root key!
______________________________________________________________
M/TC: e5
M/TC: 9e
M/TC: df
M/TC: 44
M/TC: 45
M/TC: d
M/TC: fe
M/TC: c3
M/TC: 20
M/TC: d1
M/TC: 63
M/TC: 84
M/TC: 45
M/TC: 7b
M/TC: d2
M/TC: 72
M/TC: 68
M/TC: 27
M/TC: e4
M/TC: bb
M/TC: 29
M/TC: 56
M/TC: b3
M/TC: e6
M/TC: e9
M/TC: c1
M/TC: 1e
M/TC: e4
M/TC: 76
M/TC: b7
M/TC: b9
M/TC: 13
M/TC: ______________________________________________________________

Each new boot results in a different and unique key being printed. If I reflash the module, the first boot returns the key starting with 31 ee, but every subsequent key is seemingly random. If I call tegra_se_clear_aes_keyslots before the KDF, it correctly returns 7e9e0cdcc3d8fa5376e46a8e95095a51ed56485bff4a4f02304c4b32b9e2f834 which is the KDF on a NULL key. This leads me to believe that the behavior of the Tegra Security Engine is correct, and that there is either an issue with the fuse reading or KDF implementation in OP-TEE.

I have verified that the counters used for the KDF function inside optee_os/core/drivers/tegra/t194/tegra_se_aes.c are not persisting across reboots.

Can anyone at NVIDIA verify that they are indeed getting deterministic KDFs from tegra_se_nist_sp_800_108_cmac_kdf inside of OP-TEE?

Thanks

JerryChang · September 22, 2022, 1:59am

hello eth4,

could you please refer to below flow,

Prepare a 256-bit key
Using SW-based nist_sp_800_108_cmac_kdf to derive a key
Write the 256-bit key into keyslot by tegra_se_write_aes_keyslot(KEY_BUFFER, TEGRA_SE_KEY_256_SIZE, TEGRA_SE_AES_QUAD_KEYS_256, SE_AES_KEYSLOT_KEK256)
Using HW-based tegra_se_nist_sp_800_108_cmac_kdf to derive another key
The keys in steps 2 and 4 should be identical.
Reboot and test again.

could you please test again with the same scenario with the same key in KEK256 and skip step 3.
thanks

eth4 · September 22, 2022, 9:21am

Thank you for your reply:

I performed the tests you asked.

The first series of tests I used a 256-bit key with all bytes 0x43. The output from software nist_sp_800_108_cmac_kdf and HW tegra_se_nist_sp_800_108_cmac_kdf were exactly the same every single boot. Every time the key started with 28 c3.

In other words, the first series of tests using keys loaded from SW passed.

Then I added the fused KEK256 key to the file. I tried two combinations to account for endianness:

static uint8_t kek256_key[TEGRA_SE_KEY_256_SIZE] = {
        0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42,
        0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42,
        0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41,
        0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41,
};

and

static uint8_t kek256_key[TEGRA_SE_KEY_256_SIZE] = {
        0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41,
        0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41, 0x41,
        0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42,
        0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42, 0x42
};

After the first boot after flashing, the HW KDF always produced a seemingly random key, while SW KDF returned the same key. Additionally, neither combination of the above key definition resulted in SW KDF key being the same as the HW KDF.

As a reminder, I have not burned the SecurityMode fuse on this module. Could this affect the way the keys are being read? It seems like there might be an issue with fuse loading.

Below is the source code for the function I implemented just in case.

static TEE_Result test_hw_vs_software_fuse(void)
{
        TEE_Result rc = TEE_SUCCESS;
        uint8_t derived_sw_key[TEGRA_SE_KEY_256_SIZE] = { 0 };
        uint8_t derived_hw_key[TEGRA_SE_KEY_256_SIZE] = { 0 };

        MSG("%s: Now going to perform SW KDF using kek256_key buffer...", __func__);
        rc = nist_sp_800_108_cmac_kdf(kek256_key,
                                        TEGRA_SE_KEY_256_SIZE,
                                        "context",
                                        "label",
                                        TEGRA_SE_KEY_256_SIZE,
                                        derived_sw_key);

        if (rc != TEE_SUCCESS)
                EMSG("%s: error doing software kdf on the kek256_key (%x)", __func__, rc);
        else {
                MSG("%s: successfully sw derived key:\n ______________________________________________________________", __func__);
                for (unsigned int i = 0; i < TEGRA_SE_KEY_256_SIZE; i++) {
                        MSG("%x ", derived_sw_key[i]);
                }
                MSG("______________________________________________________________");
        }
        MSG("%s: Now going to perform HW KDF using fuse array...", __func__);
        rc = tegra_se_nist_sp_800_108_cmac_kdf(SE_AES_KEYSLOT_KEK256,
                                                TEGRA_SE_KEY_256_SIZE,
                                                "context",
                                                "label",
                                                TEGRA_SE_KEY_256_SIZE,
                                                derived_hw_key);

        if (rc != TEE_SUCCESS)
                EMSG("%s: error doing hardware kdf on the kek256_key (%x)", __func__, rc);
        else {
                MSG("%s: successfully hw derived key:\n ______________________________________________________________", __func__);
                for (unsigned int i = 0; i < TEGRA_SE_KEY_256_SIZE; i++) {
                        MSG("%x ", derived_hw_key[i]);
                }
                MSG("______________________________________________________________");
        }
        MSG("%s: All done, thanks for playing....", __func__);
        tegra_se_clear_aes_keyslots();
        return rc;

}

Thanks

JerryChang · September 26, 2022, 8:32am

hello eth4,

it is BootRom to load in fuse keys into SE key slots. it might be SecurityMode fuse need to be burned to indicate fuse keys are ready.
is it possible to burn the fuse then test again?

eth4 · September 26, 2022, 8:33am

We tested with burning the security mode fuse and it has not changed the behavior.

eth4 · September 26, 2022, 2:13pm

We’ve performed some additional tests and have narrowed the issue further.

When a “cold boot” is performed (that is, when an unpowered carrier board is given power and the proper power sequencing is performed to start the Jetson) we have found that the fuses are read/processed incorrectly. However, when Jetson is already on, and a reset of the chip is triggered either through the linux kernel (sudo reboot now) or from a reset switch on the carrier board, the KDF becomes deterministic. We still don’t know if the correct values are being read from the fuses (the keys produced do not seem to match the expected ones using software KDF) but they are at least consistent and clearly from a non NULL key.

We have tested this behavior on both the NVIDIA official devkit, as well as on a 3rd party devkit. Additionally we have replicated this behavior on our own carrier board. I can confirm that at least on our carrier board we are performing the power sequencing as defined in the Xavier NX Product Design Guide.

It would be very helpful if someone could confirm that they have reproduced our findings.

JerryChang · September 28, 2022, 1:52am

hello eth4,

just for confirmation, may I know which JetPack release version you’re using?
is it JP-5.0.2 production release, thanks

eth4 · September 28, 2022, 7:35am

As mentioned above, we are using the 35.1 BSP. It was collected with JP-5.0.2, but all of these issues occur before the kernel has started, so I hope that the version of CUDA or TensorRT wouldn’t affect it. We have replicated the issue with older versions of the kernel (4.9) while still using OP-TEE, and we have replicated the issue when a corrupt/non-bootable kernel + rootfs are present on eMMC.

eth4 · October 5, 2022, 5:47pm

Hi, we are still seeing this issue. Any updates?

JerryChang · October 6, 2022, 2:51am

I’m closing this topic due to there is no update from you for a period, assuming this issue was resolved.
If still need the support, please open a new topic. Thanks

hello eth4,

I’ve arrange resources to check this internally, could you please also share a simple package for us to recreate the issue locally?