Flashing DTB on Jetson TX2 / L4T 32.4.4

[blevinstein]

Hey TX2 experts,

I am working with a Jetson TX2 with custom kernel and device table (because I have additional i2c devices attached to my Jetson TX2), and I am having problems with my flashing flow after updating to the latest Nvidia software on Jetpack 4.4.

My problem manifests as follows:

1. I install Jetpack 4.4 using a patched version of the L4T 32.4.4 kernel and DTB. After installation, system software includes nvidia-l4t-bootloader and other Nvidia packages at version 32.4.3.
2. I run “sudo apt update && sudo apt upgrade” to get latest software packages. Various Nvidia packages are upgraded (see partial output of “apt list --upgradable” nvidia-l4t-3d-core/stable 32.4.4-20201027211332 arm64 [upgradable from: 32.4.3-2 - Pastebin.com)
3. After restarting, my device table (as shown at /proc/device-tree and /boot/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb) has been replaced with a new device tree, which is not patched. The first few lines of the new DTB are here: /dts-v1/; / { nvidia,fastboot-usb-pid = <0xee16>; compatible = "nvidia,q - Pastebin.com

This problem should be recoverable; I think I just need to re-flash my custom DTB. My team has an existing process for re-flashing the kernel and DTB, using an attached “flash station”:
4. [On the flash station] Build the patched DTB and kernel.
5. [On the flash station] Copy the kernel Image and DTB to the following locations:
JetPack_4.4.1_Linux_JETSON_TX2/Linux_for_Tegra/kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb
JetPack_4.4.1_Linux_JETSON_TX2/Linux_for_Tegra/kernel/Image
6. [On the Jetson TX2] Copy the kernel Image to the boot sector:
sudo cp Image /boot
7. [On the flash station] Run this flash command:
sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1

Before I installed the updates in step 2, I could use this flashing flow to replace the DTB reported at /proc/device-tree. After step 2, this process appears to have no effect.

Perhaps somebody can help me understand:
A. Is this flashing flow sound? Is there some other step we are missing when flashing our custom DTB?
B. Are there important changes in the Jetpack 4.4 DTB between L4T 32.4.3 and 32.4.4? Is there some change in the location where the DTB is loaded from?
C. Any other suggestions for how to add my custom changes to the device DTB? Does Jetpack support a Device Table Overlay (DTO)?

Thanks in advance!
Brian

[blevinstein]

Based on this post

method to modify/use different device tree in R28.1

The dtb file is used in different locations now. Some of its content has moved to the boot loader itself, so this file named in extlinux.conf or found in “/boot” is not a complete reflection of what your system used. See the above posts about using flash instead of extlinux.conf. You’ll see in the posts earlier in this thread what basically is an evolution of getting both the TX1 and TX2 to use the same R28.1 L4T version (the driver packages differ, but the kernel and rootfs are now the same). …

I am aware that the DTB is stored in multiple places. It is perhaps notable that my kernel-and-DTB flashing flow does not appears to modify /boot/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb.

I also tried identifying the kernel-dtb partition on my TX2 and flashing directly using dd as suggested in this post

How to use UART0 as normal uart port on R28.1?

Hi Arlen0615, Please try to put the dtb into kernel-dtb area using dd on TX2: sudo dd if=your_dtb.dtb of=/dev/mmcblk0p15 And remove the “FDT” line in the extlinux.conf. I heard that the DTB will be used by both kernel and u-boot, so the “FDT” way doesn’t fully work as it can’t pass the DTB to u-boot.

but that left my TX2 in an unbootable state.

[linuxdev]

I don’t know about the specifics of this case, but in general, when testing, always leave the original boot entry in extlinux.conf untouched, and then add your own boot entry.in “extlinux.conf”. I say this for reasons of making it easier to recover when something does go wrong. There is no reason you could not have your boot entry twice, using two different names, and perhaps one could be the default entry which replaces the one the dev kit board would have used. Someone recently discovered you can have around five entries. The point being to have a backup which won’t be replaced during testing if the apt update overwrites something.

Note that in the R32.x L4T there has been a lot of work done towards secure boot and towards redundancy. At this time a partition for placing the kernel in and for placing the device tree in is optional if the fuse has not been burned. Provided the fuse is not burned, all of your testing could be files in “/boot” and edits to extlinux.conf if file names changed. Boot would first look for that content, and if not found, would only then revert to looking for the version in device tree (again, if secure boot is enabled, only partitions are allowed).

It would probably be useful to see a copy of any boot entries in “/boot/extlinux/extlinux.conf”:

• When your modified setup is in place and working, and
• When an update has overwritten your work.

We could then compare. You would also want to post if the “.dtb” file named by any FDT key/value pair entry exists for the before and after cases.

If you have a need to protect your custom device tree and/or kernel, then likely you are going to need to make them as packages such that the default NVIDIA version won’t qualify for updating it (replacing the NVIDIA versions with apt/dpkg packages rather than simply overwriting what they installed originally prior to your modification).

As an alternative, if packages modify kernel or device tree, then after yours is in place you could blacklist update of those packages.

[blevinstein]

The contents of my extlinux.conf file are unchanged after running the update (see TIMEOUT 30DEFAULT primaryMENU TITLE L4T boot optionsLABEL primary - Pastebin.com). Thanks for the tip about adding a backup boot option.

As far as I can tell, it should be booting from /boot/Image, but the /proc/device-tree does not appears to match the DTB in my boot image.

I can verify that /boot/Image is the custom kernel I have copied over (e.g. by using sha1sum), so I’m not sure where the bad DTB is being loaded from.

[JerryChang]

hello blevinstein,

below are correct steps for updating kernel image and also device tree blob binary.

@blevinstein

“flash station”:
4. [On the flash station] Build the patched DTB and kernel.
5. [On the flash station] Copy the kernel Image and DTB to the following locations:
JetPack_4.4.1_Linux_JETSON_TX2/Linux_for_Tegra/kernel/dtb/tegra186-quill-p3310-1000-c03-00-base.dtb
JetPack_4.4.1_Linux_JETSON_TX2/Linux_for_Tegra/kernel/Image
6. [On the Jetson TX2] Copy the kernel Image to the boot sector:
sudo cp Image /boot
7. [On the flash station] Run this flash command:
sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1

note, if you’re using Over-The-Air (OTA) update. it’ll also upgrade the BSP following a list of OTA update packages for BSP on Jetson devices. it’s nvidia-l4t-kernel-dtbs package to overwrite your device tree changes with the latest JetPack release version.

you may perform flash commands to flash a specific partition after OTA update.
please also check /boot/extlinux/extlinux.conf, it’s CBoot functionality to include a default booting scan sequence,
it’s by default to load device tree binary from FDT entry, otherwise, the kernel-dtb binary is loaded from the kernel-dtb partition which you’d flash to.
thanks