Set boot sequence for Xavier NX

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi everyone,

I have some questions about boot order for the Xavier NX.
The doc does not define how boot order can be set for the NX but it defines this for all other Jetsons.
(see here)
It specifies one category for the AGX and one for “other” but at other only names “Nano, TX1 and TX2” so NX is undefined.

  1. Does the the NX not have the option to set the boot order?
  2. I read the this post and am confused what happens should a device not be bootable, will the jetson be stuck in a boot loop? I assume @linuxdev suggested that there should be no boot folder in order for the device to be skipped, correct? What if it is “normally” bootable but corrupted? Can the Jetson skip the device after a number of tries?

Thanks for your help!

2

To be certain, boot information differs between something running on SD card versus an eMMC system. NX has both models, so you’d need to specify.

Also, Xavier and NX boot directly through CBoot, and do not use U-Boot. The other Jetsons use U-Boot.

It can be hard to keep up with boot documents considering it depends on release version, U-Boot versus CBoot, and install media type.

I don’t know what all of the options are, but SD card models are much more limited than eMMC models. Typically there are environment variables in the boot software (CBoot tries to emulate much of the U-Boot environment, but does not always), and these can be used for boot order search…but only if it is supported on that release and hardware type.

Someone else will need to answer, but if you state if this is the SD card model (versus eMMC), and what software release of JetPack/SDKM you are looking at, then someone will be able to answer.

3

hi @linuxdev

thanks for your answer!

We want to priorities external eMMC over internal eMMC in the final product (or the other way around) depending on our testing results.

“Xavier and NX” do you mean AGX and NX?

The rest of the message confuses me, I don’t relly know what the difference would be. We just want to set a different boot sequence than the default one and the documentation doesn’t specify wether this is possible for the NX or not. How does this depend on the SD card model or eMMC model?
Even if this regards the boot folder, shouldn’t that be the same on any memory? If the system wants to boot it looks for the boot folder (as specified in the post I linked), if that is missing it skips the device. How is this dependent of the memory model or type?

Your answer leaves me a bit confused, but surely you are pointing something out I have been missing in the last few days so further clarification is very much appreciated! Thanks!

4

Which module and the type of external eMMC does matter Cboot has its own device tree in which you can specify the device order but only the nvme entry allows you to specify which instance. So if one of your eMMCs is an sdcard and the other is the device internal eMMC then you’re fine. If not, you’re going to have to experiment or examine the cboot sources to see what the behavior is.

From bootloader/cbo.dts:

/ {
	compatible = "nvidia,cboot-options-v1";
	boot-configuration {
		/* nvme boot-order can be in "nvme:C<n>", "nvme:pcie@<addr>", or "nvme" format */
		boot-order = "sd", "usb", "nvme", "emmc", "net";
		tftp-server-ip = /bits/ 8 <192 168 0 1>;
		dhcp-enabled;
		/* remove dhcp-enabled if uncommenting static-ip configuration */
		/* static-ip = /bits/ 8 <0 0 0 0>;
		 * ip-netmask = /bits/ 8 <255 255 255 0>;
		 * ip-gateway = /bits/ 8 <0 0 0 0>;
		 */
	};
};

There’s also a “boot_pt_guid” attribute not shown in the sample where you can specify a specific GUID.

To use this, you need to modify the bootloader/cbo.dts file then flash the bootloader-dtb partition.

1 Like
5

Yes. I tend to say Xavier for the full Xavier, and NX for the NX form factor. AGX was the first Xavier, and is a bit more powerful than the NX.

Any time you want to boot the boot software will eventually get to a step where it follows rules to find a boot device. In many Jetsons the CBoot code transfers execution to U-Boot, and the functions of U-Boot (such as environment variables and macro expansion of those variables) are what determine if a boot device has been found. However, the Xavier and NX do not have U-Boot. Instead, some of the U-Boot behavior was ported from U-Boot into CBoot (not a direct port, but to the user it looks the same).

CBoot, when it performs with U-Boot functionality, is usually a subset of the full U-Boot capabilities. If it works in U-Boot, then it is worth testing if it works in the CBoot version, but it isn’t a guarantee. Those parts where one might set up a boot order by changing the environment variables in the bootloader console most likely will work identically regardless of it being CBoot or U-Boot, but there is a catch: Suppose (this is just a contrived example for illustration) U-Boot is using a USB driver for an external disk to find the boot partition, and suppose CBoot does not have a USB driver for that external disk…then the environment would make the same attempt in both cases, but the CBoot would have failed…not for lack of trying, but for lack of support in infrastructure drivers.

Historically, CBoot has always been there, and the only thing which changed for the CBoot-only-but-not-U-Boot models is that it was that CBoot used to always transfer boot to U-Boot. Cutting out U-Boot meant that some of that end user friendliness of U-Boot needed porting to CBoot. You might wonder why this complication was added. I think the answer centers on A/B boot redundancy and secure boot with a fuse. It would be far easier to put some U-Boot console environment into CBoot than it would be for U-Boot to understand the workings of a Jetson with secure boot fuses or A/B partition redundancies.

For the boot code to support A/B redundancy, or secure boot with fuses, the hardware to deal with it must be present. For eMMC models this requirement is met. However, the less expensive SD card models do not have all of the hardware. In part I don’t think this would be possible because secure boot mandates being able to stop access to some of the persistent memory, and there is no way you could stop access to the SD card. The code for the SD card boot, as a result, is very different from the code for eMMC with support of both redundancy and secure boot.

I can’t say for certain, but so far as picking between two different devices to boot to, I’m thinking one could change the boot sequence by changing the environment variables you see in the bootloader console over the serial console. But you’d have to try it to know for sure, or someone from NVIDIA would need to say that particular case works for a given L4T/JetPack/SDKM release and the particular Jetson model.

2 Likes
6

Hi @linuxdev,

thanks again for your extensive answer.
Now I understand the root of the problem.
I also previously miss interpreted that you where talking about SD Card models as in types of SD Cards.

Here are some informations that you have been asking for (and hopeing that someone from NVIDIA might help) For development we use the SD Card Models of the NX but obviously we want to use the eMMC version for the final product. On the eMMC model an external eMMC shall be connected via SDIO. For redundancy purposes we would like to be able to boot from the internal an the external eMMCs and set the boot sequence depending on what reliability we observe during our testing activities. (I am aware that the bootloader will always be on the internal eMMC)

We have not yet decided on a certain release for development but would likely choose the most recent one.

@gtj, you showed the boot configuration in your answer. If I interpret your answer correctly we would need to figure out ourselfs how the boot order works when we use two eMMC memories (where neither is an SD Card) is that correct?

THANKS!

7

Yes. I don’t have eMMC versions to test but in your scenario, the second eMMC connected to the SDIO (sdmmc3?) may be considered an sd card but I have no way to tell.

Here’s the link to the cboot boot order stuff…
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/getting_started.html#wwpID0E0QI0HA

One thing to note… Both the Devkit(SDcard) and Production(eMMC) version of the module have an onboard 32MB QSPI-NOR flash device. Depending on which configuration you use to flash the device, the bootloaders and associated files can be located in the QSPI-NOR or the SDcard/eMMC.

Here’s a link to the NX partition layouts for each configuration:
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/part_config.html#wwp102418

1 Like
8

This link addresses my initial problem, the document states that this is only for AGX and the “other” section states its only for “TX1, TX2 and Nano”… so NX is also working like the AGX?

I’ll look into the partition layouts! Thanks!

9

Yep. The NX is exactly like the AGX in that respect. That section of the document needs updating.