How to test custom carrier board with customer golden image?


we designed a carrier board for use /w Jetson Xavier NX production modules.

Those modules have no integrated SD-Card option, so we integrated one on our carrier.

Because our customer has some closed source integrated on rootfs, which he wants us to integrate in the next batches of boards we are going to deliver.
For efficiently integrating this rootfs in our production and testing process, we want him to provide us a golden image on a SD-Card which we then can use at our site to boot and test the systems with this image.

The problem now is, that all the SD-Card flash and boot methods mentioned in the Jetson Linux Developer Guide documentation (r35.3.1) are meant to be used with the developer kit SoCs, not the production modules.

How can we make it possible for him to give as a bootable SD-Card, working with the production module and our external SD-Card reader?

I’m not really answering your quesiton, but hoping to add some understanding on the topic.

Jetsons don’t have an actual BIOS the way a desktop PC does. That means this content exists in partitions (or sometimes QSPI memory on the module). A BIOS would be able to pick among boot devices, but for a Jetson, the flash itself more or less provides a “pointer” to the initial boot point. That means you can’t just make a bootable image and put it on an SD card and have it “just work”. It depends on getting that “pointer” (the boot target) to point at that media.

Since you are using R35.x, you are at least using a UEFI boot. This means the “fake” BIOS has an abstraction layer, and once you reach that, you might be able to pick more or less the same as if it has a BIOS, but I couldn’t tell you the steps for that (and that is not why I am adding my comment, but it is relevant to the topic). Much changed when going from earlier R32.x to R35.x. You might have some luck with UEFI customization, not sure.

My reason for comment is because of device tree differences. Many devices are “plug-n-play”, and can self-describe. Drivers can work with those devices at whatever address self-describes. SD card slots are not plug-n-play, and do not self-describe. This means your device tree is critical, as this is what tells drivers what is there and which driver is compatible. Because the SD card slot is on the carrier board, and not on the module, you’ve reached the correct conclusion that you cannot use the dev kit SD card boot methods.

Incidentally, boot is its own operating system (it’s only purpose though is to set up environment and overwrite itself with the next o/s). This means that using hardware in boot (such as an SD card or ethernet or SATA drive) must be able to find hardware, and the driver must be present during the boot stages. If both setup (device tree) and driver are not present at any stage, then that stage cannot use that device. You probably use the same SD card driver software in both Linux and boot stages, but the issue of finding the hardware is a problem because this hardware is not located where the dev kit software looks (it isn’t a dev kit).

You have a couple of choices: (A) Booting from SD card, which is problematic, and not really the way it should be, although you could do this with boot customization adding knowledge of the SD card, or (B) booting from eMMC and testing SD card function. I recommend the latter.

Note that even if boot stages have no concept of how to find the SD card (perhaps because device tree customization is not available), then should you get to the Linux kernel, the device tree at that stage and setup to find the SD card is more or less trivial. Yes, boot stages could also read that same device tree, but you might need to customize something in UEFI to do that. Customizing and adjusting things in Linux is far easier than doing so in boot stages.

If you were to give him just a bootable image, you’d have a lot of headaches and problems. One really needs to flash not only the o/s, but also all of that boot content, and device tree. They have to work together. It’s easy for a PC to self-flash with something like Windows or Linux, but installing Windows or Linux to a PC does not simultaneously flash the BIOS. For your golden image to work, you must also have present the “golden BIOS equivalent”.

There are several manufacturers of third party carrier boards for Jetsons of different models. You will see some variations:

  • Some copy the dev kit carrier board, and say to use dev kit JetPack software.
  • Some provide a patch to add to the official dev kit flash software (JetPack). Then one patches the NVIDIA content before flashing (usually the patch is device tree, but it can include other things, such as a custom kernel).
  • Some take the official dev kit flash software and customize it as their own, and then package it, and have the end user download this from them.

The lesson there is that I think instead of giving them a “golden image” you probably need to give them either a “golden patch” to a specific L4T release, or else an entire edited flash package. The patch has the advantage that they will have a roadmap of how to add their own patches should they desire. A fully customized installer does not have that advantage. Bisecting changes also becomes easier.

Should you choose to make the SD card itself bootable, you could modify the UEFI as mentioned earlier. It is far easier to just use the SD card after boot and use eMMC to boot on. Even if the device tree is wrong or incomplete, the eMMC and the module content is probably still correct, and boot would still occur, although the SD card might not function until doing something like adding a device tree overlay. That’s a lot simpler than modifying UEFI every time you get a new release.

As a company, you too would get some valuable information if you create a patch to the NVIDIA software, rather than just making a completely customized installer. If you were to make a completely customized installer, then you’d be better off to start with a patch to understand what goes into this.

A lot depends on whether the end user wants to boot from SD card (either by setting a pointer during flash, or by chain loading whereby eMMC is flashed, but still points to the SD card for rootfs), or if you will boot from eMMC. You can get into using an initrd image as well, which acts as a sort of “adapter” between boot and Linux. The initrd itself could contain device tree changes.

Note that if the system has already been flashed with all of the correct content for boot partitions or other firmware, then flashing on command line and saying to flash only the rootfs to the eMMC is an option. One would merely mount the SD card on the host PC, and then use dd to copy the partition as file “Linux_for_Tegra/bootloader/system.img” and use the correct command line flash. Optionally, you could create a “sparse” image from the SD card, and flash the same way (the sparse image flashes exactly like the raw image, except that it is faster because empty space is not reproduced…the size of a raw image is the size of the entire partition, whereas the size of a sparse image approaches the size of a raw image as the empty space is filled).

1 Like

Thank you for this broad and thorough summarising answer on my question.

Concluding, we will focus on bringing the NVMe boot to run and interchange the system.img with our customer to bring his application to run on our carrier board /w our BSP and kernel customisations.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.