Jetson Nano boot process maintenance

Hi,

I have a Jetson Nano Developer Kit (4GiB, SD-card) and I’d like to try a different U-Boot. The SD-card image supplied by Nvidia contains multiple partitions and I assumed that one of these would contain U-Boot. But I’ve now found that the Nano will boot when the SD-card contains only one partition (the root file system) and that all the boot code (Tboot, Cboot, U-Boot, etc) exists in a 4MiB flash chip presumably on the module.

Can anybody tell me how I can reload this flash RAM if it ever gets wiped ? Also, is it possible to get the Nano to boot using the code on the SD-card if booting via the flash RAM fails for some reason ?

If possible I would like to rebuild the flash RAM code to contain an updated U-Boot. Is there any way to do this ?

Thanks,
Steven

It is useful to know some background…

The SD card models of Jetsons (dev kit versions…regular modules have eMMC and no SD card) have QSPI memory on them. Some or all of the boot code lives on that memory on the dev kit module itself, while the rest is on the SD card. Originally only part of the boot content was on the QSPI, and the rest were the extra partitions you see on the SD card. Later releases put all boot content on QSPI, and the SD card then had only the rootfs partition and none of the boot content.

If you see an SD card with extra partitions, then the Nano’s QSPI must be flashed with one of the earlier releases which had the layout matching that SD card. Newer releases will have no extra SD card partitions, and thus the Nano’s QSPI must be flashed with one of the newer releases using that layout.

Flashing the QSPI is done via JetPack/SDKM from a Linux host PC. See this for releases:
https://developer.nvidia.com/embedded/jetpack-archive

The only partition on your SD card which needs to be backed up is the root filesystem partition. The others are “standard” and can be reproduced with either one of the pre-made SD card images or the correct release of JetPack/SDKM. However, let’s pretend you have the SD card on your Linux host PC, and that it shows up as “/dev/sdb” (you’ll have to adjust for your case). Any backup will be enormous if saved as a file, but this would copy the entire SD card in such a way that it could be used as an image to another SD card:
sudo dd if=/dev/sdb of=/some/temp/location/backup.img bs=10M
(the “bs=10M” is optional…results will not differ, but using a larger buffer can make copy faster…I stick to the minimal default size of 512 only when doing data recovery on a failing device)

The image file created will be the exact size of the entire SD card, so be careful to have enough disk space before starting. You could use an external USB drive for the destination file.

Once done that dd file can be put onto an SD card of equal or larger size and it will be an exact clone.

Hint: If the rootfs is “/dev/sdb1”, then you could clone just that partition with the same command, but change the input file name to “/dev/sdb1” instead of “/dev/sdb”. In all cases results are best if the partition is not mounted during the copy.

Thanks for the details explaining the apparent duplication of the boot components.

I’m thinking that if I want to change U-Boot (and perhaps the DTBs) it would be better if they were on an SD-card rather than the QSPI because a) excess flashing might wreck the QSPI which is critical and non-replaceable and b) if a change doesn’t work it will be easy to go back to a previous SD-card. Is it possible to install an old minimalist version of the on-board boot code and if so do you know the last JetPack version that contained this code ?

I’ve installed SDKManager on an x64 PC but in the GUI interface I can’t see any option just to flash the QSPI. I’ve checked the CLI options in the on-line manual but nothing seems applicable. Is there an option just to flash the QSPI ? Or can I run tegraflash manually to achieve this ?

Cheers,
Steven

Whether or not that content appears on the SD card versus on QSPI depends on the release version. This is not something you can easily change. It is doubtful that you would flash the QSPI so much that it destroyed it, and the QSPI is far sturdier than you are expecting.

Earlier releases are the only releases in which some of the content is on the SD card. I don’t know if U-Boot itself was on the SD card back then, but this is the only content in which you could put U-Boot on the SD card (and maybe not even the earlier releases, not sure).

Flashing just QSPI can be done on command line. A typical flash names the flash type based on the “.conf” file. For example, sometimes someone might flash with this command line from the “Linux_for_Tegra/” directory:
sudo ./flash.sh jetson-nano-devkit mmcblk0p1

The “mmcblk0p1” would not change, and isn’t even used sometimes, but the target of “jetson-nano-devkit” tells flash to look for the file “jetson-nano-devkit.conf”. Here are some possible “.conf” files for a Nano (you can simply look at your “Linux_for_Tegra/” directory for “*.conf” files):

Linux_for_Tegra/jetson-nano-2gb-devkit.conf
Linux_for_Tegra/jetson-nano-devkit.conf
Linux_for_Tegra/jetson-nano-devkit-emmc.conf
Linux_for_Tegra/jetson-nano-emmc.conf
Linux_for_Tegra/jetson-nano-qspi.conf
Linux_for_Tegra/jetson-nano-qspi-sd.conf
Linux_for_Tegra/jetson-tx1.conf
Linux_for_Tegra/jetson-tx1-devkit.conf

Thus you could also choose target “jetson-nano-qspi”. Those files are in fact human readable and you can see some of the options picked by using a specific target, but I don’t think you will get to choose when content goes to SD card versus QSPI.

Hi,

I downloaded JetPack 4.4 which is apparently the oldest available for a Jetson Nano Developer Kit and used your command ("./flash.sh jetson-nano-qspi.conf p3450 mmcblk0p1") which flashed the QSPI successfully. It now contains eight copies of an initial loader and two copies of tegraboot only. Previously there was much more code and data after the tegraboots. The system still boots from my multiple partition SD-card but fails with missing partition errors when loaded with my single partition (APP) SD-card. Thanks for your advice here - it certainly will make testing easier.

My next hurdle is how to update DTBs and U-Boot on the SD-card. On my TK1 I found using dd to update a partition failed whereas using nvflash worked. I suspect there might be a CRC which the boot code checks and using dd doesn’t load this.

Cheers,
Steven

In the above I don’t know what the “p3450” is for (I understand “p3450”, but this should not be part of the flash command). Also, one does not use the “.conf” filename extension (I suppose it is possible the script still uses the right file). So I’ll just suggest this is a valid flash command, while the above probably is not even if it works:
sudo ./flash.sh jetson-nano-qspi mmcblk0p1

Note that content in a partition (other than the rootfs) is signed. Using dd to replace content requires correctly signing the content prior to adding the replacement content. nvflash silently signs.

Thanks for questioning my command. Yes, ‘jetson-nano-qspi.conf’ is invalid as the script appends the .conf extension unconditionally. Luckily for me the script uses the penultimate argument as the target_board entry and ‘p3450.conf’ does exist and is a symlink to ‘p3449-0000+p3448-0000-qspi.conf’ as is ‘jetson-nano-qspi.conf’. This is why my command worked even though it contained an invalid and unused first argument. My corrected command (just tested) is ‘./flash.sh --no-systemimg p3450 mmcblk0p1’.

Is there any way I can sign a file (e.g. U-Boot) so I can use ‘dd’ to update the relevant partition on the SD-card ? I’m guessing I have to prepend a header containing at least a CRC and the length of the data. If I could do this it will facilitate testing as I could just update the SD-card while the system is running and then reboot.

Thanks,
Steven

Yes, there is a simple way to do this. Normally a flash will create a signed version, flash that, and then delete the signed version. One can use the “--no-flash” command to create content and not actually flash. See:
https://devtalk.nvidia.com/default/topic/1052069/jetson-tx2/creating-fuse-blobs-and-signed-encrypted-images-using-hsm/post/5352542/#5352542

The command itself should be something like this:
sudo ./flash.sh --no-flash -u <rsa_priv>.pem -v <sbk>.key jetson-nano mmcblk0p1
(exact details would vary depending on dev kit versus eMMC Nano, but the above is generally close to an actual command)

Note that in that URL the example of the command has “<b>...</b>” for bold format from earlier forum software, and that formatting should be ignored. Also, this command assumes flashing with a private key, but I think you can leave out private key for a board without fuses burned (and dev kits cannot have a fuse to burn), but a default NULL key technically is present even on boards without fuses burned. So you would leave out the “-u <rsa_priv>.pem -v <sbk>.key” part of this on a system without security fuses burned, e.g.:
sudo ./flash.sh --no-flash jetson-nano mmcblk0p1

Hi,

I’ve experimented with the “./flash.sh --no-flash p3450 mmcblk0p1” command and variants. These do create boot.img which is the file I’m interested in. But if I load it in the LNX partition on the SD-card it gets rejected by the boot process (presumably cboot) due to verification failure. When I compare the boot.img file to the original version on SD I notice the SD version contains data starting at offset 0x5a0 and contained within is what appears to be a 16-byte checksum/digest.

I’m guessing that one of the x86 tegra utilities or maybe cboot itself generates this checksum. Is there any way I can do this myself on the Nano ? I have an aarch64 version mkbootimg that creates boot.img identical to that created by the x86 version of mkbootimg. So it looks like the only thing I’m missing is the ability to create the 16-byte checksum. I’m hoping that the details are available rather than an Nvidia secret.

My goal here is to be able to make a new U-Boot on the Nano and then update the LNX partition on the SD-card (again using the Nano only) so I can reboot and try the new U-Boot. My QSPI flash contains the Jetpack 4.4 code which does source code from the SD partitions.

Thanks,
Steven

Do you find a file somewhere named “boot.img.signed”? I’m not positive, but I think the “.signed” is the suffix (someone might want to verify it). Any original boot.img, if normally created for flash, would still be there…but it is the signed version you need (and I have not looked for the specific file in a long time). What you got is likely the unsigned version. Can someone verify the suffix of a signed file being extension “.signed”?

Something you might be interested in is using the “find” command. From the “Linux_for_Tegra/” directory you might try something like this:
sudo find . -maxdepth 4 -iname '*.signed'

The “-maxdepth 4” says only recurse 4 directories deep. The “-iname” is case-insensitive name match (not really needed here but why not?). If you wanted to search for both “.signed” and “.sign”:
sudo find . -maxdepth 4 -iname '*.sign*'

Note: I don’t have any Jetson with a security fuse burned, and I don’t directly install those directories, so what I remember is limited.

No sign (pun intended) of any *.sig* files (the name spec was aster .sig aster but the asterisks seem to get dropped in my reply review here, not sure how the final post will appear).

Looking at some of the scripts I get the idea these are relevant to t18x and t19x systems mainly (although there is a reference to signing t21x rcmboot files). Regardless, the signing seems to be done by closed-source x86 programs so the algorithm isn’t accessible. I did find a hint that the signed header for t21x files is 608 bytes but this doesn’t match what I’m seeing when comparing the before and after flashing versions of the boot.img files.

It looks like the checksum/digest is 16 bytes which makes me think it could be MD5. But tests so far haven’t shown this to be the case.

Cheers,
Steven

I’m not positive, but maybe this is related to the release version, so you might post your release version. I can’t answer, but someone else might be able to comment on whether this is the correct command specifically for a Nano, and which exact file names would be found which are signed:
./flash.sh --no-flash p3450 mmcblk0p1

Normally there is a key, but the key is a “NULL” key. On the other hand, perhaps naming the specific key is needed (I don’t know) via “-u <rsa_priv>.pem -v <sbk>.key” (even if the key is from a NULL value). Would someone be able to comment on why the above command is missing a signed file?

Thanks again for all your help. I’ve learnt a lot along the way here. But the details of this 608-byte Nvidia header are still a mystery to me and perhaps for some security reasons they should remain a mystery to all. So I’ll leave this all on ice until some further details come along.

Cheers,
Steven