Jetson Orin NX 16GB boot/ mount rootfs to sata SSD

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
,
1

I’m trying to load my rootfs by using the guide here to boot from external device:
https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/IN/QuickStart.html#quick-start

I’m able to flash a USB drive to boot up. but sata does not work.
My sata disk is not detected when my jetson orin nx is in recovery mode, it only works after it boots up from the USB drive

I’ve tried cloning the flashed usb to my sata disk in hopes that it will be detected on boot. but nothing works yet.

Any advice to make the sata disk a boot option ?

3

Hi,

how do you connect your SATA drive?
Is the device a DevKit (Xavier NX DevKit/Orin Nano DevKit) or a custom board?

If you are using something like a disk enclosure,
then we have not verified such use case, so it may or may not work.

4

I am using it with a custom board from AAEON that has a sata port on it. So I have my SATADOM drive connected to it

5

Hi,

the SATA port is actually achieved with a PCIe-SATA adapter, but we don’t have the driver packed into UEFI, so it’s expected that it won’t be detected.
Please use other kinds of storage devices for booting your Jetson.

6

HI Dave,

Thanks for he info!
Is there any way I can add the driver to UEFI by building the UEFI manually?

7

Hi,

maybe the vendor does not put the driver in the upstream kernel code, so we don’t have the code.
In that case, I think it’s simply not supported.

8

I’ve confirmed with the vendor, we need a custom jetpack 5.1.2 to make it work.

In the meantime, is it possible to boot from a USB then mount my sata device as rootfs?

9

If you are sure the SATA drive is working after booting into the kernel, then I think YES.
Just modify /boot/extlinux/extlinux.conf in your USB drive to point the rootfs to your SATA drive.

10

I have checked the dmesg output and verified that the PCIe-SATA adapter kernel module is loaded later. I’ve also checked lspci to know which kernel modules is being used for the PCIe-SATA controller.

I saw this on other topics which means I need to rebuild my kernel to include the driver right?
lspci.txt (12.2 KB)

dmesg.txt (88.2 KB)

11

Oh YES.
I think the driver has to be built into the kernel directly, but not as a kernel module.

12

Directly having the driver in the kernel is easier. If using a module, then you need to create a customized initrd with all modules in that initrd. If there is an update that does not know this, then boot will fail when the initrd refreshes without the module.

13

Thanks for the help! I wanna make sure I am doing things right, here are the step I took to rebuild the kernel and flash it from my host machine:

  1. Extract the public_sources.tbz2 file
  2. modify this file
nano Linux_for_Tegra/source/public/kernel/kernel-5.10/arch/arm64/configs/tegra_defconfig
  1. Editing these lines:
CONFIG_SATA_AHCI=y
CONFIG_SATA_AHCI_PLATFORM=y

4 build the kernel following this Kernel Customization — Jetson Linux Developer Guide documentation

After flashing my device can boot but it shows a black screen. I can’t reach the OEM config.

14

Hi,

I think you also need to build the display driver.
Or the device will be accessible with UART but show black screen.

15

Never directly edit the tegra_defconfig (or any .config) if you do not know the dependencies. To be safe you should use something like the make target for “menuconfig” or “nconfig” to do this since the editors understand dependencies (nconfig has a symbol search so it is easier to use). The only value I edit manually is CONFIG_LOCALVERSION, and that is because (A) almost all builds require it, and (B) I know that there is no dependency. I still would not edit this in the “tegra_defconfig”, I only hand edit this in the “.config”.

Note that any instruction which suggests the build target tegra_defconfig, followed by actual build, can have a make target for an editor inserted right after the tegra_defconfig target, and then the edits will show up in whatever you build.

Do understand that whenever you change anything directly in the kernel Image (which an “=y” will do), then you should change the CONFIG_LOCALVERSION to something other than “-tegra” (e.g., “-sata-ahci” if you want a memorable name), and all modules and the kernel Image should be built. You should leave the old kernel and modules in place until you know the new one works (e.g., renaming the new Image after CONFIG_LOCALVERSION and adding a new boot entry; the original can be a fallback).

16

I have tried running menuconfig to configure the settings I want. And got a .config file.

I have CONFIG_LOCALVERSION="sata-on" set in the .config file

I run this:

make -C $SOURCES ARCH=arm64 O=$TEGRA_KERNEL_OUT CROSS_COMPILE=$TOOLCHAIN_PREFIX -j${nproc} Image

but got this:

The source tree is not clean, please run 'make ARCH=arm64 mrproper'

but if I run make ARCH=arm64 mrproper it will remove all my configs.

17

You can back up “$TEGRA_KERNEL_OUT/.config”, then delete everything in “$TEGRA_KERNEL_OUT”, and put the “.config” back in place. Without any other arguments you can clean up the original source from the $SOURCES:
sudo make mrproper

Then, with “$TEGRA_KERNEL_OUT” having only your saved (or newly created) .config, continue the build. You can avoid using the tegra_defconfig target if the configuration of the .config was already acceptable.

Note that “make mrproper” (or “make ARCH=arm64 mrproper”) will clean the source directory. You don’t want any config there at all. When you use the “O=$TEGRA_KERNEL_OUT” it implies keeping the original source pristine, and putting all config in “$TEGRA_KERNEL_OUT”. You want to save the “.config” in “$TEGRA_KERNEL_OUT”, and you want an absolutely clean and unmodified “$SOURCES”. Clean and mrproper will only hurt the “$TEGRA_KERNEL_OUT” if you’ve used the “O=$TEGRA_KERNEL_OUT” during the clean or mrproper. The suggestion here is to make the mrproper without “O=”. Clean up the source, not the destination; for destination clean up everything via deleting, except for the “.config” file.

Incidentally, this is one reason why I suggest the sources should be owned by root, readable by everyone, but not writable by anyone except root. Then one can only modify the “O=” location and source will remain pristine if only regular users compile the kernel.

18

Thanks for the help! But i’m running into other issues now, here is the script I’m running:

#!/bin/bash

# export env variables
export CROSS_COMPILE=$HOME/jetpack/l4t-gcc/bin/aarch64-buildroot-linux-gnu-
export KERNEL_SRC=$HOME/jetpack/Linux_for_Tegra/source/public/kernel/kernel-5.10
export KERNEL_OUT=$HOME/jetpack/kernel_out/images
export KERNEL_MODULES_OUT=$HOME/jetpack/kernel_out/modules

mkdir -p $KERNEL_OUT
mkdir -p $KERNEL_MODULES_OUT

cd $KERNEL_SRC

make -C $KERNEL_SRC ARCH=arm64 O=$KERNEL_OUT menuconfig
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4

I get this error when I’m running it:

GEN     Makefile
  DESCEND  bpf/resolve_btfids
if [ ! -d arch/arm64/boot/dts/nvidia/ ] ; then \
	mkdir arch/arm64/boot/dts/nvidia/ ; \
fi
cp -u   arch/arm64/boot/dts/nvidia/
cp: missing destination file operand after 'arch/arm64/boot/dts/nvidia/'
Try 'cp --help' for more information.

I tried unpacking it freshly the archives without sudo permissions. I don’t know why it does not work yet.

19

You are trying to run menuconfig without an initial setup. menuconfig is designed to edit an otherwise correct configuration, which you do not yet have. There is no telling what the system is trying to actually build. Dependencies for “everything” will be wrong. Try adding this line just before the menuconfig line:
make -C $KERNEL_SRC ARCH=arm64 O=$KERNEL_OUT tegra_defconfig

I also want to caution you to make sure you are configuring CONFIG_LOCALVERSION correctly before build (there are a number of ways to do this).

Next, adjust this line to become two steps (you’re not setting a target so it builds all kinds of targets you don’t want or need):
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4

The suggested two steps:

make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4 Image
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4 modules

If you are building device tree, then this would be a third step, e.g., it might become:

make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4 Image
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4 modules
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j4 dtbs

Unpack the kernel source as root. Make sure nobody else has permission to modify kernel source. Create your alternate directories ($KERNEL_OUT and $KERNEL_MODULES_OUT) as a regular user, and perform the build as that regular user. Except for “make mrproper”, make sure you use “O=/some/where” to specify configuration and temporary output to that location.

20

Thank you for the help! I managed to get it working over the weekend. I didn’t manage to make my sata ssd available as a boot option but I used the usb drive to boot then mount the rootfs to the sata ssd so everything runs from there.

Here are the steps I done:

get the standard kernel configuration from a vanilla jetson linux install.

  1. install the jetson linux normally following the Quick Start guide

  2. Once the vanilla install is up and running get the current kernel config

cp /proc/config.gz .
gunzip config.gz
  1. get the standard kernel configuration to your host pc ( if you’re cross compiling)
cp config $KERNEL_OUT/.config

Cross compile on host pc

  1. get the kernel sources and toolchain from Kernel Customization

  2. export environment variables modify these according to your setup

export CROSS_COMPILE=$HOME/jetpack/l4t-gcc/bin/aarch64-buildroot-linux-gnu-
export KERNEL_SRC=$HOME/jetpack/Linux_for_Tegra/source/public/kernel/kernel-5.10
export KERNEL_OUT=$HOME/jetpack/kernel_out/images
export KERNEL_MODULES_OUT=$HOME/jetpack/kernel_out/modules

mkdir -p $KERNEL_OUT
mkdir -p $KERNEL_MODULES_OUT
  1. Modify what drivers should be build into the kernel Edit the kernel configuration copied earlier by placing the .config in $KERNEL_OUT
cd $KERNEL_SRC
make -C $KERNEL_SRC ARCH=arm64 O=$KERNEL_OUT menuconfig

For my case I wanted to build the driver for my ASMedia Technology Inc. ASM1061 SATA IDE Controller
Looking online, I found I need these 2 configs to be set in the .config file

CONFIG_ATA=y
CONFIG_SATA_AHCI=y

I modified the items in menuconfig to have these 2 options in my .config file

  1. build the kernel
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j{nproc} Image
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j{nproc} modules
make ARCH=arm64 O=$KERNEL_OUT CROSS_COMPILE=$CROSS_COMPILE -j{nproc} dtbs

  1. copy the files to the locations in steps 5-7 here Build the kernel copy them to the sample rootfs and BSP downloaded here

  2. apply the binaries

cd Linux_for_Tegra && sudo ./apply_binaries.sh
  1. create default user
sudo ./tools/l4t_create_default_user.sh -u myuser -p mypassword --accept-license

Flash the sata ssd first

  1. plug in a USB drive then flash it using initrd flash
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device sda1 \
  -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" \
  --showlogs --network usb0 jetson-orin-nano-devkit internal
  1. take out the USB and plug in the sata ssd then flash it
sudo ./tools/kernel_flash/l4t_initrd_flash.sh --external-device sda1 \
  -c tools/kernel_flash/flash_l4t_external.xml -p "-c bootloader/t186ref/cfg/flash_t234_qspi.xml" \
  --showlogs --network usb0 jetson-orin-nano-devkit internal

this time it won’t boot because the UEFI can’t use the ASMedia Technology Inc. ASM1061 SATA IDE Controller
on my board.

  1. plug in the USB then reboot and it works
21

Don’t forget to set up a CONFIG_LOCALVERSION. This is the only part of config.gz which is not mirrored to match the current system.

My advice, if no feature is changed related to something integrated (the “=y” configurations are integrated; adding an “=y” or removing an “=y” changes an integrated feature; modules, or “=m”, do not change an integrated feature), is to keep the default “-tegra” for CONFIG_LOCALVERSION. If an “=y” (integrated) feature changes, then I recommend a modified CONFIG_LOCALVERSION named after whatever changed. For example, if you added a Wi-Fi feature (just as an example), you might change it to “-wifi”.

Whenever you change an “=y” feature you need to change the kernel Image file; if you change a module feature, then you only need to change that module. When the Image changes you need to also change all modules.