Using NVMe SSD on Jetson Orin Nano

Hi There,
I’m using a Jetson Orin Nano with a 128GB Micro SD card to test what’s possible with various VLM’s given the resource limitations of the nano. I setup the nano following these instructions 🚅 Initial Setup Guide - Jetson Orin Nano - NVIDIA Jetson AI Lab. I then added a NVMe SSD (Crucial P3 1TB M.2 PCIe Gen3 NVMe Internal SSD - Up to 3500MB/s - CT1000P3SSD801 (Acronis Edition) : Amazon.co.uk: Computers & Accessories) to store the models and containers following these instructions 🔖 SSD + Docker - NVIDIA Jetson AI Lab. It seems the docker containers are getting stored on the NVMe but I found that models were still being downloaded to the Nano location ~/jetson-containers/data/models and quickly filling up the SD card. Could someone advice how I can reconfigure it to download new models to the NVMe and also how to move current models to NVMe. I noticed several posts where people use the NVMe to host the OS and boot from it. Is this generally mode advisable in terms of performance?
Thanks in advance to anyone that can help
Aidan

Hi aidan.j.duggan,

Are you using the devkit or custom board for Orin Nano?
What’s the Jetpack version in use?

Are you using SD or NVMe SSD as rootfs?
Please share the result of the following command on your board.

$ cat /etc/nv_tegra_release
$ cat /etc/nv_boot_control.conf
$ df -h

Is this still an issue to support? Any result can be shared?

Hi Kevin,
Apologies for the delay on response (hadn’t spotted it as I thought I’d get an email if there was an update).
I haven’t been able to resolve the issue yet.
In answer to your questions:
I’m using the development kit Orin Nano.
Jetpack version - 36.3.0-20240719161631
Using SD as rootfs

output of cat /etc/nv_tegra_release

R36 (release), REVISION: 3.0, GCID: 36923193, BOARD: generic, EABI: aarch64, DATE: Fri Jul 19 23:24:25 UTC 2024
KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

Output of cat /etc/nv_boot_control.conf

TNSPEC 3767-300-0005-K.2-1-0-jetson-orin-nano-devkit-
COMPATIBLE_SPEC 3767–0005–1–jetson-orin-nano-devkit-
TEGRA_BOOT_STORAGE mmcblk0
TEGRA_CHIPID 0x23
TEGRA_OTA_BOOT_DEVICE /dev/mtdblock0
TEGRA_OTA_GPT_DEVICE /dev/mtdblock0

Output of df -h

Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p1 116G 83G 29G 75% /
tmpfs 3.8G 120K 3.8G 1% /dev/shm
tmpfs 1.5G 29M 1.5G 2% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
/dev/nvme0n1 916G 62G 809G 8% /ssd
tmpfs 763M 128K 762M 1% /run/user/1000

It seems you are using SD as rootfs.

Please refer to Workflow 4: How to flash to device with internal QSPI and an external storage device in README_initrd_flash.txt to flash your NVMe SSD and use it as rootfs.

Thanks for the update Kevin.
I wanted to use the SSD for storage of models and containers and leave the SD card for boot etc, Is it recommended to use the SSD for root file system? Is there a way to migrate the rootfs from the SD to the SSD to avoid having to download all the models and containers I currently have as I presume flashing process will wipe everything?
I haven’t used this initrd flash process before so a little cautious.

Yes, I would suggest using NVMe SSD as rootfs if you need more spaces.
There’s internal QSPI flash in the module and the boot-chains are inside.

You can refer to the steps in Case 12: rootfs A/B disabled, disk encryption disabled of 4. Using golden image in OTA update in Image_based_OTA_Examples.txt to create system raw image.
And use that image to flash NVMe SSD by adding --use-backup-image option in flash command.

1 Like

Thanks again Kevin,
I’ve read “4. Using golden image in OTA update” but I think I’m out of my depth with this process tbh. There are 2 examples with steps shown with “rootfs A/B disabled” but they seem identical and neither refers to disk encryption disabled or are called “Case 12”. In a nutshell will that process take an image of what I currently have on the Nano and then the idea is to flash that onto my SSD using Workflow 4 in n README_initrd_flash.txt? But this presumably would overwrite the contents of my SSD which is not ideal.
I’m leaning towards going back and starting from scratch rather than potentially getting stuck after making a mistake with this process. There’s a pretty good tutorial here https://www.youtube.com/watch?v=q4fGac-nrTI from jetsonhacks to follow

There’re several use cases and the README file shows the steps for them.
They may be similar but just different from few options.
You’ve requested to migrate the rootfs from SD to SSD so that I instructed you for that case of creating golden image.
It’s okay for you to refer the youtube video if it meets your requirement.