Hi,
I have only 16GB available in my board’s eMMC.
My system.img size is greater than 16GB so I wanted to ask if its possible to flash the RootFS into NVME and the kernel and bootloader into the eMMC and the system will boot successfully.
I saw an article explaining how to move the entire RootFS and mount it on another device but they do it from the target after its already flashed once. I wish to split the Rootfs,kernel,bootloader before first system bootup.
p.s
what is the difference between: system.img , system_raw.img and system_sparse.img and how I can control their sizes?
Thanks.
I don’t have direct access to the nvme (its a custom board) so I have to use the initrd “method”.
The problem is that I didn’t find explicit instructions regarding splitting the rootfs from the kernel and bootloader.
Maybe there is a gap in my understanding, but from what I understood the system.img contains the rootfs, kernel and bootloader. So if I flash it to a specific location I flash all the three to the same place.
So what I’m looking for is a command which selects a destination for the kernel and bootloader and a different destination for the rootfs. In addition it has to edit some configurations in order for the kernel / bootloader to recognise the rootfs in its different location.
I read everything and have a couple of questions please:
You wrote:
Blockquote 3. How to do?
We provide steps inside the developer guide link for booting both kernel and mount rootfs from the same drive.
Please be careful that every platform shares different bootloader capability. For example, TX2 uboot does not support usb .
If you just want to mount the file system from external, then you can check this page.
I am sorry that the title of this page is not precise. It is just mounting the rootfs from external. Not booting from anything. https://elinux.org/Jetson/L4T/Boot_From_External_Device
1.1. This page explains how to boot from external device but only in case its connected to your host first. I can’t do that, therefore I have to use the NVME flash with “initrd method”.
The problem is that with this method I flash both rootfs and kernel into the same place (NVME) and I don’t want to do it. I wish to flash the kernel and bootloader into the NOR-flash (nx) and only the rootfs into the NVME.
I didn’t find any specific instructions regarding how to flash just the rootfs into one place and just the kernel into another.
You wrote:
Blockquote 2. You have to know the difference between “boot kernel from external drive” and “mount rootfs from external drive”* Boot initrd/kernel from external drive means the uboot (Nano/TX2 series) or cboot (NX/AGX) will search the kernel and extlinux.conf from your external drive according to the boot order. Once it finds the kernel and extlinux.conf, it will use the info from it to mount the file system. * Mount the file system means kernel will mount the file system according to your “root=” in kernel command line.
For example, the most common usage is root=/dev/mmcblk0p1 which will boot from sdcard (nx/nano) and emmc.
→ Thus, if you just want to increase your disk space, you don’t really need to boot kernel from external drive, you just make sure the file system is mounted from external drive.
→ It means there are lots of variety that you can use. For example, you can boot kernel from usb and mount the file system from NVMe.
So from my understanding , I have to edit the extlinux.conf file with the “root=” arg and point into the NVME that flashed with my rootfs.
2.1. How I now what to write in this argument in order for the kernel to recognise the correct NVME device.
2.2. The extlinux.conf file resides inside the rootfs. the kernel has to read it in order to find the rootfs. So I got this circular condition: find rootfs and read extlinux.conf in order to find rootfs. So I little confused.
Make it more easier to understand…where the extlinux got loaded decides where to load the kernel.
Which means extlinux.conf needs to be same disk as your kernel.
But you could also change that in cboot… this is open source things, so what I said could be changed too.
I would say you could play more with the cboot and check uart log first… it will be more helpful than reading my theory here …
I don’t know what make you feel messy in the uart log. Just focus on the cboot part and you will notice it is just running the scan on each disk to find out the extlinux.conf…
I’ll do it now.
From what you just said, the bootloader is looking for the extlinux.conf , not the kernel. Now it makes sense.
Regardless the cboot (I’ll read its log and play with its settings) -
how can I flash the kernel and the rootfs into different places?
Lets assume I’ll manage to find the setting which direct cboot to load the kernel from specific path (the rootfs/extlinux.conf it searches device by device). I still need to flash the kernel into a seperate device than the one containing the rootfs/extlinux.conf .
for example:
bootloader - NOR flash (thats our only option)
kernel - NOR flash
rootfs - NVME
I read the developer guide.
Is it possible to flash only APP partition to device A , flash only kernel partition to device B and since there is no LINUX variable in extlinux.conf the cboot will load the kernel from kernel partition?
Ok, I apologize. Maybe I mixed between different things.
In previous comments you corrected me by saying that there is no emmc in xavier nx but a NOR flash.
So basically each time I flash the system.img (contains among others the APP and kernel partitions) into mmcblk0p1 I flash it into the NOR flash and cboot loads the kernel partition from there.
I want to put APP partition in NVME (using initrd flash
and put kernel partition in Emmc (using flash.sh)
Then , according to the dev guide , the cboot will search for the APP partition in all devices, will find extlinux.conf and since there is no LINUX var in it , it will load the kernel from the kernel partition.
Your case is possible. But the problem we have now is initrd flash cannot work on your board, as our discussion of another thread.
You just need to modify extlinux.conf on emmc and add root=/dev/__your_nvme_node. When cboot scans out the extlinux.conf from kernel, it will know where to mount the rootfs. And that will be your nvme drive.
If I flash my APP partition into the NVRAM then the extlinux.conf will be in NVRAM and not in emmc. Is that correct?
How I know which value to put in the root=/dev/__ for my NVME? does it have a permanent device name each time I flash the nx module (maybe it can change)