I am trying to boot the TX2 from a root filesystem inside an NVMe M.2 SSD drive. Initially, with the system fully booted from eMMC, can it can recognise the NVMe device. However, when I configured the NVMe as a root filesystem, it seems that the kernel doesn’t detect NVMe drive at early boot stage, leading to a freeze at [9 seconds uptime]
Waiting for root device /dev/nvme0n1p1...
Hardware
- Jetson TX2 Developer Kit
- Samsung PM961 128GB M.2 NVMe - plugged into the PCIe slot via an M.2 adapter
- Jetpack 3.2
Here are the steps I tried.
First, the latest JetPack 3.2 (L4T R28.2) was installed. Then, I plugged in the SSD. It is recognised properly on a fully booted Ubuntu from the eMMC, so I paritioned it with ext4. Then, I copied the content of the root directory from eMMC to the SSD using rsync. I also tried to chroot into it and it worked okay.
I edited extlinux.conf to add an entry for a root filesystem on the SSD.
LABEL SSD
MENU LABEL eMMC kernel SSD root
LINUX /boot/Image
APPEND ${cbootargs} <b>root=/dev/nvme0n1p1</b> rw rootwait rootfstype=ext4
Since the current version of U-Boot does not support NVMe, the kernel image is still loaded from the eMMC, and only the root filesystem is loaded from the SSD.
I checked the kernel config and found that it has NVMe support built in, not as a module.
$ gunzip < /proc/config.gz | egrep CONFIG_BLK_DEV_NVME
CONFIG_BLK_DEV_NVME=y
# CONFIG_NVMEM is not set
I tried to investigate the kernel log using serial console in two different scenarios: normal boot from eMMC and modified boot with rootfs on SSD.
First, I checked the normal boot process that loads the kernel and rootfs from the eMMC. What I found is that the NVMe driver is loaded after the PCIe driver, and this happens at a later boot stage ~20 seconds uptime. This is much later time after the root filesystem has been mounted.
Second, when the rootfs is configured to be on the SSD, then the kernel fails to detect the NVMe partition. Hence the freeze.
<b>Boot from eMMC Kernel & rootfs</b>
<kernel loaded>
... 9 seconds ....
<eMMC is mounted at />
... 20 seconds ...
<PCIe driver loaded>
<NVMe driver loaded>
<SSD mounted at /mnt>
<b>Boot from eMMC Kernel & <u>SSD</u> rootfs</b>
<kernel loaded>
... 9 seconds ...
Waiting for root device /dev/nvme0n1p1...
... nothing happened ...
I had a look at the other threads before and there seems to be an exact same problem here.
In case of TX1, there seems to be a solution by modifying the initrd script. However, the TX2 doesn’t come with a pre-built initrd, so I am not sure where to start with this (and whether this is a potential solution or not).