I try to boot from a NVM Express Card, but riunning into “could not find root” error.
What I did,
Recompiled kernel, and add the Block device into kernel. Not as module, direct to the kernel.
Tested Kernel, on internal storage, Block device was detected.
used rsync to copy / from internal to external device (of course created partition and formate before)
Changed the root=/dev/mmcblk1 to root=/dev/nvme0n1p1
reboot, and the kernel boot’s, but didn’t find the /
Any hints ?
I think, the u-boot detects the boot partition on internal storage, and boot’s this kernel.
This kernel, is the same as on the nvm-device. The root for the kernel should be enough !?
Do I need to add the nvm-device to fstab ?
Just to clarify, what was your exact flash command and configuration previous to trying to use nvme0n1p1? Did you actually flash again to use nvme0n1p1, or did you simply edit the extlinux.conf file?
NOTE: Prior flash to mmcblk0p1 versus mmcblk1p1 or any other flash spec can change where extlinux.conf is searched for…thus prior flash matters.
Just to verify L4T version, what is the output of:
head -n 1 /etc/nv_tegra_release
…I’m thinking it will show R23.x if you’ve never flashed.
I use rsync sometimes as well. I found the “-x” option (“–one-file-system”) to not cross file systems tends to be simpler than excluding various folders. This pretty much excludes anything other than the primary ext4…even mounts on the ext4 of type ext4 should be eliminated and give you a good rsync of just the “real” files you’re interested in. However, what you have should work.
In your case where you have never flashed the boot loader will look only at the extlinux.conf on eMMC (though it is a good idea to mirror as you did so it works as a backup). I would think that the boot loader itself would not need to be able to read nvme0n1p1, and handing off to the kernel would succeed as it is. Note that kernel and all files during boot loader stage are from eMMC, not the other disk; upon handing off to the kernel (which was loaded by u-boot from eMMC) the “/boot” directory becomes irrelevant regardless of whether you are using the Linux kernel or u-boot.
However, there is a possibility of one thing that could have been overlooked, depending on feature changes. It looks like starting at L4T R24.2 the the initrd image is used. There was an initrd available prior to that in default installs, but extlinux.conf never named or configured this. R24.2 made this necessary, and the extlinux.conf of R24.2 has an “INITRD” key/value pair naming “/boot/initrd”. Supposing you compiled in a new feature, or perhaps something else changed…if you missed kernel features provided as a module in the initrd (assuming your version is one that needs initrd…and it will if R24.2+ kernel configuration is used), then several features might still cause the kernel to not find the root partition. The trick here is that there is a difference between not finding nvme0, a disk, versus not finding a GPT partition, versus not understanding an ext4 file system. If any of those features are missing it won’t find “/”, and the error message may not explicitly say which in the chain of those requirements caused the failure.
So a checklist is to find your L4T version, make sure that you don’t have a version which used initrd (R24.2+), and to be sure that your kernel supports not just finding the drive, but also understanding GPT partitions and ext4 file system type. If everything else for seeing the hard drive was perfect, but you did not partition with GPT partitions or did not use ext4 formatting, it is possible that these other reasons are why rootfs was not found. Prior to rsync, how did you partition and format the drive?
Of course, the Jetson internal storage I flashed with JetPack and 24.2.
I thought, you mean, I flashed the external NVM-Disk.
Your hint with the initrd disk was great :-)
I just added some stuff to initrd, repack it and push it to both boot folders…
But this was only one part…
I also need to edit the init-script in initrd… :-(
There is only the sd* and the mmcblk0 included…!
So, if anyone want to boot from a other storage then this 2, have to edit the init script…
So, booting into works now, and it looks very fast, and stable.
Next Problem, maybe you have also a good idea…
It looks like, that “any” automounter-app (service) will try to mount the mmcblk0 device. Makes all in all slow.
So, my debug is filled by this message…
Any hint ?
[ 628.238452] [00000008] *pgd=0000000126ff5003, *pmd=0000000000000000
[ 628.245231] Library at 0x7fae716710: 0x7fae70c000 /lib/aarch64-linux-gnu/ld-2.23.so
[ 628.252905] Library at 0x7fae70f1c0: 0x7fae70c000 /lib/aarch64-linux-gnu/ld-2.23.so
[ 628.260805] vdso base = 0x7fae736000
It looks like something is calling for vDSO mapped memory, but mapping has not occurred. You’re getting a kernel message instead of a SIGSEGV because the vDSO isn’t a “real” system call. I do not know how to track the offending source, but perhaps some call related to eMMC uses vDSO as a way to enhance performance, and if the same code were used to access NVME, but not have a proper implementation, then I could see the possibility of this occurring (then again, it may be completely unrelated).
@sandmen, I assume you repacked the initrd, can you post the init script for nvme?
You also mention other changes you did “I just added some stuff to initrd” can you post those if relevant?