Lots of people have used NVMe on the TK1 and TX1, and there were some issues fixed for that. Those issues were generally because NVMe itself was so new and individual drives had quirks. However, TX2 is somewhat different in its hardware, and definitely different in going from the 3.x kernel series to the 4.x kernel series. I have not heard yet about what kind of success or failure people have found on NVMe with the newer TX2…it may be that there are still things needed beyond kernel config, e.g., firmware or user space libs. Anyone know what is needed for an NVMe drive on a tX2 to have the block device show up, rather than just the character device?
@SkyPuppy did you ever have any luck moving rootfs to the nvme drive. Although I can format the drive and set up the files from the TX2, the kernel can’t find it at boot time when I instruct it to look for rootfs on /dev/nvme0n1p1
Are you sure it is the kernel which can’t find it? Or is it the boot loader? Do you have a serial console log? Do you have the NVME kernel options enabled?
CONFIG_BLOCK_DEV_NVME is enabled, and using the nvme as a normal mount point inside linux (when rootfs is on the internal flash) works fine.
I’m not sure if it’s the kernel or the bootloader. The system begins booting but hangs waiting for the rootfs on /dev/nvme0n1p1.
What is the mount point for the drive? Is there an entry in fstab for the mount which you can post? Was the drive formatted from the Jetson?
Jason, I have successfully performed this task on Beaglebone Black and on Raspberry Pi’s. I’ve had ZERO luck with the Jetson TX2 in this effort. I got so frustrated with the thing I just gave up and now have a $600 useless paperweight sitting on the shelf. One runs out of disk space with 32 Gb these days when trying to do anything practical so that’s why I needed the extra disk space. Further, it needs to be ssd or nvme rather than typical hard drive due to wanting to use it in my drone. Sure, I bought the hardware to mount an SATA hard drive and that works fine as a data disk but 1) you can’t boot off it, and 2) you can’t use it in flight.
Your problem sounds like the bootloader is not finding the root partition on the nvme, but since I have not been successful with this $600 paperweight, my advice is worth nothing.
Linux itself understands (or can easily be configured for) many types of storage device. There is an issue if U-Boot doesn’t understand the device since it can’t hand off to a device it doesn’t understand.
A similar (but different) issue is that if the storage device uses a controller which must be reset when it hands off to Linux, then the device also fails as a root partition device.
One workaround would be to boot normally off of eMMC, but then mount the extra drive somewhere like “/usr/local” (or wherever it is needed…in fact your NVME drive could contain several partitions and be mounted on “/home”, “/var”, “/usr/local”, so on).
Linuxdev, yes, that approach seemed the most workable and I did try it. I forget now what else I needed in order to make that successful but at that point my level of frustration was so high that I did not try much longer.
Yes, I ended up giving up as well and mounting the drive in /media on boot and placing most of our files there. That works. It’s more of a hack than a solution, but it’s not worth the effort to debug this damn thing.
Jason, are you using the nvme drive on a pci board? Are you getting the grounding problem? Unless my pci board/nvme were grounded (I used the USB cable) it simply was not seen by the TX2. At all. Very odd. nVidia would not even comment on it, just ask questions about how I found out about it. :(
Yeah, using it as a sub drive works but makes writing programs that access any files a real pain in the patoot, even with relative addressing.
Yeah, I’ve got the nvme drive plugged directly into the PCI-E slot on the board. I haven’t seen the grounding problem that you described; the drive seems to work fine with or without the USB being plugged in.
I’m using a PCI-E 4 - NVME adapter with a Samsung 981A 128G SSD, and it works. The detailed steps are here:
Source code is here: https://github.com/jetsonhacks/rootOnNVMe
It was for Xavier NX but also works for TX2 - of course TX2 still boots from onboard MMC then switch rootfs to NVME SSD which is no ideal.
Also need to pay attention when upgrading OS, as mentioned on README.md on Github, to sync up MMC and SSD contents after such activities.
Hope this helps.