NVMe SSD Drive enumerated but not loading on Jetson AGX Xavier Kit


I am using a PCIE Switch card connected at the x16 PCIe Connector on the developer kit and then connect the 2 Intel 2.5" NVMe SSD drives to the switch downstream port.

I have 2 units of Intel SSD DC P3700 Series (400GB) which work fine and get mounted on the x86 workstation with Ubuntu with the same PCIe Switch card.

However when I boot the Jetson AGX Xavier with the PCIe Switch card with 2 Intel NVMe SSDs, I can see the NVME controllers enumerated with ‘lspci’ command but they do not show up in ‘lsblk’ command.

This is the error info from ‘dmesg’ log:
[ 10.998523] nvme nvme0: pci function 0005:03:00.0
[ 11.001319] nvme nvme1: pci function 0005:05:00.0
[ 11.014831] tegra-cbb 14040000.cv-noc: noc_secure_irq = 98, noc_nonsecure_irq = 97>
[ 11.027598] tegra194-isp5 14800000.isp: initialized
[ 11.028660] nvme nvme0: Minimum device page size 134217728 too large for host (4096)
[ 11.028681] nvme nvme0: Removing after probe failure status: -19
[ 11.033316] nvme nvme1: Minimum device page size 134217728 too large for host (4096)
[ 11.034188] nvme nvme1: Removing after probe failure status: -19

Using the x86 workstation, I formatted the NVMe SSDs in ext4 and FAT filesystem. This was done to set 4KB page size on the SSD by formatting but I get the same issue on Jetson AGX Xavier kit with SSDs formatted in both these filesystem.

Please let me know how to fix and have the 2 extra SSDs loaded in the Jetson AGX Xavier system.


It looks like the NVMe driver reading BAR is returning all Fs.
Can you please give us the output of the following?
a) sudo dmesg | grep -i pci
b) sudo lspci -vvvv
(Depending on the output from (a) and (b) above, I can suggest more experiments)

Hello, Please close the ticket.

The issue was that I had lot of DSPs configured on the PCIe Switch, with each requiring 2M of NP memory allocation.

The Nvidia IOMMU is pre-allocating 22MB of address space.

Once I reduced the no. of DSPs on the PCIe switch my NVMe SSDs were loading and found in the Linux system.