NVMe SSD Drive enumerated but not loading on Jetson AGX Xavier Kit

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hello,

I am using a PCIE Switch card connected at the x16 PCIe Connector on the developer kit and then connect the 2 Intel 2.5" NVMe SSD drives to the switch downstream port.

I have 2 units of Intel SSD DC P3700 Series (400GB) which work fine and get mounted on the x86 workstation with Ubuntu with the same PCIe Switch card.

However when I boot the Jetson AGX Xavier with the PCIe Switch card with 2 Intel NVMe SSDs, I can see the NVME controllers enumerated with ‘lspci’ command but they do not show up in ‘lsblk’ command.

This is the error info from ‘dmesg’ log:
[ 10.998523] nvme nvme0: pci function 0005:03:00.0
[ 11.001319] nvme nvme1: pci function 0005:05:00.0
[ 11.014831] tegra-cbb 14040000.cv-noc: noc_secure_irq = 98, noc_nonsecure_irq = 97>
[ 11.027598] tegra194-isp5 14800000.isp: initialized
[ 11.028660] nvme nvme0: Minimum device page size 134217728 too large for host (4096)
[ 11.028681] nvme nvme0: Removing after probe failure status: -19
[ 11.033316] nvme nvme1: Minimum device page size 134217728 too large for host (4096)
[ 11.034188] nvme nvme1: Removing after probe failure status: -19

Using the x86 workstation, I formatted the NVMe SSDs in ext4 and FAT filesystem. This was done to set 4KB page size on the SSD by formatting but I get the same issue on Jetson AGX Xavier kit with SSDs formatted in both these filesystem.

Please let me know how to fix and have the 2 extra SSDs loaded in the Jetson AGX Xavier system.

Thanks,
Kunal.

2

It looks like the NVMe driver reading BAR is returning all Fs.
Can you please give us the output of the following?
a) sudo dmesg | grep -i pci
b) sudo lspci -vvvv
(Depending on the output from (a) and (b) above, I can suggest more experiments)

3

Hello, Please close the ticket.

The issue was that I had lot of DSPs configured on the PCIe Switch, with each requiring 2M of NP memory allocation.

The Nvidia IOMMU is pre-allocating 22MB of address space.

Once I reduced the no. of DSPs on the PCIe switch my NVMe SSDs were loading and found in the Linux system.

Thanks.