tegra21x_xusb_firmware failing to load...

Hi,

I am currently battling a strange issue with the USB firmware failing to load when using a Debian based root filesystem on the SATA drive.

I have an unmodified 23.2 nvidia image installed on the emmc, and a headless arm64 debian root filesystem installed on my SATA drive.

I’m provisioning my root file system from the nvidia image userspace with the following commands:

cd /tmp
wget --no-check-certificate http://releases.linaro.org/debian/images/developer-arm64/16.03/linaro-jessie-developer-20160329-71.tar.gz
mkdir hd
mount /dev/sda1 /tmp/hd
tar --strip-components=1 -C hd/ -xaf linaro-jessie-developer-20160329-71.tar.gz
cp -rp /lib/modules/ /tmp/hd/lib/modules/
cp -rp /lib/firmware/ /tmp/hd/lib/firmware/

My extlinux.conf looks like this:

TIMEOUT 30
DEFAULT primary

MENU TITLE p2371-2180 eMMC boot options

LABEL vendor
      MENU LABEL vendor kernel emmc
      LINUX /boot/Image
      FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
      APPEND fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=${lp0_vec} nvdumper_reserved=${nvdumper_reserved} core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 root=/dev/mmcblk0p1 rw rootwait

LABEL development
      MENU LABEL development kernel sata
      LINUX /boot/Image
      FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
      APPEND fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=${lp0_vec} nvdumper_reserved=${nvdumper_reserved} core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 root=/dev/sda1 rootfstype=ext4 rw rootwait

When I boot the TX1 with the SATA drive, the USB firmware fails to load. Below is the kernel message I see during boot.

tegra-xhci tegra-xhci: failed to init firmware from filesystem: tegra21x_xusb_firmware

Here is my entire kernel boot log from the serial port: http://hastebin.com/gudowicepo.vhdl

I checked that the firmware is present in /lib/firmware/ and has the same permissions as the copy on the emmc.

Any idea what might be the issue here?

Thanks in advance.

Tyler

I don’t know if this is the issue, but knowing which partition provides files can sometimes be tricky when mixing SATA and eMMC. This is because the boot loader refers to the partition with eMMC early on using this scheme, but at some point some point during the boot the eMMC partition is ignored for further boot and after that point files are taken from SATA. What I’d suggest is be sure you have an exact copy of “/boot” and “/lib/firmware” on both eMMC and SATA. It probably wouldn’t hurt to also have exact copies of “/lib/modules” on both SATA and eMMC. If that works, then you could remove or rename files one at a time on eMMC or SATA until you find out which one has to be specifically on which partition.

Thanks for the suggestions. I already copy the modules and firmware directly from the vendor supplied Trusty image. I went ahead also copied /boot to the SATA drive for good measure, and it did not help. Previously, I have even gone as far as using the ‘apply_binaries.sh’ script on my SATA drive, and still the same issue persists.

I am starting to wonder if there has been custom modifications to the sample root filesystem provided, that aid in loading the firmware. Maybe something to do with udev?

One thing most people won’t realize is that the flash application itself will modify the sample rootfs slightly…you’ll find modifications of “rootfs/boot” related to firmware additions and boot image, although most of it is irrelevant because only one firmware file is really used and all firmware files from all platforms are put in place. Thus, unpacking a sample rootfs and running apply_binaries.sh gets close to cloning the installed image, but is not an exact match to what actually goes on eMMC after the flash. If the other steps you went through to copy did not succeed, I doubt this is the problem (but there is still some minor chance it is the problem).

Sometimes there are issues related to naming of partitions in places you wouldn’t expect. One example being “/etc/fstab”. The default under L4T I believe leaves this blank, but it is possible under SATA you might need some more explicit setting.

How do the kernels differ? Firmware is related to kernel features…make sure “uname -r” is what you expect and that the module directories exist under whatever “uname -r” is provided.

It’s a bit more invasive then that, I see changes in /lib/udev and /etc/udev as well.

The SATA root filsystem is mounting fine, and is functioning properly. The USB driver is not however, and I’m really just trying to get an ethernet interface working.

They are the exact same kernel, as I am loading it from emmc, just changing the root device. I’m going to add some debug to the firmware initialization in the tegra-xhci driver, so I’ll have a trace of the working and non-working scenarios.

Since CONFIG_TEGRA_XUSB_PLATFORM is set as an integrated feature (at least in R23.2) you wouldn’t be able to check via driver module load/unload. What is your “lsusb -t” output when booted to eMMC, and also again when booted to SATA? There are probably a number of reasons firmware might fail to load, but I’m curious what the kernel thinks it is actually running.

For the usb firmware load to work, it is important to use udev 216 or earlier. This is because they removed the support for userspace firmware loading in later versions. The nvidia kernel is new enough to also support kernelspace firmware loading (which is what the udev devs think you should use instead), however this does not work for the usb firmware because the driver asks for it before the root filesystem is mounted. The userspace firmware loading is asynchronous with a timeout, and because of this it works…