Cannot Flash Jetpak 4.3 to TX1

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,
I am having problem flashing my TX1 with the Jetpak 4.3 using the sdkmanager.

Following is the part of the log file related to the error I’m getting:

2020-03-06 21:05:08.060 - info: Automatic trying to flash the target device with Manual Flash option...
2020-03-06 21:05:08.063 - info: [INPUT] "[ `lsusb | grep -c \"0955:\"` -ne 1 ] && echo SDKM_END_CODE_SUCCESS_e448fd4e-7246-455f-bef3-a06f6bd96c82 || echo SDKM_END_CODE_FAILURE_e448fd4e-7246-455f-bef3-a06f6bd96c82_$?\r"
2020-03-06 21:05:08.064 - info: cmd finished success [ `lsusb | grep -c "0955:"` -ne 1 ] && echo SDKM_END_CODE_SUCCESS_e448fd4e-7246-455f-bef3-a06f6bd96c82 || echo SDKM_END_CODE_FAILURE_e448fd4e-7246-455f-bef3-a06f6bd96c82_$?
2020-03-06 21:05:08.064 - info: command finished successfully
2020-03-06 21:05:08.067 - info: [INPUT] "[ `lsusb | grep -c \"0955:\"` -gt 1 ] && echo SDKM_END_CODE_SUCCESS_455efc5c-4a45-4aaf-8079-9c9352fa5772 || echo SDKM_END_CODE_FAILURE_455efc5c-4a45-4aaf-8079-9c9352fa5772_$?\r"
2020-03-06 21:05:08.068 - info: cmd finished success [ `lsusb | grep -c "0955:"` -gt 1 ] && echo SDKM_END_CODE_SUCCESS_455efc5c-4a45-4aaf-8079-9c9352fa5772 || echo SDKM_END_CODE_
2020-03-06 21:05:08.068 - info: command finished successfully
2020-03-06 21:05:08.069 - error: Multiple Nvidia devices are connected. Please connect only one Jetson TX1 device.
2020-03-06 21:05:08.069 - info: Automatic trying to flash the target device with Manual Flash option failed!
2020-03-06 21:05:08.070 - info: Automatic trying to flash the target device with Automatic Flash option...
2020-03-06 21:05:08.074 - info: [INPUT] "[ `lsusb | grep -c \"0955:\"` -ne 1 ] && echo SDKM_END_CODE_SUCCESS_a09a4197-3e13-42bd-aff5-f8ec8e4c324c || echo SDKM_END_CODE_FAILURE_a09a4197-3e13-42bd-aff5-f8ec8e4c324c_$?\r"
2020-03-06 21:06:46.235 - info: pausing setup
2020-03-06 21:06:46.613 - info: PAUSE
2020-03-06 21:06:46.613 - info: event InstallPaused!
2020-03-06 21:06:55.167 - info: cleaning up engine
2020-03-06 21:06:55.167 - info: installManager cleanup
2020-03-06 21:06:55.168 - info: cleanning up the downloader
2020-03-06 21:06:55.180 - info: The overall duration (Download + Install) is 00:02:04

After putting the TX1 to recovery mode I run the following command to check

lsusb | grep "0955:"

Here is the output from this:

Bus 003 Device 028: ID 0955:7721 NVidia Corp.

I could not figure out what the problem is. Please help…
Thanks.

2

The lsusb looked correct for a TX1. Is the host PC a VM?

3

The host is my laptop with Ubuntu 18.04 on it.
I continually get the following error:

2020-03-06 21:05:08.069 - error: Multiple Nvidia devices are connected. Please connect only one Jetson TX1 device.

There isn’t any other USB device connected. I can see that the TX1 is correctly set to recovery mode. But sdkmanager script seems to get something wrong. I tried to find the script itself to try to debug over it. But I could not find it.

4

This seems like an obscure USB error. My gut feeling on this is that a USB device does not really have an opportunity to mess up like this, but that an intervening HUB (including the root HUB built in to the laptop) could do this. There have been several older laptops with unexplainable USB errors (not the same exact error), and after switching to another computer, the errors went away.

As a test, if you see this error, what do you then see with both “lsusb -t” (a tree view) and “lsusb”? I’d like to see how the operating system is dealing with it after there is a mistaken “multiple personality” for the recovery mode Jetson.

Ultimately though, if you have a second Linux computer to test from, then even if it isn’t Ubuntu you could try flashing from command line to see what happens.

5

Here is the output of lsusb:

~/nvidia/nvidia_sdk/Linux_for_Tegra$ lsusb
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 006: ID 5986:0266 Acer, Inc 
Bus 001 Device 005: ID 0a5c:21e6 Broadcom Corp. BCM20702 Bluetooth 4.0 [ThinkPad]
Bus 001 Device 004: ID 147e:2020 Upek TouchChip Fingerprint Coprocessor (WBF advanced mode)
Bus 001 Device 034: ID 0955:7721 NVidia Corp. 
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 007: ID 125f:a88a A-DATA Technology Co., Ltd. 
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

And here is the output of lsusb -t:

sarslan@MSA-X1:~/nvidia/nvidia_sdk/Linux_for_Tegra$ lsusb -t
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 2: Dev 7, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/3p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/3p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M
        |__ Port 2: Dev 34, If 0, Class=Vendor Specific Class, Driver=, 480M
        |__ Port 3: Dev 4, If 0, Class=Vendor Specific Class, Driver=, 12M
        |__ Port 4: Dev 5, If 2, Class=Vendor Specific Class, Driver=btusb, 12M
        |__ Port 4: Dev 5, If 0, Class=Vendor Specific Class, Driver=btusb, 12M
        |__ Port 4: Dev 5, If 3, Class=Application Specific Interface, Driver=, 12M
        |__ Port 4: Dev 5, If 1, Class=Vendor Specific Class, Driver=btusb, 12M
        |__ Port 6: Dev 6, If 1, Class=Video, Driver=uvcvideo, 480M
        |__ Port 6: Dev 6, If 0, Class=Video, Driver=uvcvideo, 480M

As you can see there is only one nvidia device listed.

Anyways, I tried flashing from command line. To do that I had to untar the Jetson-210_Linux_R32.2.3_aarch64.tbz2 file which is downloaded by the sdkmanager. Then I run

sudo ./flash.sh jetson-tx1 mmcblk01p1

The flash script run without any errors. But at the end of the process when I reset the TX1 to reboot, it does not boot any more. Now, I have a TX1 which is not booting.

Any idea?

6

Finally I got it…
I found a file named “sdkml3_jetpack_l4t_43_ga.json” in the download folder. In this file all the commands for flashing are listed as a json file.
Here are the commands I used to flash the system manually:

tar xpf ~/Downloads/nvidia/sdkm_downloads/Jetson-210_Linux_R32.3.1_aarch64.tbz2
cd Linux_for_Tegra/rootfs
sudo tar xpf ~/Downloads/nvidia/sdkm_downloads/Tegra_Linux_Sample-Root-Filesystem_R32.3.1_aarch64.tbz2
cd ..
sudo apt install qemu-user-static
sudo ./apply_binaries.sh
sudo mkdir -p rootfs/opt/nvidia/deb_repos
sudo ./flash.sh jetson-tx1 mmcblk0p1

That is it…
I was able to flash without any issues.

7

I do see a possible issue (useful for anyone else looking at this issue): The Jetson is sharing the bandwidth via a HUB with a lot of different devices. A single port is running 8 devices, and one of those is the recovery mode Jetson. It is true that a lot of the devices, e.g., mouse and keyboard, won’t take a lot of bandwidth, but this can still add latency. The USB Video Class devices (UVC) in particular will be trying to consume a lot of bandwidth, and these will make a dramatic impact.

Note that all of these devices are on “Bus 01”, and these all lead to a single root_hub. Bus 04 leads to external disk storage. Bus 03 has nothing at all on it, and bus 02 has a HUB, but that HUB has nothing connected to it. Is it possible you could move the Jetson to either directly connect as an only device on the bus 02 or 03 ports? If necessary, perhaps to connect to the HUB with nothing else on bus 02?

My feeling is that the errors related to this may just go away, and if this is the case, then you are just reaching the limitations of the USB2 ports (your mass storage is USB3, all others have no ability beyond USB2).

NOTE: Yes, manual flash often works better than via the GUI. In part I suspect because the host is under lower load and uses less RAM on command line.

8

My computer has only two USB ports available. While I was trying to solve the issue I tried all possible configurations. So, I do not think it is related to the ports or other devices connected.

But, it may be an issue about RAM usage of sdkmanager. When I first run the sdkmanager it shows me a popug window telling that “it requres 7.5GB of RAM and there is only 7.5GB available”. Since the message does not make any sense I simply skip it and continue. This isue is probably related to RAM usage of sdkmanager.

But anyways I think the sdkmanager tool requires a lot of improvement. Nvidia should have documentation describing how to do it manually without using sdkmanager. I had to spend a lot time to figure out how to do it.

Thanks anyway.

9

Hi Msarslan,
Thank you so much for the information. It would be great if you can help us to improve.
May I know what version of SDK Manager you were using?
If you do [ lsusb | grep -c "0955:" -gt 1 ] in terminal while device is connected, what’s the return value?
Thanks

10

The SDK Manager is version 1.0.1.5538

The return value is 1.
Here is a screenshot of the output:
External Image

11

The screenshot did not show up correctly. Here is the link for the screenshot:
https://www.dropbox.com/s/d3xleku6131cjgc/Screenshot%20from%202020-03-12%2022-04-14.png?dl=0

12

The error of insufficient RAM would likely be independent of lsusb. The error “it requres 7.5GB of RAM and there is only 7.5GB available” is probably something you can’t get around. I do not know where the particular requirement comes from, but command line might flash even when the GUI does not. You had posted command line earlier though, so I am wondering if the error of insufficient RAM came up only under the GUI, or if it also exists when using command line?

If it really is insufficient RAM, then this may be a case of where swap/virtual memory does not help. However, I do have to wonder, do you have swap enabled which might provide more virtual memory?

13

I do not get any error using command line. The insufficient ram error only show up in sdkmanager GUI.
And I do have swap enabled. I have 8 GB physical RAM and 8GB swap.
I is definitely a sdkmanager GUI issue.

14

Without more RAM you are probably limited to flashing on command line. In the more recent JetPack/SDK Manager you use “apt search” and “apt-get install” commands to find and install the NVIDIA packages, and thus the GUI can probably be ignored (though there would be more of a learning curve).

If you were to flash on command line, and if JetPack/SDKM had not already set up the software, then the procedure would go something like this for a TX1:

  1. Download the latest L4T R32.3.1 or newer “L4T Driver Package” (currently R32.3.1):
    https://developer.nvidia.com/embedded/linux-tegra
  2. Download the “Sample Root Filesystem” as well, same URL.
  3. Unpack the driver package as a regular user. Location is your choice, but it will need to be on a native Linux filesystem type, e.g., ext4. You will also need a lot of spare space there, e.g., over 25GB is a good start for a TX1. Over 35GB is a good start for a TX2 or Xavier.
  4. This produces a “Linux_for_Tegra/” and “Linux_for_Tegra/rootfs/” subdirectory. Within the “Linux_for_Tegra/rootfs/” directory use sudo to unpack the sample rootfs there:
  • cd /where/ever/you/want/to/install/from
  • tar xvfj /where/ever/stored/Tegra186_Linux_R32.3.1_aarch64.tbz2
    (unpacks there)
  • cd /where/ever/it/is/Linux_for_Tegra/rootfs/
  • sudo tar xvfj /where/ever/it/is/Tegra_Linux_Sample-Root-Filesystem_R32.3.11_aarch64.tbz2
    (unpacks into “Linux_for_Tegra/rootfs/” as root)
  • cd ..
    (you are now at “Linux_for_Tegra/”)
  1. Now prepare the NVIDIA drivers:
    sudo ./apply_binaries.sh
  2. If the Jetson is connected via micro-B USB and in recovery mode, then you are ready to flash. Those previous commands do not need to be repeated and the setup is one-time. Example flash:
    sudo ./flash.sh jetson-tx1 mmcblk0p1

Once flash completes it will reboot and you’ll need to create the first boot login setup. On this newer R32.3.1 (which is what JetPack/SDKM 4.3 installs) you should be able to “sudo apt update” and “sudo apt-get upgrade” to make sure software is the latest, and then install the NVIDIA extras, e.g., “apt search CUDA”, and then install the CUDA you are interested in via the “sudo apt-get install <package name>”.

This will avoid needing the GUI (and also allows using a non-Ubuntu Linux host PC).

15

This is exactly how I solved my problem. Thanks for summarizing it.