Re-Flash TX1 to restore sudo permissions (No sudo, No grub, No microUSB detection)

I made a big mistake on my first TX1, and I’m wondering whether I can restore it with a re-flash. A misuse of chmod led to the loss of sudo permissions, inaccessibility to recovery mode (grub), and no graphical interface outside of login (loops). Additionally, my host pc is also not detecting Nvidia Corp via the micro USB cable. The board is alive and the data can be seen via SSH, but I can’t really do much without sudo and recovery mode access. Any suggestions as to how to re-flash the board to try to restore things?

You should be able to flash and get a fully functioning system. As to recovery mode, simply holding down the recovery button and tapping power on (or power reset if power is already on) should put the Jetson into device mode, which in turn will make it show up via lsusb (requires the micro-B USB connector be connected). Specifically, you should see some output from:

lsusb -d 0955:7721

(0955 is nVidia, 7721 is the JTX1 in recovery mode)

I’d just use R23.2 of L4T (there will be a new release soon, but R23.2 is a very well known baseline). Some people would use JetPack 2.1 for this, I just use the driver package plus sample rootfs. These can be found here:
https://developer.nvidia.com/embedded/linux-tegra

Thanks linuxdev. When I try to place into recovery, I end up back on the login screen. Attempting to login cycles me back to the login screen. I can see that the dev board detects Nvidia Corp (itself) when I check via SSH, but have not direct access to the recovery mode and my computer (host) doesn’t detect the Nvidia Corp USB (perhaps screwed up device manager?)… I never set-up root access and don’t have sudo permissions. When I attempted to test whether I could download/modify anything via SSH I am told that the disk is full. It would be great to preserve the first board, but I’m concerned that it’s beyond repair. Should I be attempting to do any of this via serial access or is micro USB the best way?

Okay, update. After a couple of days of no detection, I can finally see NVIDIA APX with lsusb. I pressed ESC like 30 times on reset, which led to no screen coming up, which I could not accomplish when following the instructions. I can’t be certain whether this was the solution to getting the system into some type of boot, however, I started to think whether the micro USB was damaged, but I guess it wasn’t a hardware issue. Now the challenge will be to see whether I can flash the device when the disk is full and I don’t have sudo permissions.

Anything which resets power while recovery button is held down should put the Jetson in recovery mode without failure. Anything else is probably hardware failure. I know of no way to remotely trigger recovery mode. Even if the micro-USB is damaged, recovery mode should work with no login screen popping up.

FYI, Ubuntu does not set up a root account, it depends on “sudo”.

You can clone the root partition if you wish to save a copy:
http://elinux.org/Jetson/Cloning

If you have something valuable there you could clone, edit the loopback mounted clone, and re-flash re-using the cloned partition.

Hello linuxdev,

I’m on the same boat as LMun85 in which I need to re-flash the TX1 to L4T R23.2. I have a 32-bit TX1 and I was trying to follow this link (https://github.com/dusty-nv/jetson-inference). As mentioned in the recommended system requirements, the Jetson TX1 Dev Kit should have a JetPack 2.3 or newer (Ubuntu 16.04 aarch64). I know it’s just recommended and not the only way to run the inference, but I think I made a mistake of updating my Ubuntu 14.04 LTS to 16.04 LTS as I could not login to the desktop and could not use the USB 2.0 port. After trying to figure out how to resolve the common issue of infinite login loop on Ubuntu, I gave up and thought that the TX1 that I have might be only capable of the originally deployed OS (of course I might be wrong).

Now I have some questions:

  1. What’s the difference between flashing using JetPack vs. L4T?

  2. Is there any specific (or suggested) way to re-flash my TX1 in order to run the jetson inference from the link above?

  3. Lastly, does the 32-bit TX1 I have is capable of running the Jetson Inference in the link above despite it being not in the recommended system requirement for deployment?

Thanks,

Paul

First, login might fail if you didn’t use sudo at the right moment during flash. JetPack handles this for you and asks for a password at the right time (don’t run JetPack sudo, it will ask). If flashing on command line you need to use sudo while unpacking sample rootfs, while running apply_binaries.sh, and for the actual flash.sh command. On command line be sure to use this for maximum use of disk:

sudo ./flash.sh -S 14580MiB jetson-tx1 mmcblk0p1

JetPack is a front end to flashing and package utilities. You can use JetPack for the flash, or you can flash manually. Extra packages may not all be available without using JetPack to install them (although more are available than it looks like if you know how to query the JetPack software in order to manually download various packages).

L4T is just another name for the operating system when Ubuntu has direct hardware acceleration drivers added to it. The sample rootfs is pure Ubuntu, apply_binaries.sh loads it with NVIDIA files (e.g., so you can access GPU). You can detect which L4T version via “head -n 1 /etc/nv_tegra_release”. You can detect if any of NVIDIA’s special files are corrupt via “sha1sum -c /etc/nv_tegra_release”. This won’t tell you about permissions, but if sudo doesn’t work, then likely the files were not unpacked with sudo or were unpacked to a file system type not supporting Linux permissions (e.g., NTFS and VFAT guarantees sudo will fail).

The TX1 is 64-bit. It is aarch64/arm64/ARMv8-a architecture. The thing about 32-bit is it is normally known as ARMv7. However, the aarch64 hardware has a compatibility mode which can execute 32-bit ARMv7. This mode is low performance. The kernel space (where the drivers run) is 64-bit on all TX1 L4T releases. It is the user space (where user programs live) which can optionally be 32-bit. I think R24.1 was the first release making 64-bit user space available, although it also had 32-bit still available for the first couple of weeks. It just depended on which sample rootfs you picked. R23.2 and earlier were only available with user space 32-bit (kernel space 64-bit).

If you want to flash with JetPack your host must be Ubuntu. Mostly Ubuntu 14.04 PC is used since this is the actual requirement…but you can have success with an Ubuntu 16.04 PC host as well (some of the host side package installs won’t work…JetPack package utilities can also set up cross-platform development tools on the PC).

If you wish to flash with the simplest of host requirements, then you need command line. If you have an Ubuntu host and want to use it for package management, then this is probably the better way to go. It is possible to check which procedures you want in JetPack, so if you’ve previously flashed the Jetson and do not want to flash again, but instead want to check out packages, this can be done simply by checking what you want to do. Flash software (driver package plus sample rootfs) is good only for flashing or cloning. JetPack is good other tasks at any time you want.

If you flash with JetPack you are installing L4T on the Jetson. JetPack itself never installs to a Jetson, it only installs on the host PC. In all cases you would be installing Ubuntu with NVIDIA’s customized direct hardware access files.

Typically if you want some of the sample programs and support frameworks for developing, then this is most easily installed via JetPack.

Here’s a URL to historic versions of JetPack (and indirectly L4T):
https://developer.nvidia.com/embedded/jetpack-archive

Here’s a URL to historic L4T releases (driver package, sample rootfs, and some related files):
https://developer.nvidia.com/embedded/linux-tegra-archive

Hello linuxdev,

First, I want to thank you for a very detailed response to my questions.

I’m still trying to figure out how to re-flash the TX1. I’m wondering if installing the latest L4T R28.1 in my TX1 using Jetpack 3.1 would be possible given that I have “initially” a 32-bit Jetson TX1.

I checked the version of my L4T, which is R23.2 using “head -n 1 /etc/nv_tegra_release”, and checked what type of architecture I have after mistakenly upgrading to Ubuntu 16.04 LTS, which is now showing that I have a Linux aarch64, kernel version 3.10.67 (using “uname -a”). Before upgrading to Ubuntu 16.04 LTS, I remembered checking if the TX1 has either a 32-bit and 64-bit architecture. It showed 32-bit when I checked the system config through the Unity GUI, thus I mentioned “initially”.

This may sound elementary, but I’m not sure if it’s possible to upgrade the architecture of the TX1 that I have from 32-bit to 64-bit which will make it possible for me to flash the TX1 to the latest L4T version using the latest release of Jetpack.

Please advise.

Thank you very much,

Paul

The TX1 is not 32-bit, it is 64-bit in hardware with a 32-bit compatibility mode. The Linux version installed determines if it runs in 64-bit mode or 32-bit mode. Flash implies any previous software will be erased, so there is no limitation at all unless you are only copying over a part of one install to another…ordinary flash installs the entire setup from scratch.

The only software you will run into which might care about 32-bit is “user space” software. Sometimes older 32-bit ARMv7 is ported in stages to the 64-bit processors, in which case it may use 32-bit user space. This is a fairly narrow set of software which might run on a TX1 and actually require 32-bit compatibility mode…most of what is out there works with 64-bit native mode. I think some of the ROS stuff might have required 32-bit, but I have not personally found anything I use requiring this.

Just as an analogy, consider desktop PCs. These days they are all 64-bit. Even so, PCs used to be 32-bit. The desktop CPUs these days support 32-bit as a compatibility mode. 64-bit is native mode, 32-bit is compatibility mode. If you previously had an old program running in 32-bit it might have needed compatibility libraries for it to run. On the other hand, if you’ve installed a 64-bit version of that program, then this will in no way break anything. Compatibility mode is a migration path from older version to newer version when the newer version is not yet available.

So flash never changes hardware architecture. A 32-bit TK1 can never run 64-bit, it does not have the hardware…but a 64-bit TX1 or TX2 can run either 32-bit or 64-bit because it has hardware dedicated to native 64-bit, along with a lower performance 32-bit compatibility mode. The CPU simply switches back and forth between modes depending on which software is running at the time.

My advise is that unless you know you have some software which can’t work in 64-bit mode to simply dump 32-bit mode and flash to R28.1 (possibly using JetPack 3.1 if you want to use that tool to do the flash).

Thank you, linuxdev. I am still trying to flash my TX1 using an Ubuntu 14.04.5 on a VM. I installed Jetpack 3.1 on the VM and tried to flash the TX1, but it freezes in the middle of the process, particularly at “Boot Rom Communication” (similar to this post https://devtalk.nvidia.com/default/topic/981259/jetson-tx1/reflashing-tx1-is-stuck-at-boot-rom-communication-tegrarcm-chip-0x21-rcm-rcm_list_signed-xml/). I’ve tried all the other possible solution from other threads that is related to the flash freezing, but still no success. I am thinking that I could just use an Ubuntu host directly as the problems may be caused by the USB settings of the VM like you mentioned in the post.

Before doing so, I’m trying to see if switching from wireless internet connection on the TX1 to Ethernet may be a solution (although I doubt it will be) in flashing the TX1 through a VM. I tried booting up the TX1 again after the failed flash attempts to check if the TX1 is successfully connected to the internet. Unfortunately, nothing is showing up on the screen. I’m thinking that because the flash failed to finish, it may have messed up the firmware on the TX1. Am I correct to assume that and the only way to recover the TX1 is to successfully flash it?

Thanks,

Paul

VMs are a known problem for USB and are technically not supported. I would agree that an incomplete flash would cause boot failure. If your VM sits on top of a non-Linux file system type this will mean even if the flash finishes the install will not work anyway. If you really must use a VM and can use ext4 as the file system, then you will have to look closer at customizing the USB passthrough (e.g., forcing the port to not try anything but USB2, marking the recovery mode Jetson as belonging specifically to the VM, and adding buffer to the USB pipe…none of which I know how to do).

Update:

I was able to successfully flash the TX1 by switching to Ethernet connection and changing the VM’s USB setting to 3.0 instead of 2.0. Now, I’m stuck in the “Determining the IP address of target…” part of the Post Installation. My TX1 and VM is possibly connected to a switch (company network, so I’m not sure). I know the VM is connected to the network while the TX1 is only connected to the internet and I was able to ping the IP address of the TX1. Is this a firewall issue?

The default for wired ethernet is DHCP, so it will ask a router for an address. If your company network provides DHCP, then it will need to reply to the Jetson…whether your company sets this up and requires you to ask I don’t know…but if it simply replies to any DHCP request it should work

If your PC is being set up as a router, then this would be the same story…only your network would need to be routed to the VM so that the VM can take over ethernet as a router (a standard PC won’t have this issue…a host system might need to allow a VM to act as a router and pass through DHCP requests).

Using a name instead of a dotted decimal format IP address is a DNS setup issue, but since you have an IP you can ping I would say the router is set up correctly. If you can ping then there is no firewall issue for the ICMP protocol…it is possible that your host needs firewall setup before it can do forwarding, but this would not be needed if the router is somewhere else on the network. Can you ssh in and log in to ubuntu account (pass “ubuntu”) or the nvidia account (pass “nvidia”)? FYI, if someone else can reach your Jetson over the network you will need to immediately change the password since the world knows the defaults. Ssh could be a firewall issue, but the Jetson itself is not set to stop ssh connections unless you set that up.

JetPack not determining the IP address is another issue if the Jetson is otherwise reachable by ping from the host and ssh from the host. If you can ssh in (or use local login) to the Jetson post the output from “ifconfig”.

Hello linuxdev,

I was able to get past that issue by closing the Post Installation window and re-running the JetPack installer.I unchecked or changed the “Flash OS” to no action since I was able to successfully flash my TX1 and only care about the installation of the other packages such as CUDA. I was able to enter the IP address and the username and password of the default “nvidia” account of the TX1 and everything worked out fine! (helpful link: https://devtalk.nvidia.com/default/topic/898683/jetson-tk1/jetpack-2-0-on-jetson-tk1-installation-doesn-t-finish/1)

Thank you so much for all your help linuxdev! I greatly appreciate it! I wish I could buy you a beer (or a virtual beer)! Hopefully I don’t run into any more problems. :D

CHEERS!

Paul

:)