Cannot flash Jetson TX2 with JetPack 4.2 with serial console (no keyboard, no monitor)

Hi!

I’m trying to upgrade my Jetson TX2 to the newest JetPack 4.2 using SDK manager.
After several tries, I’ve realized that the availability of a keyboard and HDMI display is essential for that process, as I have to setup initial credentials. And … I do not have them.
As a result, I see a boot log frozen at:

[    7.550336] tegra-i2c c240000.i2c: no acknowledge from address 0x69
[    7.559059] bmi160 1-0069: bmi_i2c_rd ERR: 0x00
[    7.569855] bmi160 1-0069: bmi_init _id_i2c ERR
[    7.634415] random: crng init done
[    7.637823] random: 7 urandom warning(s) missed due to ratelimiting
[    8.893378] vdd-1v8: voltage operation not allowed
[    8.898197] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    8.925619] vdd-1v8: voltage operation not allowed
[    8.930428] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    8.954571] vdd-1v8: voltage operation not allowed
[    8.959516] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    8.968626] vdd-1v8: voltage operation not allowed
[    8.973487] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.085883] vdd-1v8: voltage operation not allowed
[    9.090789] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.099727] vdd-1v8: voltage operation not allowed
[    9.105496] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)
[    9.113726] vdd-1v8: voltage operation not allowed
[    9.118559] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)

Is it possible to setup Jetson TX2 to the newest JetPack 4.2 with only serial console available?

It sounds weird that I have to use GUI for the initial setup of an embedded device …

Many thanks!

Until you mentioned this I hadn’t thought about the issues of serial console when there are no accounts added. I would have to reflash my current TX2 to actually find out (and I’m trying to avoid that for the moment), but basically serial console is designed to log in to the admin user…but if you didn’t add the admin user, I’m not sure what it would do. So let me ask…is there actually a serial console login for you which accepts a login password? I doubt there is, but if exists, then command line is easy for adding new users.

Assuming there is no admin login over serial console until you create it there is still a possibility that you can add this user directly to the files being flashed and have the account installed before the system ever boots. I have not done this directly, so some experimentation may be required (and sadly each experiment could require a new flash).

To start with I’ll just give some details about the admin user (the details will be used below and assumes L4T R32.1 from SDKM4.2/JetPack4.2). The admin user should have numeric ID of 1000, and a group ID of 1000. The user name and group name are the same, and you can change the name if they both match and remain as numeric user/group IDs of 1000. I am going to assume user name “ubuntu”, but you could use “slovak194” instead if you wanted.

Supplemental groups (plus the actual “ubuntu” ID 1000 group) which will the admin user will be part of:

adm:x:4:syslog,ubuntu
cdrom:x:24:ubuntu
sudo:x:27:ubuntu
audio:x:29:pulse,ubuntu
dip:x:30:ubuntu
video:x:44:ubuntu,gdm
plugdev:x:46:ubuntu
lpadmin:x:118:ubuntu
gdm:x:124:ubuntu
ubuntu:x:1000:
sambashare:x:129:ubuntu

Traditionally the two files which anyone can see in which a user is listed are “/etc/passwd” and “/etc/group”. These exist in the sample rootfs in “Linux_for_Tegra/rootfs/etc/”, and root can edit these before flash…edits will propagate to the flashed TX2. Extra files exist for each of these two files which are readable only by root, and these contain actual hashed passwords. You will need sudo to edit any of these, but you can ignore warnings of needing special editors to edit these files since they are not part of your actual running host.

Some of the related files which will need edit (beyond “rootfs/etc/passwd” and “rootfs/etc/group”):

passwd-
group-
gshadow
gshadow-
shadow
shadow-

First a word on passwords before trying to edit anything. Passwords are not stored in any reversible format. Passwords are put through a one-way hash using a seed. What goes into the files which the general public cannot see are hashed passwords, and so if you want to create a password to put in a file, then what you have to do is create the hash and then put the hash in the correct place. I am going to make the assumption that you are adding the password “ubuntu”, but you could alter this if you desire.

Here is an interesting URL on password hashing:
https://ma.ttias.be/how-to-generate-a-passwd-password-hash-via-the-command-line-on-linux/

Note that in that URL it starts with OpenSSL for hashing, but this mentions it uses only md5. The actual password system now uses sha256, and a script is given using python to generate a hash via this method. To generate a hash via this method for the password “ubuntu”, use this:

python -c "import crypt; print crypt.crypt('ubuntu')"

Each time you run the command the output will differ…this is because a different seed is randomly used each time. Run the command several times and see how it changes each time. Any of these should do the job. Because of this two different users could use the same password and the password file would never indicate the two users are using the same password (so long as the random salt differs…when the two do not differ it is a failure known as a hash collision). I am going to use the following as my example, but you can use whatever you get:

python -c "import crypt; print crypt.crypt('ubuntu')"
<b>$6$6iLvFCj51KgtA83x$gCgmRE7tDV0Y286SinZ9mV0q5lVJJGrFvAEpgRnObonSSvXsF8gEdsnfshUr8G3H2q3I1HclxiTM1XfTEBOlR1</b>

Just as an example, if you look at your Ubuntu host, examine your shadow file ():

sudo cat /etc/shadow

Notice that there are many system accounts with no login, and those are short lines. The longer lines exist only for users capable of login through password. You could literally copy the line from your existing host for the user with ID 1000 and it should work, but I’ll illustrate manual generation because you don’t want to pass your host information on to any new machine.

The fields are separated with a “:”. The first field is the user name, the second is the hashed password. The accounts with a “" for a password are locked. I’ll start with this and create an empty template, then fill in the "” to unlock it. However, the account does not yet exist, so I’ll back up and first create the account in “rootfs/etc/passwd” and “rootfs/etc/group” (you could save backups of any file you edit prior to starting…beware that versions of this with the wrong permissions will be denied).

  1. In rootfs/etc/passwd, at the bottom of the file (because UID 1000 is the highest number...order is via numeric UID by coincidence because this is the order the accounts were originally added):
    ubuntu:x:1000:1000:,,,:/home/ubuntu:/bin/bash
    
  2. In rootfs/etc/group, probably at the end, once again order is really by the order which groups were added:
    ubuntu:x:1000:
    
  3. Now we add a password template without the actual password in "rootfs/etc/shadow" at the end of the file (the "*" will later be edited to become the generated hash...to read about the fields see "man -S 5 shadow"):
    ubuntu:*:18039:0:99999:7:::
    
    • Note that the 18039 is somewhat arbitrary, it is the number of days since Jan. 1, 1970 and is used in password aging. See this URL for a calculator of days between dates...you can use it to set the starting date if you wish, but 18002 is already done (set the start date for Jan. 1, 1970): https://www.timeanddate.com/date/duration.html Between Jan. 1, 1970, and May 23, 2019 is 18039 days.
    • Minimum password age is "0" because there is no restriction on changing this password.
    • Maximum age is chosen arbitrarily long: 99999 days (around 273 years). Login basically won't force a password update. Who knows...maybe if the Jetson is still running in 273 years in some government institution or a vehicle there will be chaos as everything shuts down :P
    • Password required change warning is 7 days. You can worry about that in 273 years less 7 days.
    • No expiration of account will occur, and the reserved end field is always empty.
    • The above now only needs the actual password to be valid. You can essentially start with this for any manually added account
    • Now add the hashed password in place of "*" to complete this file and save:
      ubuntu:$6$6iLvFCj51KgtA83x$gCgmRE7tDV0Y286SinZ9mV0q5lVJJGrFvAEpgRnObonSSvXsF8gEdsnfshUr8G3H2q3I1HclxiTM1XfTEBOlR1:18039:0:99999:7:::
      

      (this has just become user “ubuntu” with password “ubuntu”, although more work is required)

  4. Now the password hidden content backup "shadow-" file needs to be edited ("rootfs/etc/shadow-"). The backup is just the template without the password hash:
    ubuntu:*:18039:0:99999:7:::
    
  5. Finally, add the user template to the "passwd-" backup ("rootfs/etc/passwd-"):
    ubuntu:x:1000:1000::/home/ubuntu:/bin/bash
    

    the

  6. The base user is added. Now we need to add the group, followed by making the user also a member of the administrative supplemental groups.
    • Edit "rootfs/etc/group". Add the main group which is also the name of the user. This first user will always have UID and GUID of 1000, and is added in the order groups are added (so probably at the end of the file and it won't matter if it is in the middle):
      ubuntu:x:1000:
      
    • Now add the user as a supplemental user of the other groups...this is the summary copied from the listing previously shown (more than one person can be given supplemental groups...a comma delimits between them and order does not matter...if you don't have a group, e.g., if you don't have samba, then just leave it blank and don't add it):
      adm:x:4:syslog<b>,ubuntu</b>
      cdrom:x:24:<b>ubuntu</b>
      sudo:x:27:<b>ubuntu</b>
      audio:x:29:pulse<b>,ubuntu</b>
      dip:x:30:<b>ubuntu</b>
      video:x:44:<b>ubuntu,</b>gdm
      plugdev:x:46:<b>ubuntu</b>
      lpadmin:x:118:<b>ubuntu</b>
      gdm:x:124:<b>ubuntu</b>
      <b>ubuntu:x:1000:</b>
      sambashare:x:129:<b>ubuntu</b>
      
    • We now edit the group shadow file ("rootfs/etc/gshadow"). Same idea here the example edits showing the groups which user "ubuntu" is related to:
      adm:x:4:syslog,ubuntu
      cdrom:x:24:ubuntu
      sudo:x:27:ubuntu
      audio:x:29:pulse,ubuntu
      dip:x:30:ubuntu
      video:x:44:ubuntu,gdm
      plugdev:x:46:ubuntu
      lpadmin:x:118:ubuntu
      gdm:x:124:ubuntu
      ubuntu:x:1000:
      sambashare:x:129:ubuntu
      root@x2:/etc# less gshadow
      root@x2:/etc# egrep ubuntu gshadow
      adm:*::syslog,ubuntu
      cdrom:*::ubuntu
      sudo:*::ubuntu
      audio:*::pulse,ubuntu
      dip:*::ubuntu
      video:*::ubuntu,gdm
      plugdev:*::ubuntu
      lpadmin:!::ubuntu
      gdm:!::ubuntu
      ubuntu:!::
      sambashare:!::ubuntu
      

      Note that in the above the exclamation mark “!” is because the user isn’t required to use a password to access that group permission. Some of the system groups would require a password. “sudo” is the best example, and had the user been a member of “sudo” without needing a password, then the “*” would have instead been “!”…a huge security hole.

    • Next is the backup "gshadow-" file ("rootfs/etc/gshadow-"):
      adm:*::syslog,ubuntu
      cdrom:*::ubuntu
      sudo:*::ubuntu
      audio:*::pulse,ubuntu
      dip:*::ubuntu
      video:*::ubuntu
      plugdev:*::ubuntu
      lpadmin:!::ubuntu
      gdm:!::ubuntu
      ubuntu:!::
      sambashare:!::ubuntu
      

You are ready for flash once you’ve backed this up. Normally, when running on command line to flash, the sample rootfs is not reinstalled and thus you wouldn’t need a backup. However, it is possible that if you flash via JetPack or SDK Manager, then the rootfs content could get overwritten. Add to this that I am doing this from memory and not actually testing, and so if you want to repeat the experiment, then you want a backup of both the original and edit versions of these files in the “rootfs/etc/” subdirectory (if the permissions of these files are altered and not preserved, then it is possible the login and system will fail or security will be compromised…editing should not change this, but an actual file copy might):

passwd
passwd-

shadow
shadow-

group
group-

gshadow
gshadow-

Now flash and the generated image should have this built in. User “ubuntu” will preexist with password “ubuntu”. There may be a “first boot” script, but if such a script triggers to create a user, then you can ignore it. The same edits could be made to a loopback mounted “bootloader/system.img.raw”, and then flashed “reusing” the old image (you’d need to rename it “system.img” and use “-r” option) to save a small amount of time, but that is a different thread.

Example flash on command line with micro-B USB cable and TX2 in recovery mode:

sudo ./flash.sh jetson-tx2 mmcblk0p1

The booted TX2 should “just work” with account “ubuntu” and password “ubuntu”. More accounts could be added exactly the same way.

Note that you can then use JetPack or SDK Manager, and simply uncheck flash and install packages. If these edits are in place when JetPack flashes, then the same result of having the preinstalled account/password will also be true…the risk is that JetPack or SDK Manager might update the rootfs and overwrite your edits. Command line is safer because you know nothing will overwrite your edits.

On a related note I suggest that if it isn’t already done, then you edit “rootfs/etc/apt/apt.conf.d/20auto-upgrades”. My suggestion is to disable unattended automatic upgrades. The reason is that when JetPack goes to add packages it will be blocked until regular updates are completed…and on a new install this can take a very long time. During that time attempts to manually add packages will be blocked. Disabling the unattended automatic upgrades could be reversed once you are satisfied everything is in place. The edit would change the file to become:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade <b>"0"</b>;

Note that package list updates are very fast and don’t actually update the packages, although it will trigger notification that packages need update.

This entire setup could easily be scripted and added to a future SDK Manager, and I would recommend doing so. Anyone could also easily automate this in a script for a production line.

Dear, linuxdev!

Many thanks for such an exhaustive explanation!
I’ll try to reproduce that soon, but for other users who will come to this thread it would be nice to have a simple patch copied from already set up Jetson TX2 with default nvidia/nvidia credentials. With vanilla files:

passwd
passwd-

shadow
shadow-

group
group-

gshadow
gshadow-

I’ll try to create it soon, but if you have TX2 set up and running - please, share those files here.

Many thanks again!

BR,
Alex

I actually save a template of “overlay” files such as those you mention above. Any time I install a new sample rootfs I just recursively copy my overlay onto the rootfs (networking, shell aliases, ssh keys, and so on are already working on first boot).

The only reason this is an issue with password files is that in the transition from the previous release to this release there was a major version change from Ubuntu 16.04 to 18.04, and as a result accounts changed. In the case of a major version change one has to edit those files, but if you have a copy of what was there before, then it is simple copy and paste and all of that previous work (other than pasting into files) can be skipped. If the Ubuntu major version release is constant, then the files can be directly copied over without editing specific lines.

@linuxdev,

It looks that it is not enough to just modify the aforementioned files. After flashing I’ve come to the same point with:

[    9.118559] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)

But the solution from here:
https://devtalk.nvidia.com/default/topic/1048835/jetson-tx2/tx2-l4t-32-1-automatic-booting/post/5326586/#5326586
combined with your proposal is working.

Many thanks!

In terms of password files you would also need to create the home directory, but a user without a home directory can still log in (and so for example that user can still use sudo). The “/etc/skel/” is used as a template of what to put in a home directory when using a create user command which also creates the home directory, and so you could easily just copy this to “/home”. From a host side it would be a recursive copy of the content of the skeleton directory, including files starting with “.” in the name into the user’s name of “/home”, e.g., “Linux_for_Tegra/home/ubuntu”, and then recursively changing ownership to user ID and group ID 1000 (this is the ID we put in the password file, but the ID of that user on the local system might have a different name).

“chown” works with both a name (which is looked up in the host “/etc/passwd” and “/etc/group”, not the “Linux_for_Tegra/rootfs/etc/…” files) or with a number. After creating/copying skel to “Linux_for_Tegra/rootfs/home/ubuntu/” you could:

sudo chown -R 1000.1000 /where/ever/it/is/Linux_for_Tegra/rootfs/home/ubuntu

Part of my overlay files is a custom “~/.bash_aliases” and sometimes “~/.bashrc”. If I use a web browser there, then I include the bookmarks and other browser setup. My ssh keys are set up in “rootfs/home/ubuntu/.ssh/” prior to ever flashing.

FYI, this error is not directly related to the home directory or group/passwd file setup:

[    9.118559] sdhci-tegra 3440000.sdhci: could not set regulator OCR (-1)

The other solution you found is in many ways a better solution since it actually performs the equivalent steps programmatically. The programs used to do this in the other URL/thread won’t miss details if something changes. My example also did not include user “nvidia”. I use my original approach because all I need to do is copy files which I already have, but the two can easily be combined…direct file edits for things you know you need, and then the other step for anything possibly missed.

The best reason for mentioning all of these ways of doing the same thing is that I don’t think users always realize how much power they have to customize and get what they want out of the box. People working on commercial products and in need of flashing many Jetsons at once should understand what all can be done in the “rootfs/” prior to a flash. Limitations, like accidentally giving everyone the same password should also be understood.