How to replicate a Jetson TK1 SD card

We have two Jetson TK1, one TK1 login process failed and always back to the login screen, we tried many following suggestions in other posts online, nothing we tried seemed to work.

Then we used the SD card from another TK1 and it works well. So we would like to replicate the good SD card, we encountered a lot of permission warnings. None of us in the team are expert with Linux and ubuntu system.

Is there an easy way to replicate the TK1 SD card? Or if there is instructions online we can follow? Appreciate if someone can help to answer the question.

Anything preventing creation of certain temporary files will cause a login loop such as what you mention. There are also other reasons why this might be, e.g., a mismatched OpenGL library after an update could have overwritten one of the NVIDIA-specific files.

As for temporary files imagine a full file system. No room left, no more files.

Alternatively, one cannot create a file if the permission to do so does not exist. Many temp files related to login require some sort of direct or indirect sudo mechanism. Take a look at “/usr/bin/sudo”:

ls -l /usr/bin/sudo

You should find the result is:

-rw<b>s</b>r-xr-x. 1 root root 111784 May 29  2017 sudo*

…emphasis on the “s” of “rwsr”. If the sudo command itself does not have this set, then it isn’t possible to execute sudo.

One common mistake which causes permissions such as this to not be valid is flashing from a host where the file system is not a native Linux file system. “ext4” understands suid, but VFAT and NTFS (which are Windows) have no concept of this and so copy from something like NTFS during flash removes this. In this case you’d have to reflash since it means 100% of the system permissions are suspect.

Note that most live DVD distributions use a Fuse file system which also would not work correctly.

Is your SD card formatted as ext4? Does your card have enough space? A command to list that information would be to cd to where the partition is mounted and run “df -H -T .”. You can also get information about unmounted file system types with “lsblk -f”.

Thank you so much for the reply! We do have sudo permission to the system, and ran sudo command when we troubleshooting the login issue earlier. however none of us here are Ubuntu or Linux expert, we might do something wrong and got error messages other than login issue. Then we switched to another good SD card we was able to login.

So we would like to give up the troubleshooting and just replicate the good SD card from another TK1.

First we will backup the files on the “bad” SD card. Permission alerts popped up when we tried to copy files from SD to a backup disk. We was able to use sudo to give read permissions on some files on SD, then the backup process proceed.

Once this is done, based on your suggestion above, we should format the “bad” SD card to ext4. Our question is
then how to replicate the files from good SD? We don’t want to make any permission changes on the good SD.

There are many ways to do this, some more flexible than others, but it depends on the situation. Can I assume you have the ability to read the SD card when it is not the active root partition? If this is read-only (and not the active system’s running partition) at the time of copy you get far more options.

The permission alerts imply you need to use “sudo” for any kind of copy or backup which reads a file system (ext4 is a file system, the Windows world equivalent is NTFS or VFAT…these file systems have zero chance of understanding Linux permissions). Did your permission alert issue occur on command line, or was it from some particular application? Did you use “sudo”?

What is the partition scheme of the working and non-working SD card? The name of the disk will differ on different systems depending on order added, but let’s say that you run “dmesg --follow” and then insert and SD card on your host PC; the message says this is “/dev/sdb” (just an example, it could be anything). So to see the partition scheme from the host (“mmcblk0” is another example of a disk, and so is “mmcblk1”):

sudo gdisk -l /dev/sdb

Note that a Jetson demands the first partition to be the rootfs (on eMMC Jetsons give this the label “APP”), and the first partition in the above example is “/dev/sdb1” (“mmcblk0p1” is another example of a first partition on a disk). If you wanted to make an exact clone of the entire SD card, bit-for-bit, then you would do this from the host (you wouldn’t mount the partitions, though it wouldn’t hurt if it is read-only or you get lucky and no writes occur…a disk which changes during clone is bad):

sudo dd if=/dev/sd<b>b</b> of=backup_entire_sd.img bs=512

This wouldn’t actually be that useful unless the next card you create from the clone is the same exact spec. More likely the better clone would be just that of the first partition by itself:

sudo dd if=/dev/sdb<b>1</b> of=backup_partition1.img bs=512

And in fact unless you have some special purpose in mind I’d only clone the first partition. It is possible to extract a partition from an entire disk dd clone, but you would have to know the offsets of the partition…it isn’t convenient. If you do take that route, then you are advised to store the “gdisk -l” output along with the clone in case you want to work with it later.

FYI, the block size of 512 (“bs=512”) doesn’t actually change the result, but it does change performance (time required to read and write). Most everything out there is either size 512 or a multiple of 512 times a power of two. 512 is just the most common value, 4096 does get used on some newer disks.

If you had an SD card which was an exact match in specs (same size, brand, etc.) to the good SD, and if that card shows up upon insert as “/dev/sdc” (just another example, it could be many different formats), then this would create a card which is an exact match:

sudo dd if=/where/ever/it/is/backup_entire_sd.img of=/dev/sdc bs=512

If you had instead manually created the first partition on the new SD card as the same exact size (the card itself wouldn’t necessarily have to be a match, e.g., the card as a whole could be larger), then this would imply the partition could be replaced by the clone and it would function perfectly:

sudo dd if=/where/ever/it/is/backup_partition1.img of=/dev/sdc<b>1</b> bs=512

(remember, the “1” is the first partition and the Jetson demands it be “1”…you can restore to any partition and get an exact match, but it won’t be useful as a root partition from a Jetson)

If you just want to copy files, then that also works, but you must take care to copy numeric IDs and not use a command which copies content of a pseudo file. Most files exist on the disk, a pseudo file has content which is the result of a driver in the kernel. A good example is the random number generator, or the pseudo random number generator. Imagine trying to copy the content of “/dev/urandom”…it is infinite content. Whenever you use dd to create a clone of a partition you get what is actually on the disk, and do not copy any pseudo file content (this is exactly what you want). There are various methods of directly copying from a running system which can also avoid this, but you have to be aware of the need to do so. If you need to directly copy files just describe what you need to do and an example can be given.

Btw, if you have a clone of the first partition, and that partition is an ext4 formatted partition, then you can mount the clone via loopback and it’ll look just like the disk. An example of mounting the clone on “/mnt” read-only for examination or copy of individual files would be:

sudo mount -o loop,ro /where/ever/it/is/backup_partition1.img /mnt
# Examine or copy from "/mnt"...
# Leave "/mnt", don't leave a terminal session with the current directory being in "/mnt", e.g., this will work:
cd
# Now umount:
sudo umount /mnt

There are variations on this using “losetup” and mount of the loop device. The “mount -o loop” part just does those steps in one command.

For individual file copy you will find there are commands such as rsync which are aware of the difference between pseudo files and regular files. There are also various options to copy user and group ID numerically…if one system does not have the same user and group IDs (or indeed, different users), then a translation is done and you won’t get a valid copy so far as ownership goes.

I think it is usually a good thing after cloning to use a tool such as gparted, for:

  • resizing the partition if target is bigger than source
  • check filesystem for errors
  • give it a new UUID. Not sure, it may depend on your release, but I imagine udev on Ubuntu14 might have some persistent rules linked to UUID and if the clone card is not same model it might be a problem, or …
    Not sure, but I’d think these safer.

Hi linuxdev,

Thanks for the detailed instruction, I followed your steps and replicated the SD card fromt he good one.

However we encountered a strange thing, the login username and password is no longer accepted, we tried using the combination of old username/password, new username/password of the good SD card, neither one worked out.

What should we do to fix this issue?

Is the login failure over ssh, or is it local? If ssh, then it may be the client end at issue and not host end (if anything changes in the key chain, then login will be considered a man-in-the-middle attack).

If you can log in to any account at all, and then get in to an sudo shell, see if the checksum matches for the following files on the good source SD card versus the bad SD destination card:

sudo -s
sha1sum /etc/gshadow /etc/gshadow- /etc/shadow /etc/shadow- /etc/passwd /etc/passwd-
exit

If sudo itself fails, then that is a clue. Run that same command on both cards (and in fact you could do this from a host PC if you adjust the path to “cd /where/ever/sd/is/mounted/etc; sha1sum ./gshadow ./gshadow- ./shadow ./shadow- ./passwd ./passwd-”, but you wouldn’t be testing “sudo” itself since the host sudo is used instead of the Jetson “sudo”).

Check the permissions of each of these for a match:

sudo -s
ls -l /etc/gshadow /etc/gshadow- /etc/shadow /etc/shadow- /etc/passwd /etc/passwd-
exit

Note that if you added or removed an account, or changed a password, then I would expect the checksum to fail matching.