Mirco SD Card Usage - Memory

Hi all,

I am currently hitting a wall when trying to install openCV that I don’t have enough space left on the Xavier, and I am a little too novice of a user to try and do what JetsonHacks did with the SSD (Install NVMe SSD on NVIDIA Jetson AGX Developer Kit - JetsonHacks) but I did see that the xavier has a slot for a micro SD card and so I was wondering if it was possible to just put one in and use as extra storage for the device?

If so, is there any particular set-up/mounting I would need to do?

Thanks!

This would function just like an SD card on a desktop PC. There is probably some automount which would try to mount somewhere in “/media” when you log in to the GUI, but you could also pick a mount point manually and put it in “/etc/fstab”. Manual mount could go by UUID so that other SD cards would continue to automount if there. The fstab entry could be made to mount only if the SD card is found, and thus to not hang up on boot when missing.

A typical place to add a partition is on top of “/usr/local/”. If you were to mount a partition here, then existing content would go away during the mount, but then come back after umount. Something you could consider is recursively copying “/usr/local/*” onto the SD card when ext4 formatted and mounted in an alternate location, and then setting the SD card to mount there…you’d have the SD card copy and the eMMC copy would appear (without any updating) if SD is missing.

Some people also do this with “/home/”. An SD card, if large enough, could actually have two partitions with one mounted on “/usr/local/”, and another on “/home/”. Just “/usr/local/” is the simplest route and does a lot of good since CUDA and many other tools exist there…you could choose to delete the eMMC content there if you are convinced your SD card content is good…or you could just “bzip2 -9” all of that content to compress it unless needed later, and then bunzip2 that content.

What options are you interested in?

Oh wow thank you! I think doing the partition on /usr/local/ makes the most sense for me, it won’t be a huge SD card so would it make sense to copy everything over?

How should I go about doing the partition?

So how large is the SD card? Do you want to use it all on that mount point? I will assume so.

To start with (and this can be done from either the Jetson or the Linux PC host) you’ll want to edit the card with gdisk. To do so you must know the name of the file belonging to the card, and the card must not be mounted anywhere. If you monitor “dmesg --follow” as you plug in the card, then it will obtain a name such as “/dev/mmcblk1”. If you are on a PC this might be “mmcblk0”, but on the Jetson it would be “mmcblk1”. Be careful to not get the wrong device. I am going to assume this is “/dev/mmcblk1”, but adjust for your case…the naming could be quite different other than the “/dev”.

Open up a text console. If you see any message about the device being automounted, first “sudo umount /dev/mmcblk1”. Then run “sudo gdisk /dev/mmcblk1”. This is a simple text-only application, there are GUI alternatives though. The command “?” lists some commands, the command “p” prints the current setup. I’m guessing you have one partition, thus it would look something like this (this is a different customized SD card so it won’t be exactly the same):

GPT fdisk (gdisk) version 1.0.4

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sdi: 30702592 sectors, 14.6 GiB
Model: STORAGE DEVICE  
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 54F02AB3-B275-4057-A5DF-9A2540B026DF
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 30702558
Partitions will be aligned on 2048-sector boundaries
Total free space is 3005 sectors (1.5 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048        30701567   14.6 GiB    0700  colibri

Command (? for help):

You will start by deleting all existing partitions. In this example above there is only one partition, and it is numbered “1”, so the command for deleting “1” is:

d 1

When all partitions are deleted the “p” command will not show any. Then you can add a partition with “n” (new):

Command (? for help): n
Partition number (1-128, default 1): 
1

(I’m avoiding actually deleting my partition so I can’t quote the exact output of writing)

You will then want its type set for ordinary ext4, which is type 8300. If you were to type “l” and hit the enter button a few times it would list all types:

8300 Linux filesystem

To actually change the type, run command “t”, enter “8300”, and then “1” for the partition.

Use “p” to verify you have one partition using the entire content, and that it is type 8300. Then save and exit by using the command “w”.

Assuming the SD card as a whole is “/dev/mmcblk1”, then that first partition will be “/dev/mmcblk1p1”. You can verify it exists:

ls /dev/mmcblk1p1

To format this (and be careful to not name the wrong device):

sudo mkfs.ext4 /dev/mmcblk1p1

To see the label and verify information (name the disk as a whole):

lsblk -f /dev/mmcblk1

The UUID is very useful, write this down. We’ll tell it to auto mount if present to “/usr/local”. This is done by editing “/etc/fstab” (such as via “sudo nano /etc/fstab”). You’ll add a line below any existing content. The default will probably be a single line for the rootfs. Your line will need to be edited for your UUID. I’ll use an example UUID of “abc12345-1234-abc1-a123-123456789abc”, and this is not complete, but will allow you to first test:

UUID=abc12345-1234-abc1-a123-123456789abc   /usr/local   ext4  noauto,user  0 2

(btw, you can use extra spacing between items to make the items line up like a table if you want to)

The last parameter “2” is an increment from “1” which is the root filesystem…if fsck is performed, then it is after the rootfs. The “0” says don’t dump for backup, but if you one day wish to have automated backup services, then you might set it to “1”. The SD card is identified by its UUID, so other SD cards will not get mounted here and would behave as expected. Only the card with that UUID would get mounted here. The current specification is for manual mount, not automatic. To test:

sudu mount /usr/local
ls /usr/local
sudo umount /usr/local
ls /usr/local

Assuming this works, then you are ready to make this an optional mount which doesn’t halt boot if not present. If you were in fact wanting this card to be mandatory, then this is what you’d edit that line to be:

UUID=abc12345-1234-abc1-a123-123456789abc   /usr/local   ext4  <b>nofail</b>,rw,suid,dev,exec,auto,nouser,async   0 2

If you were to look up the man page for fstab, then you’d find the item “defaults” is the same as the comma separated list (no spaces allowed):

rw,suid,dev,exec,auto,nouser,async

I simply prepended “nofail,”, which says that if the SD card fails to mount to keep booting and not halt. Incidentally, you can manually change a UUID. If you were to accidentally create a mount option which mandates that SD card to be present, then if you knew the UUID, you could create a different SD card with the same UUID and get around the failure to boot when the original SD dies or goes missing. The reason I said to write the UUID down is in case something goes wrong.

If you reboot, then “df -H -T -t ext4” should show the partition, and so should “lsblk -f”, and the partition will be mounted on “/usr/local”. As long as nothing is using the partition at the moment you can also manually umount (you can either name the mount point or the device):

sudo umount /usr/local

This would then allow you to mount at an alternate location. If you want to copy content from the original “/usr/local”, then having the SD mounted there and covering content isn’t what you want, so we’ll mount it on “/mnt” instead (temporarily…mmcblk1p1 should be the correct partition, please don’t get the wrong partition):

sudo mount /dev/mmcblk1p1 /mnt

After this there are a number of ways to copy content from the eMMC version of “/usr/local” to the SD card at its temp mount point “/mnt”. Example (and you can post more questions if this doesn’t work for you):

cd /usr/local
sudo cp -adpR * /mnt
cd
sudo umount /mnt

Now, at this moment in time, the SD card should have the same content as the eMMC for that subdirectory. You can either “sudo mount /usr/local” or reboot, and then “df -H -T -t ext4” or “lsblk -f” should verify that the SD card is providing the content of “/usr/local”. If you really need space and are ok with it, you could umount the SD and recursively delete (or selectively delete) some of the content of “/usr/local” (I’d advise a back to another computer if you choose that).

I just wanted to say thank you for being super helpful and going through step by step with me! :D

However, I seemed to have hit a snag! When I go to test the the mount on /usr/local after editing the /etc/fstab file I cannot umount. The following error occurs:

sudo: unable to execute /usr/local/bin/umount: Permission denied

I even tried going into the root (sudo -i) and tried the command, but the same error occured.

So it seems as though it is stuck mounted on the xavier! How should I proceed?

Fascinating…“umount” should not be located there. What do you show from “echo $PATH” and from “sudo echo $PATH”? What do you see from “ls -l /usr/local/bin/umount” and “ls -l /bin/umount”?

Try naming the full path:

sudo <b>/bin/</b>umount /usr/local/bin

A disk which is in use cannot be unmounted (umount). If the command to umount is on the disk to umount, it is a self-defeating problem. Now normally I would expect the PATH to show the “/usr/local” version prior to the “/usr” versions of any “bin” directory, but what I am wondering about is that any method to backup “/usr/local/” from the eMMC to the SD card should not have copied “/bin/umount”. It may be a simple case of changing how you populated the SD card, or else “umount” was already in “/usr/local/bin” and should not have been there. So look for “ls /usr/local/bin/umount” with and without the SD card (both cases should not have umount available at that location). You can boot without the SD card to avoid issues rather than just pulling the card while already booted.

Hi, sorry for the delay in response

The whole Xavier ended up not working- every command I would run resulted in a permission denied error, then suddenly I was unable to click on anything and it stopped responding, even after turning it on and off again.

I had to re-flash and start from the beginning.

I have not re-attempted to mount the SD card as I am worried this may happen again

In part I suspect that content copied to the SD card was intercepting the normal mechanisms. The “mount” command would normally never exist at “/usr/local/bin/”. Your PATH environment variable put that content prior to the content on the rest of the system. Also, if auto-mount had already mounted something, then changing its mount would yield permission denied. You should be ok to continue, but how you copy content to the SD card prior to selecting “/usr/local/” as a semi-permanent mount point should be reexamined.

Note that you had content from the root of the entire Linux install on the SD card…the idea would have been to have content only from “/usr/local/”.

You can also clone your rootfs partition (takes a lot of time and disk space on the host) if you worry (you can flash a clone instead of generating a new rootfs each flash).

1 Like