Hi all.
I’m using Jetson nano with an external SD card.
As you know, it has only 16 Gb of internal storage, so after installing all my programs I reach the storage limit.
I have an external SD card connected with 32 Gb of memory, and I wanted to ask if there is an option to maybe install CUDA for example on the SD card? Or is there any option to move another folder to SD that is not needed on boot?
Thank you!
Most CUDA content is located within “/usr/local”. If you have a partition on your SD card, then you can easily copy that content into that partition, followed by mounting the partition on “/usr/local”. There are a few details, but mostly this is simple. Actually booting from an SD card on an eMMC model is not simple (and perhaps not even supported).
When your SD card is inserted, what do you see from “lsblk -f”? This would give information for setting this up.
Thank you!
/usr/local is quite large but he is only 3GB, and I wish to move at least 10GB of data.
Is there a way you know move more than just /usr/local.
These are the results of lsblk -f:
NAME FSTYPE LABEL UUID MOUNTPOINT
loop0 vfat L4T-README 1234-ABCD
mmcblk0
├─mmcblk0p1 ext4 f2487e47-e801-4863-a652-e09e92cc5203 /
├─mmcblk0p2
├─mmcblk0p3
├─mmcblk0p4
├─mmcblk0p5
├─mmcblk0p6
├─mmcblk0p7
├─mmcblk0p8
├─mmcblk0p9
├─mmcblk0p10
├─mmcblk0p11
├─mmcblk0p12
├─mmcblk0p13
├─mmcblk0p14
├─mmcblk0p15
├─mmcblk0p16
└─mmcblk0p17
mmcblk0boot0
mmcblk0boot1
mmcblk0rpmb
mmcblk1
├─mmcblk1p1 ext4 89b53264-9942-4638-9b91-834c206faa1c
└─mmcblk1p2 ext4 cb4c2787-aca8-4f47-95ae-4408664c79f8
zram0 [SWAP]
zram1 [SWAP]
zram2 [SWAP]
zram3 [SWAP]
Where mmcblk1 is the SD card
Note that eMMC is mmcblk0, and will be ignored. Your current SD card (mmcblk1) has two partitions on it, and this is basically what you want if you want more than one subdirectory to be replaced with the SD card handling it. The simple method is to just mount a partition at points which use a lot of storage, but which are not related to boot. This can be extended with multiple partitions. However, I don’t know the size of your current SD card, nor do I know if there is content on mmcblk1p1 or mmcblk1p2 which you want to keep and not destroy.
For the purpose of examples, and not because this is what you really have, I will pretend that mmcblk1p1 and mmcblk2p2 are both 32GB on an SD card of size 64GB. I will use the actual UUIDs though, but if you format or repartition, then it is likely the UUID will change.
The first goal is to make an entry in “/etc/fstab” which “allows” mount of the first partition on “/usr/local”, but which does not require mount for boot to continue. Consider adding this line into fstab:
UUID=89b53264-9942-4638-9b91-834c206faa1c /usr/local ext4 noauto,user 0 1
The above, if the partition is not already mounted (be sure to “umount /dev/mmcblk1p1” prior to experiment with this if the system auto mounted the partition), will allow a user to mount the partition, but won’t require or attempt to mount the partition during boot (we’ll adjust that in steps):
sudo mount /usr/local
From that point on only that partition will be used from that specific UUID without requiring naming the UUID or device partition. Note that in “mount /usr/local” the system itself determines which partition to use. Note that most people would use “defaults” instead of the “noauto,user” argument in field 4. This is why the partition won’t automatically mount. The equivalent to “defaults” can be found in “man fstab”, but I’ll note it here:
rw, suid,dev,exec,auto,nouser,async
Even the default would be ok for use with the SD card mount at that location, but I wanted you to try to manually “sudo mount /usr/local” and “sudo umount /usr/local” just to see if it works. Next we’ll turn it into “nearly” “defaults”…we’ll make it so that missing the partition does not cause boot to fail:
UUID=89b53264-9942-4638-9b91-834c206faa1c /usr/local ext4 rw, suid,dev,exec,auto,nouser,async,nofail 0 1
(the “nofail” is extremely important since it allows boot without that specific partition)
Unfortunately, auto mount when a user logs in to the GUI needs to be disabled if it is automatically mounted. I have mine disabled already, can you tell me the name of files you find in “ls /etc/udev/rules.d/*”? One of them probably has a rule for mounting to a “run” location. That one could be temporarily disabled by changing the name…I prefer to compress that file with gzip to disable (then it can be reenabled by gunzip).
If you can set up a line which allows that to mount when present, but won’t fail when missing, and does not try to automount, then we can move on to the next steps.
Thank you.
This is the result of the command:
/etc/udev/rules.d/90-alsa-asound-tegra.rules /etc/udev/rules.d/92-hdmi-audio-tegra.rules /etc/udev/rules.d/99-nv-l4t-usb-host-config.rules /etc/udev/rules.d/99-nv-wifibt.rules /etc/udev/rules.d/99-tegra-mmc-ra.rules
/etc/udev/rules.d/91-xorg-conf-tegra.rules /etc/udev/rules.d/99-nv-l4t-usb-device-mode.rules /etc/udev/rules.d/99-nv-ufs-mount.rules /etc/udev/rules.d/99-tegra-devices.rules
Thank you very much for the answer, but the /usr/local folder is only 3GB of storage, is there any other way you know to save more space?
Were you able to manually mount or umount after adding the fstab entry?
I’m not sure which udev rule matters for this case, but can you attach this to the thread (or mouse copy and paste):
/etc/udev/rules.d/99-nv-ufs-mount.rulesI’m thinking that if reboot and the partition is auto mounted, then neutralizing that file will stop auto mount. Assuming your exact partition is not auto mounted to a generic location, then we could set up the rest of the fstab and have the device mount correctly if present.
Note that your partition will replace the 3GB on “/usr/local”, and if your partition is 64GB, then “/usr/local” will become 64GB whenever the mount is present.
Were you able to manually mount or umount after adding the
fstabentry?
Yes I was able to do so.
All the code of this file is anyway disabled:
#
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Mount UFS card when detected.
#ACTION=="add", KERNEL=="sd[a-z][0-9]", SUBSYSTEM=="block", RUN{program}+="/usr/bin/systemd-mount --no-block --automount=yes --collect $devnode /media/%E{ID_FS_UUID}"
# Unmount UFS card when removed.
#ACTION=="remove", KERNEL=="sd[a-z][0-9]", SUBSYSTEM=="block", RUN{program}+="/usr/bin/systemd-umount /media/%E{ID_FS_UUID}"
Maybe because I have no GUI whatsoever.
Now, After I added:
/dev/mmcblk1p1 /usr/local ext4 rw, suid,dev,exec,auto,nouser,async,nofail 0 1
to the /etc/fstab I couldn’t run the mount command. I get:
mount: /etc/fstab: parse error at line 9 -- ignored
mount: /usr/local: can't find in /etc/fstab.
edit2:
It worked! Everything seems to work.
Thank you. Do you think it will work for other /usr folders?
edit:
When I tried the fstab entry with the UUID it worked.
What do I need to do next? Do I need to copy the data to there and remove the old one?
Thanks
Just something to think about as you go through what follows: When you mount a new partition on “/usr/local”, that original content remains, but is not visible. Reads and writes will refer to the mounted device until you umount the mounted device. The original content remains as backup until you umount. This is a bit of a safety.
Now that you know you can manually get the correct mount there are these steps:
/mnt” (which is traditional)./usr/local/*” (including device special files, so normal “cp” is not the method) to the temp location, which I will assume is “/mnt” once the partition is mounted there./usr/local/*” content you can, but probably should not yet be performed even if you can (wait till you actually need the space…you might not need the space).fstab entry to automatically mount if present, rather than to only manual mount. Make sure the entry continues to have the “nofail” option.About backing up content of “/usr/local/*” to “/mnt” (assuming your partition is temporarily mounted there): Recursive copy with “cp” may work, but there are a lot of special cases (especially for pseudo files) where it won’t do the job. I’ll recommend rsync for this (which is designed for true backup of everything of every type).
FYI, a bonus note prior to describing rsync, is that it has an option “--dry-run”. Using that option stops anything from actually occurring, aside from the messages which show up on what would be done if it were actually working. You can always add that to see what would happen without actually performing a sync. Then remove that option when you are convinced the right thing will actually occur.
Also note that this can occur from one machine to another, e.g., your Nano could be connected to a host PC by ethernet, and rsync could occur to the mounted SD card on the host PC. I am just going to assume this is performed locally on the Jetson from “/usr/local/*” to “/mnt” with the correct partition mounted to “/mnt”.
Try this (first with “--dry-run”) to see if it can duplicate the “/usr/local/*” content:
sudo rsync -avcrltxP --info=progress2,stats2 --delete-before --numeric-ids --exclude 'lost+found' /usr/local/ /mnt
(don’t forget to also use “--dry-run” first as a test)
Some of the options in the above are redundant, and if there is no ext4 “lost+found/” subdirectory in the source content then this would not be needed, but it is a good idea to just include this anyway.
If you can copy the content, and then browse that content to see everything is a match (e.g., some file sizes and permissions can be examined to see matching content), then you are ready for automatic fstab mounting. Is the content copy working?
Thank you very much!
I did what you had said, and it all seems to work.
I now can work with the external SD card on /usr/local from boot!
Do you think this process can also work on other /usr folders?
That depends. Folders/directories which are not part of boot can do this. So for example, you could not use the entire “/usr” on the SD card. Boot needs that content, and would not be able to mount the SD card without the content, and thus a bit of a “catch 22”. This is where a modification of the “initial ramdisk” (“initrd”) is used…a minimal amount of content would be placed in the initrd to replace the missing “/usr” content required during boot, and the initrd would then no longer require “/usr”.
Other directories, e.g., “/home”, are never required for boot, so those can be replaced without issue.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.