Can not boot because the storage is full

Hi, I have Jetson AGX Xavier (Ubuntu 18.04) then I use it to compile a library for some hours using container. Then I got the error because there is no space left and my bad I shutdown it before I clear some file, library or app to give the storage for booting. And now my Jetson stuck on the black screen with some message. What should I do? I have tried to recovery mode but no success. I don’t matter if I lost the data in my jetson.

You may need to reflash your device by JetPack, you need to use a host PC with Ubuntu OS to install the SDK manager to do that.

Thanks for the fast response. Okay, reflashing my jetson is my last step since I only have ubuntu 22.

This is not “can not boot”. But may just “GUI fails to launch”.

You can still access your board by using serial console.

I’ve tried to reflash but got this error

Was the flash error while using the Ubuntu 22.04 host PC? If so, then you’ll probably need Ubuntu 20.04 or Ubuntu 18.04. One thing I do on my host PC is dual boot with Windows, and I have 1/3 of that space for Windows, 1/3 for Ubuntu 18.04, and 1/3 reserved for future use. When it gets to where I must use Ubuntu 20.04 I’ll install it to the reserved 1/3, and slowly discontinue use of the 18.04 partition (which will eventually be reserved for future 22.04). If your hard drive is big enough you might consider reserving space so you can have two releases of Ubuntu.

FYI, if you mount “/home” on a separate disk, then the small storage of 1/3 of your main drive isn’t too hard to live with.

I’ve installed ubuntu 18.04 then flashed my jetson but I got error. You can see the error at the my comment above. I attached the screen capture.

Is there any button or I/O pin to hard factory reset? Should I use sdk manager or CLI to do that? There is no space on my jetson, so I can’t remove app or library because removing app or library needs some storage to cache it. So I have to remove some file first to get some space, then remove app or library. But my jetson GUI is not launched, so I can’t remove file first. Do flashing jetson need space too (like remove library/app)?

Thanks all. Now I can connect to my jetson using ssh. I’ve deleted docker image then reboot my jetson but my jetson still can not launch GUI.

disk usage

nvidia@nvidia-desktop:~$ df -m
Filesystem     1M-blocks  Used Available Use% Mounted on
/dev/mmcblk0p1     28095 27649         0 100% /
none               15585     0     15585   0% /dev
tmpfs              15961     1     15961   1% /dev/shm
tmpfs              15961   205     15756   2% /run
tmpfs                  5     1         5   1% /run/lock
tmpfs              15961     0     15961   0% /sys/fs/cgroup
tmpfs               3193     0      3193   0% /run/user/1000

I also try to remove chromium

nvidia@nvidia-desktop:~$ sudo apt remove chromium-browser
Reading package lists... Done
Building dependency tree       
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 chromium-browser-l10n : Depends: chromium-browser (>= 91.0.4472.101-0ubuntu0.18.04.1) but it is not going to be installed
                         Depends: chromium-browser (< 91.0.4472.101-0ubuntu0.18.04.1.1~) but it is not going to be installed
 cuda-toolkit-10-2 : Depends: cuda-documentation-10-2 (>= 10.2.300) but it is not going to be installed
                     Depends: cuda-samples-10-2 (>= 10.2.300) but it is not going to be installed
 libvisionworks-sfm-dev : Depends: libvisionworks-samples (>= 1.6.0) but it is not going to be installed
 libvisionworks-tracking-dev : Depends: libvisionworks-samples (>= 1.6.0) but it is not going to be installed
 tensorrt : Depends: libnvinfer-samples (= 8.0.1-1+cuda10.2) but it is not going to be installed
            Depends: libnvinfer-doc (= 8.0.1-1+cuda10.2) but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

But if I run fix broken I get the following error

nvidia@nvidia-desktop:~$ sudo apt --fix-broken install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  cpp-8 libgfortran-8-dev libgfortran5
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  cuda-documentation-10-2 cuda-samples-10-2 libnvinfer-doc libnvinfer-samples libvisionworks-samples
The following NEW packages will be installed:
  cuda-documentation-10-2 cuda-samples-10-2 libnvinfer-doc libnvinfer-samples libvisionworks-samples
0 upgraded, 5 newly installed, 0 to remove and 241 not upgraded.
Need to get 630 MB of archives.
After this operation, 1,196 MB of additional disk space will be used.
E: You don't have enough free space in /var/cache/apt/archives/.

You can just try to use rm command to remove unnecessary file and give at least 3~5 % space. Your GUI may come back then…

From my point of view, you can go to /var/log/ and delete some logs… Most of them are just cache and no need to keep them there…

I have two 5 GB docker image. I’ve deleted all using sudo docker image rm -f [image] then power off/reboot my device but my disk space is same (no space left).

nvidia@nvidia-desktop:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   28G   28G     0 100% /
none             16G     0   16G   0% /dev
tmpfs            16G  4.0K   16G   1% /dev/shm
tmpfs            16G  141M   16G   1% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs            16G     0   16G   0% /sys/fs/cgroup
tmpfs           3.2G     0  3.2G   0% /run/user/1000
tmpfs           3.2G  4.0K  3.2G   1% /run/user/120

The truth looks like you didn’t successfully in delete anything…

Solved, thanks for your all supports.
I run docker system prune to delete:

- all stopped containers
  - all networks not used by at least one container
  - all dangling images
  - all dangling build cache

There is no “reset” other than flashing. “Recovery mode” makes the Jetson available as a custom primitive device which the flash software understands, but does not in and of itself make any kind of change to the Jetson. The JetPack/SDK Manager GUI is just a front end to the actual flash software. The “flash.sh” software is basically what flashes on command line. Flash software will not care what is already on the Jetson, and will not care if it is full or completely empty since it would replace that content.

If the unit boots, then you can use serial console to delete content and free up space. This will have no effect on flash, but could free up space if you have a full system and need to work with it without flash. Since you could end up with a full filesystem again it is better to just use serial console or ssh as you have, and then eliminate something. The trick is choosing what to eliminate.

Keep in mind that some content could be added back in later, but based on what is visible, perhaps try this (naming all of those packages on a single command line):

sudo apt --fix-broken remove chromium-browser chromium-browser-l10n cuda-toolkit-10-2 libvisionworks-sfm-dev libvisionworks-tracking-dev tensorrt

There may also be non-package deletes available, e.g., is there anything in “~/Downloads/*”?

Thanks for the suggestion. I’ve deleted the content using ssh and now my jetson can boot normally without flashing. But I think I have another problem about the space. I need to compile a library for Jetson usage so I have to compile it on an ARM device which is jetson it self. After I’ve installed all needed library and deleted unnecessary files and apps, the space is not enough. I also deleted documentations and samples package. The compiling process stucks at 98% and shows the error “no space left on device”. So the device need more space to compile, maybe 2 or 3 GB of extra space is enough. Then I found this to minimize jetson disk usage, and I tried to remove GUI but no success.

Sorry, the nvubuntu-bionic-packages_only-in-desktop.txt link on the guideline is not correct. And now I can run my jetson without GUI after downloading from the right link.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.