Jetson nano does not boot after power outage: File system corrupted. NO SUPPORT from Nvidia

A working Jetson Nano gets stonedwith a power outage. Does not boot. File system found to be corrupted. Customer support clueless or does not care or does not have a fix. My customer is hopping mad. Says: If a simple Raspberry pi has support for OverlayFS why can the same not be in Jetpack? Is Nvidia serious about jetson Nano being used or it is just a orphaned baby now?..warning to all who plan to use it for real life solutions? Its a toy only?

Nothing prevents YOU as a system design engineer from mounting partitions read-only in fstab, splitting the whole image into multiple partitions, installing some kind of overlayfs/unionfs/aufs, and implementing some kind of recovery mechanisms. It is YOUR sole duty as a vendor to choose hardware and software components and to put everything together correcty for YOUR product. You can’t blame others for failing to do so.

Corruption of writeable filesystems is a well-known problem, and there are many ways to solve this problem. You can install unionfs or overlayfs also on Jetson systems. Its the same as in any other Linux system, there is nothing special here. You can also mount partitions read-only. You can use ram disks (tmpfs). Been there, done it.

I’ve designed custom TX2 carrier boards that get shut down just by removing power. That’s the standard procedure. There are several 100 boards out there, and there has not been a single problem because of that. These systems don’t have a desktop, are locked down, boot off write-protected root partions and only have a single 1G writeable partition for chache and configuration etc. If this partition gets damaged, its erased, formatted and reinitialized from a .tar.gz again. Works for me.

fchk

I am talking about OS corruption @fchkjwlsq It can ONLY be mounted on a partition and there needs to be a well defined procedure for implementing the OverlayFS. Plz see how Raspian (ie Raspberry pi OS) does it. They have a clear menu driven procedure…that prevents people from doing things their own way and creating more issues.

Is this a Joke sir? U mean u are providing a battery on the Carrier board??? Come on…this is an embedded system and should work without such “crutches”. I should be able to yank off power and all should still be fine. Do u ever do a “graceful shutdown” of yr router???/

“there needs to be a well defined procedure for implementing the OverlayFS”

Yes, everything is there. Just read the docs.
https://docs.kernel.org/filesystems/overlayfs.html
There is everything you need to know.

fchk

There is no battery on these boards. There are no graceful shutdowns ever. The system is just hardended to withstand these conditions.

fchk

btw. These are specialized image processing systems.

@fchkjwlsq
Sir Thank you for responding. I went thru the details of the link. To my understanding this only describes the ‘features of the OverlayFS’ which more or less are known. I did NOT see clear well defined steps for its implementation. Can u plz point me to such steps? I will be grateful.

@fchkjwlsq : Sir if there are no batteries can I assume that includes absence of super-caps on the carrier board hardware too? If yes then are you shipping your carrier boards with changes to the Jetpack to be done after installation?

Hi,
We support A/B redundancy of roofs from TX2. When roofs of A is corruprted, it will boot with rootfs of B. However, it is not supported on Jetson Nano, so it may not be recoverable when roots is corrupted.

1 Like

@DaneLLL
Thank you sir. This definitely makes sense. I would still prefer a simpler and more reliable OverlayFS support in Jetpack. For Pi this single feature has made a HUGE difference. I will not be surprised to know that the recent surge in demand for Pi boards is driven by this feature: A true embedded system where one can yank off power at will and not be bothered about OS corruption. Thank you again for clarifying sir!

My carrier boards are part of a larger system and ship with a heavy modified bootloader and Jetpack/Ubuntu LInux plus the required applications for the hardware. There is nothing for the user to service, it’s closed and locked down for a reason.

1 Like

@fchkjwlsq
Just curious: For such a closed system why would you want to touch the bootloader and other components of the jetpack, when it is much easier to build in a 10 second uptime UPS using supercaps or batteries? 10 secs is enough to shut down the system automatically (a service script) when the power goes off. Before Raspbian (Pi) came up with a proper OverlayFS support we used to do that manually in code…and updates became inconvenient …any particular reason for changes to Bootloader and jetpack sir?

I wanted to protect the important filesystems not only from power outage but also from other inadvertent modifications (possible software bugs etc). Read-only boot, root and usr partitions are the safest way to accomplish this. These are never writable in my system, can’t be modified and will not break.

My hardware is very different from a standard devkit, so I had to customize UBoot and kernel and other things anyway.

fchk

1 Like

Thankyou sir!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.