hello,
i’m using Jetson tx2. lately we bought some more jetsons.
sometimes i copy files through scp to my jetson. and after a restart the file i copied is with 0 bytes size.
before the reset it looked fine.
it not always happens.
sometimes even after i copy it again, it keeps remaining 0 size.
is anyone have the same problem?
thanks
Are you using a clean shutdown? Filesystems cache, and as a result, if there were not a journal to replay after an unclean shutdown, then the filesystem would be corrupt and unusable without some form of recovery. The journal prevents corruption, but results in truncating any data changes which had not been committed from cache. How did you shutdown?
i don’t know if always i do clean shutdown. sometimes, i simply turn power off the device.
sometimes send the reboot command.
what do you mean by “journal to replay”?
why should it take so much time (more than 1 second) to the filesystem to sync to the flash device?
as i said, it is not happening on all Jetsons. there are some that are more vulnerable.
Turning off power is a good reason why a file could be zero size.
The ext4 filesystem has a “journal” which is filled in as changes are made/written to the disk (the journal does not use any kind of buffer…the journal is “synchronous”). Only after the content the journal entry refers to is guaranteed flushed to disk (the journal says “I am about to write this node, but have not yet completed the write”) will the journal entry be deleted. A fully flushed disk (no cache or temporary operation in operation) is guaranteed to maintain exactly what it has on power off. Upon reboot, if there is a journal, then it means some content was not guaranteed written. The entry is replayed in reverse, and any changes are reverted. Corruption is prevented at the cost of loss of those final writes.
What determines if this occurs varies. The system uses caching to improve performance. Time and “pressure” to write schedules what is written, and when. Changes in system load and other unpredictable details implies you won’t know which system will lose something in cache versus which will have written the cache to disk before you remove power. The Jetsons and the filesystem are doing as intended to keep the system bootable. Keeping partial data implies you have a high risk of losing all data.
So…always shut down cleanly. You could mount in synchronous mode, but your eMMC will die much faster and performance will be orders of magnitude slower.
as i said, it not always happen, so i guess it is something related to the flash of the Jetson that make problems.
the odd thing is that the file is in the filesystem with its name, but its size is 0. so something did made through.
i also talk about seconds (more than 10 if not more) that there is no shutdown. i would expect the file system to sync.
as you say i also prefer not to change the eMMC way of work (i also don’t know how :-))
If you shut down cleanly, do you still get a file size of 0? FYI, files are not written in a single pass, at least so far as caching and buffers going to actual disk are concerned. When you first write a file, and then read a file, you are only guaranteed that the buffer has the content…the content may not have been committed to disk yet. In that case a clean shutdown would guarantee flush of buffer and cache to disk prior to completing shutdown. This is completely unpredictable as the schedule for committing to disk is generally out of our hands. If you still get an error after a clean shutdown, where you know the file size is tested as greater than zero prior to shutdown, but then shows as zero after reboot, then you have a genuine bug.
You wouldn’t want to change an old technology mechanical hard drive to run synchronously due to extreme loss of performance. You wouldn’t want to change eMMC to synchronous due to both a dramatic performance loss and due to death of the eMMC in a short time.
If you have checked that a file is greater than zero size, and then the file reverts to zero after a clean shutdown, then you should note that on the forums because it would be a real bug. All bets are off with loss of power or unclean shutdown. 10 seconds of wait won’t matter since there is a kind of scheduler…the system won’t commit writes on a solid state until it needs to, this is the way the driver protects solid state memory from wearing out too soon. Even on an old tech hard drive you wouldn’t get a “guarantee” at 10 seconds, but eMMC purposely avoids unnecessary writes…the scheduler has no idea you are about to pull the plug.
thanks for the answers :-)
“If you still get an error after a clean shutdown, where you know the file size is tested as greater than zero prior to shutdown, but then shows as zero after reboot, then you have a genuine bug.”
this is what i see. the file has size before power down and then it becomes zero.
as i said, it doesn’t happen on all jetsons… one jetson every 20 or so.
i guess it is a bug.
Yes, a clean shutdown and empty file (after seeing a positive file size) is an error. Do you have serial console connected? If so, can you run “dmesg --follow” on the serial console, and log whatever shows up during the shutdown? There are still other reasons to differentiate between: Whether there was an unclean umount due to a process interfering, or if the reason is occurring despite umount succeeding. If a process is stuck and locking the filesystem, then you’d have the same effect as an unclean shutdown.