Hello,
We are in the process to migrate our fleet of agx xavier from JP4 (32.7.2) to JP5 (35.6.2) and during stress-tests, we have observed that sporadically when performing mender OTA update and without rebooting, we re-mount the inactive partition some files are incorrectly empty (ratio 0.5%). It is unclear why this is happening. One of the last times that it happened, this was the content of the /bin directory.
If we don’t remount and perform a reboot instead, then we do not observed any empty files. Adding a sleep delay then some errors persisted pointing to possible filesystem issue.
Multiple tests on jetpack5 showed no errors over thousands of flashes, suggesting again the issue may be hardware-specific or related to filesystem state.
Between JP4 and Kernel 4.9 and JP5 and Kernel 5.10 or mender’s related script might differs greatly though.
JP4’s Kernel and its ext4 implementation is being known to have shortcoming with:
- Journal replay timing.
- Mounting data partitions before they are fully clean.
- Systemd services writing too early.
Is similar issue has been reported by other customer or discovered internally? Do you have some suggestions to overcome this problem?
Thanks