Actually, our Xavier doesn’t have this problem. We just tested powering it off for 8 hours and it still maintains its clock. It looks like there’s a small RTC clock backup battery (or supercap) installed on DevKit PCB (C512?), maybe it went bad on your kit? Check the photo attached.
You can try to fix/replace backup battery/supercap first, and if not successful, then maybe add an external RTC, like DS3231 commonly added to Raspberry Pi boards?
-albertr
