The Xavier is very sensitive to voltage fluctuations. Your power supply really needs to be top-tier for it to work, or you need to re-buffer.
What I do is re-buffer with a LiPo battery and 14 gauge wiring, which is probably overkill, but as this is a mobile robot, I had it available already. Also, the power plug for the devkit is surprisingly sensitive – I’m about to solder some pigtails straight to the kit board (wish me luck!)
If you don’t have a LiPo battery, then perhaps a few thousand uF of low-ERS capacitors could help, but then you have the problem of the power supply inrush current causing problems instead.
The limits are 9V to 20V as per the datasheet:
That being said, there is the dV/dt circuit mentioned above, which is quite trigger happy.