If you can, plug an microUSB cable into the Xavier and connect that to a terminal program - teraterm, minicom - whatevery you have and try to capture the bootup output and post it.
There is also a UART1 pin (tx, rx) on the RasPi 80 pin connector that will come up with a login prompt that’s useful. You have to use a TTL-USB adapter - $3 on Amazon.
Post you bootup log- this seems diagnosable.
And you rebuilt the kernel from source, or just flashed with the SDK Manager?