This is an issue caused by the USB auto suspend mechanism on your host PC, and it will happen on all Orin series devices (AGX Orin, Orin NX, Orin Nano), and you should look at this:
I don’t believe it to be a solution, and you may get it to work just because you are lucky this time.