Did you start with flashing the AGX Xavier with SDKmanager, selecting Jetpack 4.51 and checking all the options ? Then if you use the standard build and deploy with Isaac SDK 2021.1, there are no missing modules. So I’m guessing that a step was skipped during the initial flashing of the AGX Xavier.
Also you need to run the Isaac SDK script to install dependencies, both for the host and the AGX Xavier (there are different scripts for host PC and Jetson). Maybe you forgot the script for the Jetson.