Can the NGC PyTorch images be used on Jetson?

Can I use NGC PyTorch images on Jetson devices? From my tests on a Jetson Xavier AGX with Jetpack 5.0.2, the NGC PyTorch images have trouble running some PyTorch functions when a Tensor is on GPU.

The NGC PyTorch images I am talking about are the non-l4t ones. They are labeled as being compatible with both amd64 and arm64 architectures, so I suppose it could run on Jetson devices. I test the 22.12-py3 version of the NGC PyTorch images on a Jetson Xavier AGX with Jetpack 5.0.2. It seems that when the Tensor is on GPU, certain PyTorch operation will fail.

However, the NGC L4T PyTorch images work. I tested the r35.1.0-pth1.12-py3 version, and the following is the result.
Screenshot from 2022-12-24 15-14-03

I’d like to use the NGC PyTorch image instead of the NGC L4T PyTorch one because I need Torch-TensorRT, which is not present in the current L4T images. Compiling Torch-TensoRT for the L4T image on Jetson devices is such a long (about 4 hours on Xavier AGX if things are successful) and painful process.

Any help is appreciated, thank you!