CUDA runtime on Jetson Orin AGX

If you are installing torchvision like pip3 install torchvision, it will install the torchvision/PyTorch that was built without GPU support. Instead, install torchvision from source like found under the Installation section of this post: PyTorch for Jetson

I don’t know if the PyTorch wheels will work with CUDA 12 or not, because they were built with CUDA 11.4. You might need to rebuild them for CUDA 12 if you have an issue.