PyTorch for Jetson

arjunslvrj · January 29, 2023, 8:20am

When I try to import torchvision it is showing an error. SyntaxError: future feature annotations is not defined. I am using jetpack 4.6 rev3 with pytorch v1.8.0 and torchvision v0.9.0 .

dusty_nv · January 30, 2023, 2:21am

Hi @arjunslvrj, can you try running pip3 install 'pillow<9' ?

arjunslvrj · January 30, 2023, 11:04am

Thank you dusty

franciscon9k63 · February 8, 2023, 12:47am

CMake can’t find TorchVision when using l4t-pytorch:r35.1.0-pth1.13-py3 docker image and warning of missing kineto library when using libtorch. I am currently trying to load a torchscript model in C++ that requires me to inclued torchvision/vision.h .

CMake Error at CMakeLists.txt:24 (find_package):
  By not providing "FindTorchVision.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "TorchVision", but CMake did not find one.

  Could not find a package configuration file provided by "TorchVision" with
  any of the following names:

    TorchVisionConfig.cmake
    torchvision-config.cmake

  Add the installation prefix of "TorchVision" to CMAKE_PREFIX_PATH or set
  "TorchVision_DIR" to a directory containing one of the above files.  If
  "TorchVision" provides a separate development package or SDK, be sure it
  has been installed.

I have tried to add the following CMAKE_PREFIX_PATHs with no sucess:

list(APPEND  /usr/local/lib/python3.8/dist-packages/torchvision-0.13.0a0+da3794e-py3.8-linux-aarch64.egg/)
list(APPEND  /usr/local/lib/python3.8/dist-packages/torchvision-0.13.0a0+da3794e-py3.8-linux-aarch64.egg/torchvision/)

CMake can find libtorch when I use the following:
list(APPEND CMAKE_PREFIX_PATH /usr/local/lib/python3.8/dist-packages)

But it does give a warning:

CMake Warning at /usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
  CMakeLists.txt:25 (find_package)

dusty_nv · February 8, 2023, 2:19am

Hi @franciscon9k63, it doesn’t appear that the torchvision C++ headers get installed when torchvision is built, and I’m not sure how to build it with that enabled. This is how I build/install torchvision in the container’s dockerfile:

github.com

dusty-nv/jetson-containers/blob/eb2307d40f0884d66310e9ac34633a4c5ef2e083/Dockerfile.pytorch#L79


      
          
          RUN apt-get update && \
              apt-get install -y --no-install-recommends \
          		  git \
          		  build-essential \
                      libjpeg-dev \
          		  zlib1g-dev \
              && rm -rf /var/lib/apt/lists/* \
              && apt-get clean
          
          RUN git clone https://github.com/pytorch/vision torchvision && \
              cd torchvision && \
              git checkout ${TORCHVISION_VERSION} && \
              python3 setup.py install && \
              cd ../ && \
              rm -rf torchvision
          
          # note:  this was used on older torchvision versions (~0.4) to workaround a bug,
          #        but has since started causing another bug as of torchvision 0.11.1
          # ARG PILLOW_VERSION=pillow<7    
          # pip3 install --no-cache-dir "${PILLOW_VERSION}"

Perhaps you could clone that branch of the torchvision repo to get the headers?

Kic1101 · February 9, 2023, 11:53am

hi, I meet a problem. I need your help.
machine: Jetson Xaview, Jetpack 5.1
torch version: torch-1.14.0a0+44dac51c.nv23.1

when executed "torchrun **** ",
Traceback (most recent call last):
File “/home/nvidia/.local/bin/torchrun”, line 5, in
from torch.distributed.run import main
File “/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/run.py”, line 383, in
from torch.distributed.elastic.rendezvous.utils import _parse_rendezvous_config
File “/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/elastic/rendezvous/init.py”, line 131, in
from .api import * # noqa: F403
File “/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/elastic/rendezvous/api.py”, line 10, in
from torch.distributed import Store
ImportError: cannot import name ‘Store’ from ‘torch.distributed’ (/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/init.py)

dusty_nv · February 10, 2023, 2:50pm

Kic1101:

when executed "torchrun **** ",
Traceback (most recent call last):
File “/home/nvidia/.local/bin/torchrun”, line 5, in
from torch.distributed.run import main
File “/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/run.py”, line 383, in
from torch.distributed.elastic.rendezvous.utils import _parse_rendezvous_config
File “/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/elastic/rendezvous/init.py”, line 131, in
from .api import * # noqa: F403
File “/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/elastic/rendezvous/api.py”, line 10, in
from torch.distributed import Store
ImportError: cannot import name ‘Store’ from ‘torch.distributed’ (/home/nvidia/.local/lib/python3.8/site-packages/torch/distributed/init.py)

Hi @Kic1101, these wheels were built with USE_DISTRIBUTED disabled, so they don’t support torch.distributed. If you need that, you would need to rebuild PyTorch from source.

franciscon9k63 · February 11, 2023, 12:14am

Thanks that was it. I followed the instructions from : GitHub - pytorch/vision: Datasets, Transforms and Models specific to Computer Vision

I did the following

ARG LIBTORCH_PATH=/usr/local/lib/python3.8/dist-packages/torch/

RUN git clone https://github.com/pytorch/vision torchvision && \
    cd torchvision && \
    git checkout ${TORCHVISION_VERSION} && \
    mkdir build && \
    cd build && \
    cmake -DCMAKE_PREFIX_PATH=${LIBTORCH_PATH}-DWITH_CUDA=on .. && \
    make -j$(nproc) && \
    sudo make install && \
    rm -rf /torchvision

dusty_nv · February 11, 2023, 1:46am

Thanks @franciscon9k63, that’s good to know how you built it with that enabled 👍

Kic1101 · February 13, 2023, 8:04am

It explains the matter. Thanks a lot.

mkrlng · February 24, 2023, 9:58am

I got problems while installing torchvision. I installed torch 1.8 and wanted to install torchvision 0.9.0, but the result is:

Finished processing dependencies for torchvision==0.9.0a0+01dfa8e
nvidia@ubuntu:~/torchvision$ python3
Python 3.6.9 (default, Nov 25 2022, 14:10:45) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import torchvision
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nvidia/torchvision/torchvision/__init__.py", line 7, in <module>
    from torchvision import datasets
  File "/home/nvidia/torchvision/torchvision/datasets/__init__.py", line 1, in <module>
    from .lsun import LSUN, LSUNClass
  File "/home/nvidia/torchvision/torchvision/datasets/lsun.py", line 2, in <module>
    from PIL import Image
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/home/nvidia/.local/lib/python3.6/site-packages/Pillow-9.4.0-py3.6-linux-aarch64.egg/PIL/Image.py", line 59, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 951, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 894, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1157, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1131, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1112, in _legacy_get_spec
  File "<frozen importlib._bootstrap>", line 441, in spec_from_loader
  File "<frozen importlib._bootstrap_external>", line 544, in spec_from_file_location
  File "/home/nvidia/.local/lib/python3.6/site-packages/Pillow-9.4.0-py3.6-linux-aarch64.egg/PIL/_deprecate.py", line 1
SyntaxError: future feature annotations is not defined

the strange thing: when I want to install a different version of torch, it installs everytime torch 1.8.0. also when I delete and uninstall the package.
Does anyone knows what I can do?

dusty_nv · February 24, 2023, 9:26pm

Please try running pip3 install 'pillow<9'

Try double-checking the URL of the wheel that you downloaded. Perhaps the wheel is for PyTorch 1.8. Each version has a different URL.

Spange · March 10, 2023, 11:01am

I see Jetpack 5.0 comes with Python3.8 and PyTorch v1.12.0.
I’ve installed python3.11 on my Jetson (Xavier), is there any chance of getting PyTorch to work in python3.11? Or should I just stick with python3.8 which after all is provided with Jetpack 5.0

dusty_nv · March 10, 2023, 2:37pm

@Spange I’m not sure if Python 3.11 is officially supported/test in PyTorch yet (especially for aarch64+CUDA), but I did find this thread about it:

github.com/pytorch/pytorch

Support Python 3.11

opened 12:01AM - 10 Oct 22 UTC

closed 11:06PM - 05 Jun 23 UTC

EwoutH

module: ci triaged enhancement module: python frontend

Currently Python 3.11 isn't fully supported in Pytorch yet, including testing in… CI and publishing wheels to PyPI. Python 3.11 is currently in Release Candidate state and will be released in about two weeks. Major packages like NumPy, Pandas, SciPy and Matplotlib have already uploaded their Python 3.11 wheels to PyPI. Python 3.11 new features include: > - [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include Fine-Grained Error Locations in Tracebacks > - [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups and except* > - [PEP 673](https://www.python.org/dev/peps/pep-0673/) -- Self Type > - [PEP 646](https://www.python.org/dev/peps/pep-0646/)-- Variadic Generics > - [PEP 680](https://www.python.org/dev/peps/pep-0680/)-- tomllib: Support for Parsing TOML in the Standard Library > - [PEP 675](https://www.python.org/dev/peps/pep-0675/)-- Arbitrary Literal String Type > - [PEP 655](https://www.python.org/dev/peps/pep-0655/)-- Marking individual TypedDict items as required or potentially-missing > - [bpo-46752](https://bugs.python.org/issue46752)-- Introduce task groups to asyncio > - The [Faster Cpython Project](https://github.com/faster-cpython) is already yielding some exciting results: this version of CPython 3.11 is ~ 19% faster on the geometric mean of the [PyPerformance benchmarks](https://speed.python.org/), compared to 3.10.0. Pytorch is an import package in the deep learning stack, so support at release will help speed up Python 3.11 adoption. It would be great to have full Python 3.11 support in Pytorch by the time Python 3.11.0 gets released, ([expected](https://peps.python.org/pep-0664/) Monday October 6th, 2022). This includes testing in CI and publishing wheels to PyPI. This issue can be used as a tracking issue. This includes: - [x] Pytorch builds fully on Python 3.11 - [x] All Pytorch tests pass on Python 3.11 - [x] All CI is run and green on Python 3.11 - [x] Wheels are uploaded to PyPI for at least one Pytorch release - [x] Manylinux (x86_64) - [x] Manylinux (aarch64) - [x] Windows - [x] macOS cc @seemethere @malfet @pytorch/pytorch-dev-infra @albanD

We only build PyTorch wheels for the default version of Python that comes with Ubuntu (and on JetPack 5.0, that’s Python 3.8). However you could try building it yourself from source.

adam-erickson · March 17, 2023, 12:09am

Heads up that an update is needed as newer PyTorch pip wheels are available, such as for JetPack SDK 5.1 here:

https://developer.download.nvidia.com/compute/redist/jp/v51/pytorch/

Looks like everything is still Python 3.8 while 3.11.2 is the latest stable version.

Thank you, @dusty_nv !

dusty_nv · March 17, 2023, 12:46am

Thanks @adam-erickson, yes new PyTorch wheels will now be on that server, and I’ve posted a link to the official docs at the top of this thread: https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html

Yes, we distribute the PyTorch wheels for the default version of Python that comes with Ubuntu, and for Ubuntu 20.04 (JetPack 5.0/5.1) that is Python 3.8. For Python 3.11, you would need to build PyTorch from source.

Hamzeh.nv · March 27, 2023, 10:47am

maybe you have to reinstall jetpack with the SDK manager

cory_weng · April 3, 2023, 2:13pm

But I installed torch-1.14.0a0+44dac51c.nv23.02 and torch-2.0.0a0+8aa34602.nv23.03, it are incompatible with torchvision.

I have tried 0.15.1, 0.14.1, 0.14.0.

dusty_nv · April 4, 2023, 1:33pm

Hi @cory_weng, have you tried building torchvision from source like shown under the Installation section of the first post of this topic, as opposed to installing torchvision from pip?

brbl · April 4, 2023, 1:56pm

I am trying to compile pytorch v1.13.1 from scratch for the ORIN (initially on 5.0.2), so I can enable distributed and I patched utils/ccp_extension.py per the instructions at the top of the thread. But i am running into a CMake error: Unknown CUDA Architecture Name 7.2, 8.7 in CUDA_SELECT_NVCC_ARCH_FLAGS from cuda_select_nvcc_arch_flags.

Any help would be greatly appreciated. I get why the distributed option is not a common option when using the ORIN but it sure would be nice if there was a version that ran on 5.1 that did have it enabled.

Thanks, and sorry if this has been answered in the thread.

bb