Issues Installing pytorch through jetson inference install script

chace.medeiros · September 30, 2023, 6:13pm

Hi,

After several attempts, several SD card reflashes, and lot’s of googling - i am still unable to successfully install pytorch through the inference install script on my Jetson Nano 4gb. Here is my jetpack version:

chace@chace:~/jetson-inference/build$ sudo apt-cache show nvidia-jetpack
Package: nvidia-jetpack
Version: 4.6.4-b39
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-l4t-jetson-multimedia-api (>> 32.7-0), nvidia-l4t-jetson-multimedia-api (<< 32.8-0), nvidia-cuda (= 4.6.4-b39), nvidia-tensorrt (= 4.6.4-b39), nvidia-nsight-sys (= 4.6.4-b39), nvidia-cudnn8 (= 4.6.4-b39), nvidia-opencv (= 4.6.4-b39), nvidia-container (= 4.6.4-b39), nvidia-visionworks (= 4.6.4-b39), nvidia-vpi (= 4.6.4-b39)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6.4-b39_arm64.deb
Size: 29388
SHA256: adf7a6660f73cdc4f95bc15c48d8588688e3afa5ee18bfd5b3a3caa3a458aa02
SHA1: 5abbe0df74f71579c1a0ee30ab7c2c236e1bcdbb
MD5sum: ec293a56d17f2b2793448d621811330d
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.6.3-b17
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-l4t-jetson-multimedia-api (>> 32.7-0), nvidia-l4t-jetson-multimedia-api (<< 32.8-0), nvidia-cuda (= 4.6.3-b17), nvidia-tensorrt (= 4.6.3-b17), nvidia-nsight-sys (= 4.6.3-b17), nvidia-cudnn8 (= 4.6.3-b17), nvidia-opencv (= 4.6.3-b17), nvidia-container (= 4.6.3-b17), nvidia-vpi (= 4.6.3-b17)
Homepage: Jetson - Embedded AI Computing Platform | NVIDIA Developer
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6.3-b17_arm64.deb
Size: 29368
SHA256: 694254a8667ebbf13852548bdd13a5b8ae61481ac059845b706398eefdcb9e01
SHA1: 67140fc8463ec61fd69352b225244b639c799edd
MD5sum: afa1382b6caded6b736d494fc481bab4
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.6.2-b5
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.6.2-b5), nvidia-opencv (= 4.6.2-b5), nvidia-cudnn8 (= 4.6.2-b5), nvidia-tensorrt (= 4.6.2-b5), nvidia-visionworks (= 4.6.2-b5), nvidia-container (= 4.6.2-b5), nvidia-vpi (= 4.6.2-b5), nvidia-l4t-jetson-multimedia-api (>> 32.7-0), nvidia-l4t-jetson-multimedia-api (<< 32.8-0)
Homepage: Jetson - Embedded AI Computing Platform | NVIDIA Developer
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6.2-b5_arm64.deb
Size: 29378
SHA256: 925f4abff97e6024d86cff3b9e132e7c7554d05fb83590487381b7e925d5b2bb
SHA1: e3ef727e87df5c331aece34508c110d57d744fe9
MD5sum: 7cb2e387af41bc8143ac7b6525af7794
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.6.1-b110
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.6.1-b110), nvidia-opencv (= 4.6.1-b110), nvidia-cudnn8 (= 4.6.1-b110), nvidia-tensorrt (= 4.6.1-b110), nvidia-visionworks (= 4.6.1-b110), nvidia-container (= 4.6.1-b110), nvidia-vpi (= 4.6.1-b110), nvidia-l4t-jetson-multimedia-api (>> 32.7-0), nvidia-l4t-jetson-multimedia-api (<< 32.8-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6.1-b110_arm64.deb
Size: 29366
SHA256: acfd9e75af780eab165361d61de4b4fe4974890864fe191060b402ac4c2f54d5
SHA1: a016568ac53705acc145a9f7e60505707bea259f
MD5sum: 79be976b184a8c885bd9169ea5b7fb7b
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

And here is the error message I am getting when trying to execute the script (and after following the other steps in the procedure leading up to this):

Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-yfzvgbyx/grpcio/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-b0li_63y-record/install-record.txt --single-version-externally-managed --compile --user --prefix=” failed with error code 1 in /tmp/pip-build-yfzvgbyx/grpcio/

----------------------------------------

Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-qi1bxm78/numpy/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-jve_f0v9-record/install-record.txt --single-version-externally-managed --compile --user --prefix=” failed with error code 1 in /tmp/pip-build-qi1bxm78/numpy/
[jetson-inference] failed to install torch-1.6.0-cp36-cp36m-linux_aarch64.whl
[jetson-inference] – command: pip3 install torch-1.6.0-cp36-cp36m-linux_aarch64.whl
[jetson-inference] – error code: 1
[jetson-inference] failed to install PyTorch v1.6.0 (Python 3.6)

[jetson-inference] installation complete, exiting with status code 0
[jetson-inference] to run this tool again, use the following commands:

$ cd <jetson-inference>/build
$ ./install-pytorch.sh

chace.medeiros · September 30, 2023, 6:15pm

More debug info:

Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-yfzvgbyx/grpcio/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-b0li_63y-record/install-record.txt --single-version-externally-managed --compile --user --prefix=” failed with error code 1 in /tmp/pip-build-yfzvgbyx/grpcio/
Exception information:
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/pip/basecommand.py”, line 215, in main
status = self.run(options, args)
File “/usr/lib/python3/dist-packages/pip/commands/install.py”, line 360, in run
prefix=options.prefix_path,
File “/usr/lib/python3/dist-packages/pip/req/req_set.py”, line 784, in install
**kwargs
File “/usr/lib/python3/dist-packages/pip/req/req_install.py”, line 878, in install
spinner=spinner,
File “/usr/lib/python3/dist-packages/pip/utils/init.py”, line 725, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-yfzvgbyx/grpcio/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-b0li_63y-record/install-record.txt --single-version-externally-managed --compile --user --prefix=” failed with error code 1 in /tmp/pip-build-yfzvgbyx/grpcio/
torch-1.6.0-cp36-cp36m-li 100%[==================================>] 256.58M 3.45MB/s in 66s
Processing ./torch-1.6.0-cp36-cp36m-linux_aarch64.whl
Collecting numpy (from torch==1.6.0)
Using cached https://files.pythonhosted.org/packages/51/60/3f0fe5b7675a461d96b9d6729beecd3532565743278a9c3fe6dd09697fa7/numpy-1.19.5.zip
Collecting future (from torch==1.6.0)
Using cached https://files.pythonhosted.org/packages/8f/2e/cf6accf7415237d6faeeebdc7832023c90e0282aa16fd3263db0eb4715ec/future-0.18.3.tar.gz
Building wheels for collected packages: numpy, future
Running setup.py bdist_wheel for numpy … error
Complete output from command /usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-qi1bxm78/numpy/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” bdist_wheel -d /tmp/tmp1wvtuzvqpip-wheel- --python-tag cp36:
Running from numpy source directory.
Cythonizing sources

Error compiling Cython file:

…
cdef sfc64_state rng_state

  def __init__(self, seed=None):
      BitGenerator.__init__(self, seed)
      self._bitgen.state = <void *>&self.rng_state
      self._bitgen.next_uint64 = &sfc64_uint64
                                 ^

_sfc64.pyx:90:35: Cannot assign type ‘uint64_t (*)(void ) except? -1 nogil’ to 'uint64_t ()(void *) noexcept nogil’. Exception values are incompatible. Suggest adding ‘noexcept’ to type ‘uint64_t (void *) except? -1 nogil’.
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_common.pyx
Processing numpy/random/mtrand.pyx
Processing numpy/random/_sfc64.pyx
Traceback (most recent call last):
File “/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py”, line 235, in
main()
File “/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py”, line 231, in main
find_process_files(root_dir)
File “/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py”, line 222, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File “/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py”, line 188, in process
processor_function(fromfile, tofile)
File “/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py”, line 78, in process_pyx
[sys.executable, ‘-m’, ‘cython’] + flags + [“-o”, tofile, fromfile])
File “/usr/lib/python3.6/subprocess.py”, line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command ‘[’/usr/bin/python3’, ‘-m’, ‘cython’, ‘-3’, ‘–fast-fail’, ‘-o’, ‘_sfc64.c’, ‘_sfc64.pyx’]’ returned non-zero exit status 1.
Traceback (most recent call last):
File “”, line 1, in
File “/tmp/pip-build-qi1bxm78/numpy/setup.py”, line 508, in
setup_package()
File “/tmp/pip-build-qi1bxm78/numpy/setup.py”, line 488, in setup_package
generate_cython()
File “/tmp/pip-build-qi1bxm78/numpy/setup.py”, line 285, in generate_cython
raise RuntimeError(“Running cythonize failed!”)
RuntimeError: Running cythonize failed!

Failed building wheel for numpy
Running setup.py clean for numpy
Complete output from command /usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-qi1bxm78/numpy/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” clean --all:
Running from numpy source directory.

setup.py clean is not supported, use one of the following instead:

- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
                    files that aren't checked into the git repo)

Add --force to your command to use it anyway if you must (unsupported).

Failed cleaning build dir for numpy
Running setup.py bdist_wheel for future … error
Complete output from command /usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-qi1bxm78/future/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” bdist_wheel -d /tmp/tmpq7xdp2b6pip-wheel- --python-tag cp36:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] …]
or: -c --help [cmd1 cmd2 …]
or: -c --help-commands
or: -c cmd --help

error: invalid command ‘bdist_wheel’

Failed building wheel for future
Running setup.py clean for future
Failed to build numpy future
Installing collected packages: numpy, future, torch
Running setup.py install for numpy … error
Complete output from command /usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-qi1bxm78/numpy/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-jve_f0v9-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
Running from numpy source directory.

Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:

  - `pip install .`       (from a git repo or downloaded source
                           release)
  - `pip install numpy`   (last NumPy release on PyPi)


Cythonizing sources

Error compiling Cython file:
------------------------------------------------------------
...
    cdef sfc64_state rng_state

    def __init__(self, seed=None):
        BitGenerator.__init__(self, seed)
        self._bitgen.state = <void *>&self.rng_state
        self._bitgen.next_uint64 = &sfc64_uint64
                                   ^
------------------------------------------------------------

_sfc64.pyx:90:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'uint64_t (void *) except? -1 nogil'.
numpy/random/_bounded_integers.pxd.in has not changed
numpy/random/_common.pyx has not changed
numpy/random/mtrand.pyx has not changed
Processing numpy/random/_sfc64.pyx
Traceback (most recent call last):
  File "/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py", line 235, in <module>
    main()
  File "/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py", line 231, in main
    find_process_files(root_dir)
  File "/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py", line 222, in find_process_files
    process(root_dir, fromfile, tofile, function, hash_db)
  File "/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py", line 188, in process
    processor_function(fromfile, tofile)
  File "/tmp/pip-build-qi1bxm78/numpy/tools/cythonize.py", line 78, in process_pyx
    [sys.executable, '-m', 'cython'] + flags + ["-o", tofile, fromfile])
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'cython', '-3', '--fast-fail', '-o', '_sfc64.c', '_sfc64.pyx']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-qi1bxm78/numpy/setup.py", line 508, in <module>
    setup_package()
  File "/tmp/pip-build-qi1bxm78/numpy/setup.py", line 488, in setup_package
    generate_cython()
  File "/tmp/pip-build-qi1bxm78/numpy/setup.py", line 285, in generate_cython
    raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!

----------------------------------------

Command “/usr/bin/python3 -u -c “import setuptools, tokenize;file=‘/tmp/pip-build-qi1bxm78/numpy/setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install --record /tmp/pip-jve_f0v9-record/install-record.txt --single-version-externally-managed --compile --user --prefix=” failed with error code 1 in /tmp/pip-build-qi1bxm78/numpy/
[jetson-inference] failed to install torch-1.6.0-cp36-cp36m-linux_aarch64.whl
[jetson-inference] – command: pip3 install torch-1.6.0-cp36-cp36m-linux_aarch64.whl
[jetson-inference] – error code: 1
[jetson-inference] failed to install PyTorch v1.6.0 (Python 3.6)

[jetson-inference] installation complete, exiting with status code 0
[jetson-inference] to run this tool again, use the following commands:

$ cd <jetson-inference>/build
$ ./install-pytorch.sh

dusty_nv · September 30, 2023, 7:30pm

Hi @chace.medeiros, can you try running pip3 install 'Cython<3' first?

Also, you only need PyTorch for training your own models. You can run all the inferencing stuff by skipping it. You can also run the jetson-inference docker container and that will have PyTorch/ect already installed in it.

You can also run the procedure manually like in PyTorch for Jetson or here in the script:

github.com

dusty-nv/jetson-inference/blob/44c7661dfbb97018fbc4651fa4608d5491d3f39e/tools/install-pytorch.sh#L554


      
          	install_deb_package "zlib1g-dev" FOUND_ZLIB
          	install_deb_package "libopenblas-base" FOUND_OPENBLAS
          	install_deb_package "libopenmpi-dev" FOUND_OPENMPI
          
          	# install pip packages
          	pip3 install Cython
          	pip3 install numpy --verbose
          	pip3 install tensorboard --verbose
          	
          	# install pytorch wheel
          	download_wheel pip3 "torch-1.6.0-cp36-cp36m-linux_aarch64.whl" "https://nvidia.box.com/shared/static/9eptse6jyly1ggt9axbja2yrmj6pbarc.whl"
          
          	local wheel_status=$?
          
          	if [ $wheel_status != 0 ]; then
          		echo "$LOG failed to install PyTorch v1.6.0 (Python 3.6)"
          		return 1
          	fi
          
          	# build torchvision
          	move_ffmpeg

chace.medeiros · September 30, 2023, 9:05pm

Hi Dusty,

That did the trick! Thank you for the quick feedback and working solution.

My one follow up question is why? Is it because the packages associated with the working nano jet pack versions are getting stale?

Please let me know, thank you.

-Chace

dusty_nv · October 1, 2023, 12:45am

It’s because Cython, one of the most widely used C-to-Python libraries, released a new major version (v3) that was not backwards-compatible and pip’s default behavior is to pull the latest. In the process, it broke a ton of other packages that haven’t yet migrated to Cython3 (or might never). But yes, I need to update jetson-inference’s PyTorch installer script (and in the larger sense, should probably revisit having that script at all, in lieu of just using container if/when someone wants to train with PyTorch)

Anyways, sorry about that!, and glad you got it working :)

chace.medeiros · October 2, 2023, 11:55am

Hey Dusty,

Thanks for the insight. Is it possible to use the train_sad script to fine tune an existing model I tuned using that same script?

If so what is the best way?

-Chace

dusty_nv · October 2, 2023, 1:44pm

Hi Chance, yes you should be able to use the --resume or --pretrained-ssd arguments to train_ssd.py, and specify the path to the .pth checkpoint that you want to load. Note that the epoch count will appear to have restarted from zero. I also recommend saving resumed models to a different --model-dir to keep them straight.

system · October 16, 2023, 1:45pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Cant install Pytorch on JetsonNano P3450 Jetson Nano pytorch	21	2379	August 16, 2023
PyTorch for Jetson Announcements pytorch , python , kb	1366	666637	May 14, 2024
Pytorch support Jetson Nano	31	4491	October 18, 2021
Issues with installing torchvision 0.13.0 Jetson AGX Orin pytorch	13	62	October 30, 2024
PyTorch Install with Python3 Broken Jetson TX2	26	10785	October 18, 2021
Tensorflow install doesn't work Jetson Xavier NX tensorflow	4	1240	October 18, 2021
Install Pytorch with cuda on Jetson Orin nano Devloper Kit Jetson Orin Nano cuda , pytorch	13	1260	July 30, 2024
Issue to install pytorch on Jetson Orin platform Jetson Orin NX jetson-inference , pytorch	10	4625	June 7, 2023
Jtop install fail Jetson Xavier NX jetson-inference	9	14670	April 6, 2022
Installing pytorch - /usr/local/cuda/lib64/libcudnn.so: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status Jetson TX2 pytorch	20	5292	March 11, 2022

Issues Installing pytorch through jetson inference install script

Error compiling Cython file:

Related topics