As i remember, no error or warning.
I did came through this solution, and i tried that, but no luck.
My thinking now is to reflash xavier from scratch
And reinstall pythorch.
Maybe my steps were wrong,
Could you please send my step by step
That could be very helpful
Thanks in advance.
It might be that PyTorch needs to be compiled after OpenBLAS or libblas is already installed, I am not sure. I will have to look into it for PyTorch v1.4.0 when that gets released.
OK, I get the same error when running that with the torch-1.3.0 wheel. When PyTorch 1.4 is released, I will try rebuilding it with OpenBLAS installed beforehand. That may or may not require everyone to install OpenBLAS or libblas as a pre-requisite, I’m not sure, so we’ll have to see about that.
In the meantime, you could try recompiling PyTorch after you have installed OpenBLAS. When setup.py bdist_wheel is run, you should see something along the lines of “OpenBLAS…detected” near the beginning of the process when it is configuring the build.
Hi,
I follow commend flow:
pip3 install numpy torch-1.3.0-cp36-cp36m-linux_aarch64.whl
Install failed with information:
Collecting numpy
Using cached https://files.pythonhosted.org/packages/40/de/0ea5092b8bfd2e3aa6fdbb2e499a9f9adf810992884d414defc1573dca3f/numpy-1.18.1.zip
Processing ./torch-1.3.0-cp36-cp36m-linux_aarch64.whl
Building wheels for collected packages: numpy
Running setup.py bdist_wheel for numpy ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-0xuwkh30/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpe_g4dwdupip-wheel- --python-tag cp36:
Running from numpy source directory.
Cythonizing sources
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_generator.pyx
Traceback (most recent call last):
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 61, in process_pyx
from Cython.Compiler.Version import version as cython_version
ModuleNotFoundError: No module named 'Cython'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 238, in <module>
main()
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 234, in main
find_process_files(root_dir)
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 225, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 191, in process
processor_function(fromfile, tofile)
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 66, in process_pyx
raise OSError('Cython needs to be installed in Python as a module')
OSError: Cython needs to be installed in Python as a module
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-0xuwkh30/numpy/setup.py", line 488, in <module>
setup_package()
File "/tmp/pip-build-0xuwkh30/numpy/setup.py", line 469, in setup_package
generate_cython()
File "/tmp/pip-build-0xuwkh30/numpy/setup.py", line 275, in generate_cython
raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!
----------------------------------------
Failed building wheel for numpy
Running setup.py clean for numpy
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-0xuwkh30/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
Running from numpy source directory.
`setup.py clean` is not supported, use one of the following instead:
- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
files that aren't checked into the git repo)
Add `--force` to your command to use it anyway if you must (unsupported).
----------------------------------------
Failed cleaning build dir for numpy
Failed to build numpy
Installing collected packages: numpy, torch
Running setup.py install for numpy ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-0xuwkh30/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-ijydcqk6-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
Running from numpy source directory.
Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:
- `pip install .` (from a git repo or downloaded source
release)
- `pip install numpy` (last NumPy release on PyPi)
Cythonizing sources
numpy/random/_bounded_integers.pxd.in has not changed
Processing numpy/random/_generator.pyx
Traceback (most recent call last):
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 61, in process_pyx
from Cython.Compiler.Version import version as cython_version
ModuleNotFoundError: No module named 'Cython'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 238, in <module>
main()
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 234, in main
find_process_files(root_dir)
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 225, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 191, in process
processor_function(fromfile, tofile)
File "/tmp/pip-build-0xuwkh30/numpy/tools/cythonize.py", line 66, in process_pyx
raise OSError('Cython needs to be installed in Python as a module')
OSError: Cython needs to be installed in Python as a module
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-0xuwkh30/numpy/setup.py", line 488, in <module>
setup_package()
File "/tmp/pip-build-0xuwkh30/numpy/setup.py", line 469, in setup_package
generate_cython()
File "/tmp/pip-build-0xuwkh30/numpy/setup.py", line 275, in generate_cython
raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-0xuwkh30/numpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-ijydcqk6-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-0xuwkh30/numpy/
how to solve?
Hi, it seems you should install cython first
pip3 install Cython
OK, pip wheels for PyTorch v1.4.0 have been posted here. These were built with support for OpenBLAS, and USE_DISTRIBUTED with OpenMPI backend.
Note that as per the PyTorch Release Notes, Python 2 support is deprecated and PyTorch v1.4.0 is the last version to support Python 2.
Hey Dustin,
is the new commands (openblas-base + Cython) solve the LAPACK issue?
P.S.
i think the whl is corrupt,
the size is only 14Kb
@danie62192
The whl is not corrupted. The file is on nvidia’s box account and you need a box account to obtain it. Note that I signed up for the free trial and cannot download the file anyway.
@dustin_nv
Is it possible to upload the wheel somewhere else to provide access to it. Thanks!
Hi
I think V 1.4.0 “https://nvidia.box.com/shared/static/ncgzus5o23uck9i5oth2n8n06k340l6k.whl -O torch-1.4.0-cp36-cp36m-linux_aarch64.whl” may be corrupt? When I run pip3 install numpy torch-1.4.0-cp36-cp36m-linux_aarch64.whl it shows up in linux as an HTML file and running pip3 install throws a bad zipfile error. If I try the same for version 1.3.0 everything works fine.
Thanks
Anthony
Same problem
pip3 install torch-1.4.0-cp36-cp36m-linux_aarch64.whl
Processing ./torch-1.4.0-cp36-cp36m-linux_aarch64.whl
ERROR: Exception:
Traceback (most recent call last):
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/cli/base_command.py”, line 153, in _main
status = self.run(options, args)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/commands/install.py”, line 382, in run
resolver.resolve(requirement_set)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/legacy_resolve.py”, line 201, in resolve
self._resolve_one(requirement_set, req)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/legacy_resolve.py”, line 365, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/legacy_resolve.py”, line 313, in _get_abstract_dist_for
req, self.session, self.finder, self.require_hashes
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/operations/prepare.py”, line 194, in prepare_linked_requirement
progress_bar=self.progress_bar
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/download.py”, line 452, in unpack_url
unpack_file_url(link, location, download_dir, hashes=hashes)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/download.py”, line 416, in unpack_file_url
unpack_file(from_path, location, content_type)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/unpacking.py”, line 252, in unpack_file
flatten=not filename.endswith(‘.whl’)
File “/usr/local/lib/python3.6/dist-packages/pip/_internal/utils/unpacking.py”, line 114, in unzip_file
zip = zipfile.ZipFile(zipfp, allowZip64=True)
File “/usr/lib/python3.6/zipfile.py”, line 1131, in init
self._RealGetContents()
File “/usr/lib/python3.6/zipfile.py”, line 1198, in _RealGetContents
raise BadZipFile(“File is not a zip file”)
zipfile.BadZipFile: File is not a zip file
Ah, sorry about that - it was a permissions problem. Should be fixed now - let me know if you still have problems downloading those new URLs. There is no login required.
Yes, I tested it with torch.Solve() and it works now.
[quote=“dusty_nv”]
Do we need export NCCL or not?
Thanks Dustin, it works now.
In a nvcr.io/nvidia/l4t-base:r32.3.1 docker image I only had to additionally apt install mpi-default-dev, which I apparently didn’t need for Pytorch 1.3
The Pytorch 1.4 release also allows me to use the latest Pillow version (7.0.0) in stead of having it to pin down to 6.2.1 like for the previous Pytorch releases. Nice.
[quote=“”]
Previous builds didn’t work with NCCL, I dont think it is provided for Jetson.
MPI seems to be installed already on the default JetPack image, but as you pointed out maybe it isn’t in the Docker image. I will have to make a note of that, thanks.
Hey Dustin,
After successful installing pytorch , and was able to run some “test” script, to check that everything working
Im currently facing problem with Magma libraries.
=> eval
=> loaded checkpoint (epoch 0, t 247000)
Traceback (most recent call last):
File “run.py”, line 264, in
main()
File “run.py”, line 260, in main
restart=args.restart
File “/home/xavier/deblurring/utils.py”, line 283, in run
self._eval()
File “/home/xavier/deblurring/utils.py”, line 224, in _eval
loss, images, to_print = self._eval_step(i, x, y)
File “run.py”, line 218, in _eval_step
burst, _, pred = model(x, force_fm=force_fm)
File “/home/xavier/.local/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 532, in call
result = self.forward(*input, **kwargs)
File “/home/xavier/deblurring/bpn.py”, line 237, in forward
registered_burst = self._register_burst(burst, features)
File “/home/xavier/deblurring/bpn.py”, line 173, in _register_burst
registered_burst = self._stn(unregistered_burst, features)
File “/home/xavier/.local/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 532, in call
result = self.forward(*input, **kwargs)
File “/home/xavier/deblurring/stn.py”, line 106, in forward
return self._move_edges(burst, deltas)
File “/home/xavier/deblurring/stn.py”, line 86, in _move_edges
homographies = self._dlt(edges, edges + deltas)
File “/home/xavier/.local/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 532, in call
result = self.forward(*input, **kwargs)
File “/home/xavier/deblurring/stn.py”, line 56, in forward
params = torch.solve(b, a)[0]
RuntimeError: solve: MAGMA library not found in compilation. Please rebuild with MAGMA.
How can i fix it?
Can you try moving the tensor to CPU before torch.solve(b, a)[0] gets run?
MAGMA library doesn’t appear to be included in the Ubuntu APT repo or pip, so if desired you may need to build MAGMA from source and then re-compile PyTorch.
Doesnt we want it to work on GPU for faster results?
If no,
How i can pass it to the CPU?
Thanks for the help