Cannot install scipy

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello, I cannot install scipy on Jetson Nano successfully.
If I use
~$ sudo pip install scipy
the error is

Collecting scipy
  Downloading scipy-1.4.1.tar.gz (24.6 MB)
     |████████████████████████████████| 24.6 MB 12.2 MB/s 
  Installing build dependencies ... /done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp02ghnnwh
         cwd: /tmp/pip-install-398k0h4_/scipy
    Complete output (140 lines):
    lapack_opt_info:
    lapack_mkl_info:
    customize UnixCCompiler
      libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/aarch64-linux-gnu']
      NOT AVAILABLE
    
    openblas_lapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/aarch64-linux-gnu']
      NOT AVAILABLE
    
    openblas_clapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/aarch64-linux-gnu']
      NOT AVAILABLE
    
    atlas_3_10_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/aarch64-linux-gnu
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib/aarch64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
      NOT AVAILABLE
    
    atlas_3_10_info:
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/aarch64-linux-gnu
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib/aarch64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_3_10_info'>
      NOT AVAILABLE
    
    atlas_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/aarch64-linux-gnu
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/aarch64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_threads_info'>
      NOT AVAILABLE
    
    atlas_info:
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/aarch64-linux-gnu
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib/aarch64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE
    
    accelerate_info:
      NOT AVAILABLE
    
    lapack_info:
    customize UnixCCompiler
      libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/aarch64-linux-gnu']
      NOT AVAILABLE
    
    lapack_src_info:
      NOT AVAILABLE
    
      NOT AVAILABLE
    
    setup.py:420: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-m2l6caf7'), proceeding with generating Cython sources and expanding templates
      ' '.join(sys.argv[1:])))
    Running from scipy source directory.
    /usr/local/lib/python3.6/dist-packages/numpy/distutils/system_info.py:636: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()
    /usr/local/lib/python3.6/dist-packages/numpy/distutils/system_info.py:636: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()
    /usr/local/lib/python3.6/dist-packages/numpy/distutils/system_info.py:636: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
        main()
      File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py", line 110, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/usr/local/lib/python3.6/dist-packages/setuptools/build_meta.py", line 156, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/usr/local/lib/python3.6/dist-packages/setuptools/build_meta.py", line 237, in run_setup
        self).run_setup(setup_script=setup_script)
      File "/usr/local/lib/python3.6/dist-packages/setuptools/build_meta.py", line 142, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 540, in <module>
        setup_package()
      File "setup.py", line 536, in setup_package
        setup(**metadata)
      File "/usr/local/lib/python3.6/dist-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "setup.py", line 435, in configuration
        raise NotFoundError(msg)
    numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp02ghnnwh Check the logs for full command output.

if I use
~$ sudo apt-get install python3-scipy
the error is

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python3-scipy : Depends: python3 (< 3.6) but 3.6.7-1~18.04 is to be installed
E: Unable to correct problems, you have held broken packages.
2

My Jetson Nano have 2 versions of Python,

Python 2.7.17

Python 3.6.9

3

I tried installing scipy to get it to work with segmentation-models (a python3 segmentation library that uses Keras). But, it would not work no matter what I tried.

Absolutely the only thing that worked was compiling it manually following these instructions.

https://devtalk.nvidia.com/default/topic/1066391/jetson-nano/keras-and-scipy-throwing-weird-errors-while-installing-on-jetson-nano-no-links-could-help-me-/post/5421802/#5421802

This worked on a Jetson Nano/Xavier with Jetpack 4.3 installed, and the username of nvidia.

4

“sudo pip install” can break your system and is not recommended unless you have a very good reason.

Please see discussion here for proper installation procedures. Again, ‘sudo pip install’ is explicitly not recommended by the scipy documentation, and it’s general good advice.

You should only need to do:

sudo <b>apt</b> install <b>python3-scipy</b>
5

Thanks, but that does not work.
Still showed the following error:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python3-scipy : Depends: python3 (< 3.6) but 3.6.7-1~18.04 is to be installed
E: Unable to correct problems, you have held broken packages.
6

scipy==1.3.3 will work.

7

Thank you, but unfortunately the version 1.3.3 cannot be installed, either, with the same error code in the first blank.

8

Can you check your pip version?

pip -V
pip 20.0.2

As far as I have tried, it looks like this:

pip 20.0.2 + scipy 1.3.3 success
pip 20.0.2 + scipy 1.4.0 fail
pip 20.0.2 + scipy 1.4.1 fail
pip 19.0 + scipy 1.3.3 fail
pip 19.0 + scipy 1.4.0 fail
pip 19.0 + scipy 1.4.1 fail
9

Your apt package configuration seems to have problems. Try:

sudo apt install python3
sudo apt --fix-broken install

This is what should have happened:

$ sudo apt install python3-scipy
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  python-scipy-doc
The following NEW packages will be installed:
  python3-scipy
0 upgraded, 1 newly installed, 0 to remove and 3 not upgraded.
Need to get 8,622 kB of archives.
After this operation, 36.4 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic/universe arm64 pyt
hon3-scipy arm64 0.19.1-2ubuntu1 [8,622 kB]
Fetched 8,622 kB in 2s (4,651 kB/s)
debconf: delaying package configuration, since apt-utils is not inst
alled
Selecting previously unselected package python3-scipy.
(Reading database ... 141905 files and directories currently install
ed.)
Preparing to unpack .../python3-scipy_0.19.1-2ubuntu1_arm64.deb ...
Unpacking python3-scipy (0.19.1-2ubuntu1) ...
Setting up python3-scipy (0.19.1-2ubuntu1) ...

If it doesn’t work. I would reflash since your system seems to be in an odd state.

10

Sometimes the Python modules in apt can be a bit older than those in pip - but yes the apt ones should install, so your system might need refreshed.

Afterwards, to install scipy with pip, first install ‘sudo apt-get install libopenblas-dev’ (or libatlas-base-dev, see this post)
That should clear up the initial errors from your first post.

11

A full list of dependencies for building from source (installing with pip) on Ubuntu at listed in the documentation (probably also want python3-dev).

As dusty_nv mentions, the version included with apt is going to usually always be older, but it avoids having to go through a lengthy build process.

12

In my experience the version with apt installs just fine, but is so old that everything I use that uses scipy ends up trying to install it via pip.

Where they have something like scipy > 1.0.0 as a requirement.

I believe most of my own issues with installing scipy via pip were being caused by using a pip version too old. I’m currently testing that theory out. I find it hard to believe I forgot such a basic step, but who knows. I seem to remember reading about a bunch of bugs in the latest version 20, and I may have simply not tried it.

13

I can confirm that “pip3 install scipy” fails when the version it downloads is 1.4.1 using pip version 20.0.2

On my Jetson Nano it fails with “Could not build wheels for scipy which use PEP 517 and cannot be installed directly”.

Update: I can get scipy 1.4.1 to install with pip3 if I first install pybind11 using “pip3 install pybind11”

14

That happens. If you have to build, however, usually the dependencies for major distros like Ubuntu are listed in the documentation. If they aren’t, I laid out a trial and error procedure in this post you can use to build almost anything.

Re: upgrading pip. Pip has some very specific instructions not to upgrade it on distros where it’s included (also listed in that other thread). It’s fine in a venv, but if you try to install it systemwide (with sudo pip, for example) it absolutely will break things. There is a very specific warning in the docs not to do it.

15

The hard part with scipy is figuring out exactly what needs to be installed. When I have a chance I’m going to test what the bare minimum is for scipy version 1.3.3 or greater to install on a fresh Jetpack 4.3 machine.

So far I have:

Update pip using “python3 -m pip install --upgrade pip”

sudo apt-get install gfortran python3-dev libopenblas-dev liblapack-dev cython

Then do “pip3 install pybind11” due to the dependency of version 1.4.1 on it.

After all those are installed then do “pip3 install scipy”

16

For Ubuntu, the system build dependencies to install are listed in the documentation. That, optionally substituting python3-dev for python-dev should be all you need.

So if you do choose to build from source because you need a new version, these two should be the only two commands necessary:

sudo apt-get install gcc gfortran python-dev python3-dev libopenblas-dev liblapack-dev cython cython3
pip3 (or pip for python 2) install --upgrade scipy

–upgrade since I already have a scipy installed in /usr/lib/…

Some of the apt dependencies listed above are duplicates that can be removed if you care about size, like if you’re making a Docker image. You could also do the pip install in virtual env if you wanted.

I am starting that now with the default pip on my Xavier (which has the same rootfs). I will update if it builds or fails.

Edit: it failed with both the default pip and an upgraded one.

error: Command "aarch64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPOCKETFFT_PTHREADS -I/home/user/.local/include/python3.6m -I/usr/local/include/python3.6 -I/home/user/.local/lib/python3.6/site-packages/numpy/core/include -I/usr/include/python3.6m -c scipy/fft/_pocketfft/pypocketfft.cxx -o build/temp.linux-aarch64-3.6/scipy/fft/_pocketfft/pypocketfft.o -MMD -MF build/temp.linux-aarch64-3.6/scipy/fft/_pocketfft/pypocketfft.o.d -std=c++14 -fvisibility=hidden" failed with exit status 1

I am going to try a “pip3 install pybind11” and update as suggested by S4WRXTTCS. It seems to already be installed. If it suceeds, the setup.py for scipy should probably be updated.

Edit, it works:

$ pip3 install pybind11
Collecting pybind11
  Using cached https://files.pythonhosted.org/packages/4b/4d/ae1c4d8e8b139afa9682054dd42df3b0e3b5c1731287933021b9fd7e9cc4/pybind11-2.4.3-py2.py3-none-any.whl
Installing collected packages: pybind11
Successfully installed pybind11-2.4.3
 $ pip3 install --no-cache --upgrade scipy
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/04/ab/e2eb3e3f90b9363040a3d885ccc5c79fe20c5b8a3caa8fe3bf47ff653260/scipy-1.4.1.tar.gz (24.6MB)
    100% |████████████████████████████████| 24.6MB 49.8MB/s 
Installing collected packages: scipy
  Running setup.py install for scipy ... done
Successfully installed scipy-1.4.1
17

Nice! After I did “pip3 install pybind11” in my environment, “pip3 install --upgrade scipy==1.4.1” successed.
Of course, other libraries (python3-dev, libopenblas-deb etc.) are installed.

18

Same here. looks like it didn’t even need to build. I had the apt version of pybind11 installed, but that wasn’t working. It does look like the scipy setup may need to be adjusted so it requires the updated version of pybind11 it depends on.