6

I currently have Raspbian OS

and I tried:

sudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose

which worked fine but installed an old version (0.10)

And:

sudo apt-get install libatlas-base-dev gfortran python-pip

sudo pip install scipy

which after more than an hour working ended up with some errors.

What did you do to install Scipy 0.12 on your Raspberry?

10

TL;DR: skip to the last paragraph to know how to increase swap space and avoid memory clogging. Also, don't use pip for numpy and scipy.

To get the latest version of scipy on the raspberry Pi you need to build from source. The main reason is that scipy relies on compiled C and fortran libraries, that need to be compiled on the same architecture. Usually pip install would fetch prebuilt packages for you, but Raspberry Pi's ARM architecture is not really supported.

If this was all you needed it would be relatively easy (main instructions can be found here and are summarized below), but there is an additional problem with the raspberry pi that I realized only after a week of research and trial and error (skip at the end if you just need the fix), which is due, I believe, to the small amount of memory available.

I'll just try and summarize the whole process (so that people don't have to jump around the internet like I did).

Requirements

There are a few requirements for building scipy. As far as python packages go, you'll need numpy, cython, setuptools, and (if you want to build the documentation) Sphinx. You should try and use the version of these packages that comes with Raspbian (in packages such as python3-numpy), but they might need to be built separately (OT).

As far as system requirements go, you'll need a few packages that can be installed with apt-get, namely:

  • A BLAS/LAPACK math library with development headers, e.g. libopenblas-base and libopenblas-dev;
  • python-dev;
  • C and Fortran compilers, gcc and gfortran;

Finally you need the source code, that you can download from here (Scipy 1.0.0 is the latest stable version as I am writing). Then it's just a matter of tar -xzvf scipy-v1.0.0.tar.gz cd scipy

Compiling the source

At this point, if you start the build process, it will seem to go fine, but it will hang after a few minutes.

Adding bigger swap space

This is due to the compiling script occupying the totality of both RAM and swap memory by spawning multiple processes (and replicating memory by consequence). The problem is that in the Raspberry Pi the swap space is particularly small (only 100MB I think), while the norm would be to have it the same size of your RAM.

As explained here and here, swap space can be increased typing the following:

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
sudo /sbin/mkswap /var/swap.1
sudo chmod 600 /var/swap.1
sudo /sbin/swapon /var/swap.1

which will give you 1GB of swap space.

Then one can finally build and install with

python3 setup.py build

python3 setup.py install --user

(you can drop the --user flag if you want to install it system-wide, but you'll need root privilege).

Finally, one remove the extra swap and restore the default:

sudo swapoff /var/swap.1
sudo rm /var/swap.1
  • 1
    I have been solving few days this memory issue with raspberry. Men you saved my life. – Taras Vaskiv Nov 15 '18 at 19:26
  • 1
    @zurfyx IMHO this post should be marked as the answer – jlandercy Dec 20 '18 at 16:12
  • Thanks @jlandercy, I was hoping for that but I'm afraid the OP has abandoned the platform. – teoguso Jan 15 at 11:36
3

Debian, as goldilocks pointed out, doesn't always ship with the latest versions of packages. Sometimes it ships with really ancient versions, like its Netpbm distribution, which is over 12 years out of date.

There are two ways of dealing with this:

  1. Live with it: are you sure you really have to have the latest version? If your code requires functions or bugfixes from the latest version, then this isn't an option. Debian packages are usually pretty stable, so if you can make your code work with it, you won't have to worry about maintaining your own upgrades. Here are the SciPy 0.10.0 Release Notes — would these features work?

  2. Build from source: the easiest way to get the source dependencies for the current release is to use apt-get build-dep — but it needs a source line in your /etc/apt/sources.list:

    • To /etc/apt/sources.list, add the line: deb-src http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi (basically, the same as the deb line, but with deb-src. If you don't, apt will complain.)
    • Now issue sudo apt-get build-dep python-scipy ; this returns a large list of packages, many more than your post included. Your error message may have been about missing dependencies.
    • Try building scipy again.

Note that there may still be dependency issues, as the current version (0.14 or so) may require newer dependent packages than Raspbian knows about. If you can, try to use a tool like checkinstall that will tell Raspbian that you've installed a newer version with special dependencies, so that it shouldn't get stomped all over in routine upgrades. This process is a bit fiddly; some pointers on how to do it are here: How to build a Debian/Ubuntu package from source?

2

I'm not a python user, but the (probable) reason for the old version is that it's debian policy to take a sometimes excessive amount of time to update versions, plus the package may be obscure (hence updated even less), plus raspbian itself is pretty obscure in relation to debian (hence the packages there are updated even less). Ie, it's a consequence of available resources (the people who maintain the packages, who are usually not the upstream developers).

after more than an hour working ended up with some errors

Presumably this is an issue with compiling one or more of the modules, although you did not actually say what the error was. Compiling python modules should mostly work on the pi, I think, but keep in mind it is an unusual architecture for such, with libraries compiled specifically for it. Unfortunately, this brings the risk of previously unnoticed bugs in some part or another surfacing (if you want to find bugs in a compiled language -- parts of some python modules are in C -- try compiling the code on a few different platforms, lol...). Locating the culprit would take a bit more work and attending to the specific errors.

2

Install using a wheels for pi. It may not be the very latest scipy but it's quite up to date.

  • update pip: pip install pip --upgrade (or pip3, adapt to your python version)
  • go to https://www.piwheels.org/simple/scipy/
  • select the scipy version and target platform (ex. armv6l for pi zero and earlier pi; armv7l for 3)
    • example: scipy-1.2.1-cp35-cp35m-linux_armv7l.whl on a 3 model B
  • download the .whl file
    • using for example: wget https://www.piwheels.org/simple/scipy/scipy-1.2.1-cp35-cp35m-linux_armv7l.whl#sha256=270be300233af556e6ee3f55a0ae237df0cb65ac85d47559010d7a9071f2e878
  • go to where the .whl file was downloaded and: pip install the-scipy-file.whl

Notes:

  • this can be done in a virtualenvironment (just tested)
  • this is way faster than any other method I've seen or tried.
1

1) Increase swap to 1GB

sudo dphys-swapfile swapoff
sudo vim /etc/dphys-swapfile

CONF_SWAPSIZE=1024

sudo dphys-swapfile swapon

2) Reboot to apply swap changes

sudo reboot

3) Than build, here is my Dockerfile

FROM python:3-alpine
MAINTAINER codertarasvaskiv

RUN apk add --no-cache g++ gfortran python3-dev openblas-dev bash
RUN pip install --no-cache-dir numpy
RUN pip install --no-cache-dir scipy
RUN pip install --no-cache-dir <my custom package>

ENTRYPOINT ["bash"]

So it takes 10 mins to build numpy, than almoust 3 hours to build scipy in container.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.