sudo apt-get install python-scipy
This successfully installed scipy on an arm64 Ubuntu 16.04 (on NVIDIA's Jetson TX1 with Jetpack 2.3).
The additional packages include lapack and blas packages, which the error alluded to.
The following additional packages will be installed: libblas-common libblas3 libgfortran3 liblapack3 python-decorator python-imaging python-numpy python-pil
And then to install keras 1.0.6:
pip install --user keras==2.1.5