深度学习开源框架caffe使用ubuntu进行安装较为顺畅,但是如果是centos则会有些无伤大雅的小问题,本文进行简单整理和总结,并针对目前版本提供一个可用的一键脚本。由于验证机器无N卡,所以此文未包括CUDA和GPU设定相关部分的小坑。
No.1 现象:yum安装时提示:No package xxx available
比如如下信息
No package leveldb-devel available.
No package hdf5-devel available.
原因
在centos中通过yum进行安装,但是一部分包,诸如hdf5-devel等不在缺省的仓库中,需要通过epel-release进行安装的
对策
执行yum install -y epel-release
No.2 现象:提示Command “python setup.py egg_info” failed with error code 1信息
详细描述
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-t9E3ak/ipython/
原因
python的依赖需要使用scikit-image进行安装
对策
pip install scikit-image
No.3 现象:显示/bin/sh: g++: command not found
原因
未安装c++编译器,由于Caffe需要使用cpp的编译器进行编译,centos下可以考虑使用g++
对策
yum -y install gcc gcc-c++
No.4 现象:不存在hdf5.h的头文件
详细描述
src/caffe/layers/hdf5_data_layer.cpp:13:18: fatal error: hdf5.h: No such file or directory
原因
未正确安装hdf5-devel,可能No package hdf5-devel available的提示被无视了。
对策
对策1: 事前使用yum install -y epel-release之后再使用yum install hdf5-devel即可
对策2: 可以自行下载hdf5的源码(比如:wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.1/src/hdf5-1.10.1.tar.gz),从源码开始编译,此种方式同时需要修改caffe的Makefile.config,确保在编译的时候编译器能找到hdf5设定的prefix的include路径
No.5 现象:提示leveldb/db.h: No such file or directory信息
详细描述
./include/caffe/util/db_leveldb.hpp:7:24: fatal error: leveldb/db.h: No such file or directory
原因
未正确安装leveldb-devel,可能No package hdf5-devel available的提示被无视了。
对策
事前使用yum install -y epel-release之后再使用yum install leveldb-devel即可
No.6 现象:链接阶段找不到cblas和atlas库文件
详细描述
/usr/bin/ld: cannot find -lcblas
/usr/bin/ld: cannot find -latlas
原因
在centos下名字不一样了,自然找不到
对策
对策1:
修改Makefile.config文件(前提是宿主机器的CentOS为64bit):
BLAS_INCLUDE := /usr/include/atlas
BLAS_LIB := /usr/lib64/atlas
修改Makefile
LIBRARIES += cblas atlas
修改为
LIBRARIES += satlas tatlas
对策2:
使用ln -sv将/usr/lib64/atlas下面的lib做成和caffe的makefile链接要求名字一致也能通过,熟悉Unix/Linux下c/C++编程的开发者可能会深谙此道,个人建议使用对策1,所以此处不再赘述
对策3:
给caffe提个issue,改一下caffe的Makefile,目前整个Linux都是一种名称,再判断一点即可。不过这样无伤大雅的小问题,不爱改不改也行,毕竟已经提供了标准的镜像,使用镜像自然会更加简单。
ifeq ($(LINUX), 1)
ifeq ($(BLAS), atlas)
# Linux simply has cblas and atlas
No.7 现象:提示numpy/arrayobject.h: No such file or directory信息
详细描述
[root@caffe caffe]# make pycaffe
CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
python/caffe/_caffe.cpp:10:31: fatal error: numpy/arrayobject.h: No such file or directory
原因
没有正确安装numpy
对策
pip install numpy
No.8 现象:使用时提示ImportError: No module named caffe
详细描述
[root@caffe caffe]# python
Python 2.7.5 (default, Apr 11 2018, 07:36:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named caffe
>>>
原因
未设定PYTHONPATH
对策
[root@caffe caffe]# export PYTHONPATH=`pwd`/python:${PYTHONPATH}
[root@caffe caffe]# python
Python 2.7.5 (default, Apr 11 2018, 07:36:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
No.9 现象:提示ImportError: No module named skimage.io信息
详细描述
[root@caffe caffe]# export PYTHONPATH=`pwd`/python:${PYTHONPATH}
[root@caffe caffe]# python
Python 2.7.5 (default, Apr 11 2018, 07:36:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/root/caffe/python/caffe/__init__.py", line 1, in <module>
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
File "/root/caffe/python/caffe/pycaffe.py", line 15, in <module>
import caffe.io
File "/root/caffe/python/caffe/io.py", line 2, in <module>
import skimage.io
ImportError: No module named skimage.io
>>>
原因
未安装依赖scikit-image
对策
pip install scikit-image
当前相关的依赖被整理到requirements.txt中了,主要对应关系如下:
组件 | 版本要求 |
---|---|
Cython | >=0.19.2 |
numpy | >=1.7.1 |
scipy | >=0.13.2 |
scikit-image | >=0.9.3 |
matplotlib | >=1.3.1 |
ipython | >=3.0.0 |
h5py | >=2.2.0 |
leveldb | >=0.191 |
networkx | >=1.8.1 |
nose | >=1.3.0 |
pandas | >=0.12.0 |
python-dateutil | >=1.4,<2 |
protobuf | >=2.5.0 |
python-gflags | >=2.0 |
pyyaml | >=3.10 |
Pillow | >=2.3.0 |
six | >=1.1.0 |
一次性的安装方法:
[root@caffe caffe]# cd python
[root@caffe python]# ls
CMakeLists.txt caffe classify.py detect.py draw_net.py requirements.txt train.py
[root@caffe python]# for req in $(cat requirements.txt); do pip install $req; done
No.10 现象:Fontconfig warning: ignoring UTF-8: not a valid region tag
原因
locale设定不正确,基本无影响
对策
设定locale.conf,加入如下语句即可
[root@caffe caffe]# cat /etc/locale.conf |grep CTYPE
LC_CTYPE=”en_US.UTF-8”
[root@caffe caffe]#
结果确认
[root@caffe caffe]# python
Python 2.7.5 (default, Apr 11 2018, 07:36:10)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> exit()
[root@caffe caffe]#
使用import,不出现错误,即可认为caffe基本安装成功,后续即可使用caffe进行实验了。
安装步骤整理
Step 1: 使用yum进行所需依赖的安装
yum install -y epel-release
yum install -y protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel
yum install -y gflags-devel glog-devel lmdb-devel
yum install gcc gcc-c++ python-pip
Step 2: 使用PIP安装相关依赖
pip install scikit-image
pip install numpy
Step 3:git clone并进行事前准备
git clone操作
git clone https://github.com/BVLC/caffe.git
安装python依赖
cd caffe/python
for req in $(cat requirements.txt); do pip install $req; done
修改Makefile.confg
cd ..
cp Makefile.config.example Makefile.config
只验证CPU方式:将# CPU_ONLY := 1的#去掉
修改BLAS的配置信息(INCLUDE):
# BLAS_INCLUDE := /path/to/your/blas
====>
BLAS_INCLUDE := /usr/include/atlas
修改BLAS的配置信息(LIB):
# BLAS_LIB := /path/to/your/blas
====>
BLAS_LIB := /usr/lib64/atlas
修改numpy的信息,由于64位的numpy位置在lib64下
/usr/lib/python2.7/dist-packages/numpy/core/include
====>
/usr/lib64/python2.7/site-packages/numpy/core/include
修改Makefile
LIBRARIES += cblas atlas
====>
LIBRARIES += satlas tatlas
设定环境变量
环境变量PYTHONPATH建议设定到用户的profile或者整体机器的/etc/profile中以持久化保存
Step 4: 编译安装
pwd确认当前目录在caffe根目录下
make clean
make all
make test
make runtest
make pycaffe
Step 5: 结果确认
export PYTHONPATH=`pwd`/python:${PYTHONPATH}
python -c “import caffe; exit()”
CentOS下一键安装脚本
事前准备
一台最小化安装的CentOS即可
[root@liumiaocn ~]# uname -a
Linux liumiaocn 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@liumiaocn ~]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
[root@liumiaocn ~]#
git clone easypack
执行日志
[root@liumiaocn ~]# git clone https://github.com/liumiaocn/easypack.git
Cloning into 'easypack'...
remote: Counting objects: 1059, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 1059 (delta 32), reused 6 (delta 3), pack-reused 985
Receiving objects: 100% (1059/1059), 159.38 KiB | 36.00 KiB/s, done.
Resolving deltas: 100% (467/467), done.
[root@liumiaocn ~]#
安装
cd easypack/machinelearning/caffe
sh installcaffe4centos.sh
执行日志
[root@liumiaocn ~]# cd easypack/machinelearning/caffe
[root@liumiaocn caffe]# sh installcaffe4centos.sh
Thu May 17 20:21:57 EDT 2018
## Step 1: install yum deps.
...省略
Complete!
## Step 2: install python deps.
Collecting scikit-image
...省略
## Step 3: git clone & config.
Cloning into 'caffe'...
remote: Counting objects: 54253, done.
此步git clone可能较慢,也有可能会因为timeout失败导致后续失败,请确认网络状况和日志信息
...省略