1.环境
本文介绍了在Centos系统下安装Scrapy,因Scrapy是Python针对爬虫开发的框架,所以在安装前因先安装Python,Centos下安装Python3.6.2,按照https://mp.csdn.net/postedit/80076166即可。
2.安装Scrapy
#yum -y update //更新yum
#yum install gcc libffi-devel python-devel openssl-devel //安装gcc及扩展包
#yum groupinstall -y development //安装开发工具包
#yum install libxslt-devel //安装libxslt-devel支持xml
#yum -y install python-pip //安装pip (pip是一个Python包管理工具,主要是用于安装PyPI上的软件包,可以替代easy-install工具)
No package python-pip available该操作对于最小系统版,有可能出现如下错误:
No package python-pip available. Error: Nothing to do
解决办法:
#yun -y install epe;-release
再执行
#yum -y install python-pip
安装完成后清理
#yum clean all
#pip install scrapy //安装Scrapy
3.检查Scrapy安装结果
#scrapy
Scrapy 1.5.0 - no active project
Usage:
scrapy <command> [options] [args]
Available commands:
bench Run quick benchmark test
fetch Fetch a URL using the Scrapy downloader
genspider Generate new spider using pre-defined templates
runspider Run a self-contained spider (without creating a project)
settings Get settings values
shell Interactive scraping console
startproject Create new project
version Print Scrapy version
view Open URL in browser, as seen by Scrapy
[ more ] More commands available when run from project directory
Use "scrapy <command> -h" to see more info about a command
则表示安装scrapy1.5.0成功。