系统:deepin 15.5
python版本:python3.5
爬虫开发常用库的安装
pip3 install requests selenium lxml beautifulsoup4 pyquery pymysql pymongo redis flask django jupyter
安装chromedriver以及phantomjs
sudo apt-get install xvfb
sudo apt-get install unzip
chromedriver安装
chrome版本:62.0.3202.62
chromedriver下载地址:http://chromedriver.storage.googleapis.com/index.html 我下载的是2.34版
(chromedriver与chrome版本对应:http://chromedriver.storage.googleapis.com/2.33/notes.txt )
下载完成后终端进入下载的目录中,
unzip chromedriver_linux64.zip
chmod +x chromedriver
sudo mv -f chromedriver /usr/local/share/chromedriver
sudo ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver
sudo ln -s /usr/local/share/chromedriver /usr/bin/chromedriver
安装完后简单测试下,
打开终端输入python,然后依次输入:
from selenium import webdriver
driver = webdriver.Chrome()
selenium和chromedriver安装正确的话,会自行打开chrome,如下图所示
再尝试输入
driver.get('https://www.python.org')
会自动打开python官网
phantomjs安装
官网下载http://phantomjs.org/download.html
tar -xvf phantomjs-2.1.1-linux-x86_64.tar.bz2
sudo mv phantomjs-2.1.1-linux-x86_64 /usr/local/share/phantomjs
sudo ln -s /usr/local/share/phantomjs/bin/phantomjs /usr/local/bin/phantomjs