1.参考中文资料:http://www.scrapyd.cn/doc/140.html
2.下载:
pip install Scrapy
3.终端里创建项目HelloWorld
scrapy startproject HelloWorld
4.pycharm打开此项目,在Terminal下创建:
scrapy genspider lab lab.scrapyd.cn
5.会出来一个lab.py文件,打开:
import scrapy
class HelloWorld(scrapy.Spider): # 需要继承scrapy.Spider类
name = "lab" # 定义蜘蛛名
allowed_domains = ['lab.scrapyd.cn']
start_urls = ['http://lab.scrapyd.cn/']
def parse(self, response):
print(response)
print(type(response))
# print(response.text)
result = response.xpath('//div[contains(@class,"quote post")]/span')
for item in result:
print(item.get())
6.Terminal下运行:
scrapy crawl lab
完成。