scrapy crawl min_spider -o min_spider.json
说明: 我的爬虫名字为min_spider,这条命令会在当前文件夹下生成一个min_spider.json的文件
class MinSpiderSpider(scrapy.Spider):
name = 'min_spider'
allowed_domains = ['baidu.com']
start_urls = ['http://www.baidu.com/']
还支持csv xml pickle marshal等格式 代码都一样
scrapy crawl min_spider -o min_spider.csv
scrapy crawl min_spider -o min_spider.xml
scrapy crawl min_spider -o min_spider.pickle
scrapy crawl min_spider -o min_spider.marshal