知识点:
- Beautiful Soup是一个可以从HTML或XML文件中提取数据的Python库
- CSS选择器
步骤:
- 环境要安装好BeautifulSoup,具体可以看官网文档
- 进入京东网,输入要查询的商品,向服务器发送网页请求。这里用“笔记本”
- 检查网页源码,利用CSS选择器复制数据
具体代码:
import numpy as np from urllib.request import urlopen from urllib.parse import quote from bs4 import BeautifulSoup def get_product(keyword): url = 'https://search.jd.com/Search?keyword='+ quote(keyword) + '&enc=utf-8' html = urlopen(url).read().decode('utf-8') soup = BeautifulSoup(html,'lxml') items = soup.select('li.gl-item') for i in range(1,len(items)): title = soup.select('div.p-img > a')[i]['title'] print('title: ', title) url = soup.select('div.p-img > a')[i]['href'].split('//')[1] print('url: ', url) try: img = soup.select('div.p-img > a > img')[i]['source-data-lazy-img'].split('//')[1] print('img: ', img) except Exception as e: img = soup.select('div.p-img > a > img')[i]['source-data-lazy-img'].split('//')[1] print('img: ', img) price = soup.select('div.p-price > strong > i')[i].get_text() print('price: ', price) print('===========================') if __name__ == '__main__': get_product('笔记本')
输出: