利用CSS选择器和BeautifulSoup抓去京东网商品信息

知识点:

  1. Beautiful Soup是一个可以从HTML或XML文件中提取数据的Python库
  2. CSS选择器

步骤:

  1. 环境要安装好BeautifulSoup,具体可以看官网文档
  2. 进入京东网,输入要查询的商品,向服务器发送网页请求。这里用“笔记本”
  3. 检查网页源码,利用CSS选择器复制数据

具体代码:

import numpy as np
from urllib.request import urlopen
from urllib.parse import quote
from bs4 import BeautifulSoup

def get_product(keyword):
    url = 'https://search.jd.com/Search?keyword='+ quote(keyword) + '&enc=utf-8'
    html = urlopen(url).read().decode('utf-8')
    soup = BeautifulSoup(html,'lxml')
    items = soup.select('li.gl-item')
    for i in range(1,len(items)):
        title = soup.select('div.p-img > a')[i]['title']
        print('title: ', title)
        url = soup.select('div.p-img > a')[i]['href'].split('//')[1]
        print('url: ', url)
        try:
            img = soup.select('div.p-img > a > img')[i]['source-data-lazy-img'].split('//')[1]
            print('img: ', img)
        except Exception as e:
            img = soup.select('div.p-img > a > img')[i]['source-data-lazy-img'].split('//')[1]
            print('img: ', img)
        price = soup.select('div.p-price > strong > i')[i].get_text()
        print('price: ', price)
        print('===========================')


if __name__ == '__main__':
    get_product('笔记本')

输出:


猜你喜欢

转载自blog.csdn.net/qq_21325977/article/details/80782772