最近618大法的n1打折准备剁一波手,不过先分析一波用户评论在剁手(emmmm科学剁手)
----------------------------------------------------------------------------------------------------------
废话结束准备开始
1:在商品页面f12或右键审查元素,点击network 输入存用户评论的js文件productPageComments 刷新
点击跳转到第二页评论的按钮会在刷出一个js文件
双击这个文件打开网页复制网页url到记事本以便找出规律
叮!规律识破!
2:定个小目标先爬它个15页
先存到本地等以后有时间在洗数据
import requests import urllib3 import json import urllib import urllib.request from bs4 import BeautifulSoup for i in range(1, 15): url1 = 'https://sclub.jd.com/comment/productPageComments.action?callback=fetchJSON_comment98vv4403&productId=3487485&score=3&sortType=5&page=' url2 = str(i) uel3 = '&pageSize=10&isShadowSku=0&rid=0&fold=1' finalurl = url1+url2+uel3 xba = requests.get(finalurl) # print(xba.text) for i in range(1, 15): u1="E:\\pachong\\" u2=str(i) u3=".txt" finalu=u1+u2+u3 file = open(finalu, "w") file.write(xba.text) file.close() st = xba.text print('finish')