爬虫05-ajax

获取ajax数据

获取上海证券交易所的数据 url = 'http://www.sse.com.cn/market/price/report/'

打开页面,检查页面源代码,发现页面数据时ajax加载的,分析请求头,获取数据接口,

请求头的数据:

callback: jQuery111208931742373497886_1536390774668
select: code,name,open,high,low,last,prev_close,chg_rate,volume,amount,tradephase,change,amp_rate
order: 
begin: 75
end: 100
_: 1536390774672

 最后程序如下:

from urllib import request, parse
from chardet import detect
# 获取的ajax数据接口 反序列之后得到如下的url
url = parse.unquote('http://yunhq.sse.com.cn:32041/v1/sh1/list/exchange/equity?callback=jQuery111208931742373497886_1536390774668&select=code%2Cname%2Copen%2Chigh%2Clow%2Clast%2Cprev_close%2Cchg_rate%2Cvolume%2Camount%2Ctradephase%2Cchange%2Camp_rate&order=&begin=75&end=100&_=1536390774671')
print(url)
jsonp = request.urlopen(url)
html = jsonp.read()
charset = detect(html)['encoding']
print(charset)
data = html.decode(charset)
print(data)

运行结果如下:

http://yunhq.sse.com.cn:32041/v1/sh1/list/exchange/equity?callback=jQuery111208931742373497886_1536390774668&select=code,name,open,high,low,last,prev_close,chg_rate,volume,amount,tradephase,change,amp_rate&order=&begin=75&end=100&_=1536390774671
GB2312
jQuery111208931742373497886_1536390774668({"date":20180907,"time":151502,"total":1481,"begin":75,"end":100,"list":[["600096","云天化  ",5.05,5.14,5.03,5.08,5.05,0.59,2952818,15025295,"E111",0.03,2.18],["600097","开创国际",10.68,10.80,10.65,10.75,10.69,0.56,980523,10506732,"E111",0.06,1.40],["600098","广州发展",6.11,6.16,6.11,6.15,6.11,0.65,1874980,11515770,"E111",0.04,0.82],["600099","林海股份",6.52,6.64,6.50,6.64,6.56,1.22,1738405,11429608,"E111",0.08,2.13],["600100","同方股份",9.55,9.69,9.40,9.51,9.53,-0.21,6157698,58682937,"E111",-0.02,3.04],["600101","明星电力",6.87,6.93,6.85,6.87,6.87,0.00,735700,5063001,"E111",0.00,1.16],["600103","青山纸业",2.94,2.97,2.92,2.95,2.94,0.34,2956291,8709167,"E111",0.01,1.70],["600104","上汽集团",28.43,28.85,28.24,28.41,28.28,0.46,7786537,222294890,"E111",0.13,2.16],["600105","永鼎股份",3.77,3.81,3.66,3.72,3.78,-1.59,7400010,27615993,"E111",-0.06,3.97],["600106","重庆路桥",3.12,3.31,3.09,3.22,3.10,3.87,19178331,61481453,"E111",0.12,7.10],["600107","美尔雅  ",5.55,5.66,5.52,5.66,5.51,2.72,3166557,17708981,"E111",0.15,2.54],["600108","亚盛集团",3.08,3.12,3.03,3.04,3.09,-1.62,11900903,36608414,"E111",-0.05,2.91],["600109","国金证券",6.73,6.85,6.71,6.74,6.74,0.00,7898187,53473907,"E111",0.00,2.08],["600110","诺德股份",4.48,4.62,4.48,4.57,4.58,-0.22,9446390,43070296,"E111",-0.01,3.06],["600111","北方稀土",10.06,10.16,9.92,9.98,10.05,-0.70,12593092,126365873,"E111",-0.07,2.39],["600112","天成控股",3.42,3.59,3.42,3.48,3.46,0.58,20788666,73041336,"E111",0.02,4.91],["600113","浙江东日",7.89,7.96,7.79,7.87,7.83,0.51,2334842,18403738,"E111",0.04,2.17],["600114","东睦股份",7.86,8.04,7.77,7.88,7.94,-0.76,2337331,18463253,"E111",-0.06,3.40],["600115","东方航空",5.43,5.56,5.42,5.55,5.40,2.78,35526472,195839643,"E111",0.15,2.59],["600116","三峡水利",6.72,6.78,6.63,6.68,6.81,-1.91,4504530,30174753,"E111",-0.13,2.20],["600117","西宁特钢",4.54,4.64,4.51,4.56,4.50,1.33,11271871,51411714,"E111",0.06,2.89],["600118","中国卫星",19.21,19.46,18.72,19.18,19.31,-0.67,13617051,260530797,"E111",-0.13,3.83],["600119","长江投资",7.60,7.83,7.56,7.73,7.63,1.31,3167963,24360948,"E111",0.10,3.54],["600120","浙江东方",12.23,12.38,12.13,12.20,12.20,0.00,1845758,22645501,"E111",0.00,2.05],["600121","郑州煤电",3.47,3.55,3.46,3.50,3.45,1.45,5545350,19474210,"E111",0.05,2.61]]})

改变start和end可以改变url获取的数据

猜你喜欢

转载自blog.csdn.net/qwerLoL123456/article/details/82496474