import urllib.request
向指定网址的服务器发起请求
response = urllib.request.urlopen(‘http://www.baidu.com‘)
data = response.read() #data是byte类型的文件
把百度的原码写进指定路径下的file文件中
with open(r’D:\file.html’,’wb’) as f:
f.write(data)
”’
data = response.readline() #读取一行
data = response.readlines() #读取多行
破解别人的反爬虫技术
import urllib.request
url = ‘https://www.qiushibaike.com’
防止别人运用反爬虫技术。下面这行代码在F12上查看。模拟请求头
headers = {‘User-Agent’:’Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36’}
设置一个请求体
req = urllib.request.Request(url,headers=headers)
发起请求
response = urllib.request.urlopen(req)
data = response.read()
with open(r’D:\file.html’,’wb’) as f:
f.write(data)
”’