asyncio是Python 3.4版本引入的标准库,直接内置了对异步IO的支持。
asyncio的编程模型就是一个消息循环。我们从asyncio模块中直接获取一个EventLoop的引用,然后把需要执行的协程扔到EventLoop中执行,就实现了异步IO。
用asyncio实现Hello world代码如下:
import asyncio
@asyncio.coroutine
def hello():
print('Hello world!')
# 异步调用asyncio.sleep(1)
r = yield from asyncio.sleep(1)
print('Hello Again!!!')
# 获取Eventloop
loop = asyncio.get_event_loop()
# 执行coroutine
loop.run_until_complete(hello())
loop.close()
@asyncio.coroutine把一个generator标记为coroutine类型,然后,我们就把这个coroutine扔到EventLoop中执行。
hello()会首先打印出Hello world!,然后,yield from语法可以让我们方便地调用另一个generator。由于asyncio.sleep()也是一个coroutine,所以线程不会等待asyncio.sleep(),而是直接中断并执行下一个消息循环。当asyncio.sleep()返回时,线程就可以从yield from拿到返回值(此处是None),然后接着执行下一行语句。
把asyncio.sleep(1)看成是一个耗时1秒的IO操作,在此期间,主线程并未等待,而是去执行EventLoop中其他可以执行的coroutine了,因此可以实现并发执行。
我们用Task封装两个coroutine试试:
import threading
import asyncio
@asyncio.coroutine
def hello():
print('Hello world! (%s)' % threading.currentThread())
yield from asyncio.sleep(2)
print('Hello Again!!! (%s)' % threading.currentThread())
loop = asyncio.get_event_loop()
tasks = [hello(), hello()]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
观察执行过程:
Hello world! (<_MainThread(MainThread, started 3232)>)
Hello world! (<_MainThread(MainThread, started 3232)>)
(暂停约2秒)
Hello Again!!! (<_MainThread(MainThread, started 3232)>)
Hello Again!!! (<_MainThread(MainThread, started 3232)>)
由打印的当前线程名称可以看出,两个coroutine是由同一个线程并发执行的。
如果把asyncio.sleep()换成真正的IO操作,则多个coroutine就可以由一个线程并发执行。
我们用asyncio的异步网络连接来获取sina、sohu和163的网站首页:
import asyncio
@asyncio.coroutine
def wget(host):
print('wget %s ...' % host)
connect = asyncio.open_connection(host, 80)
reader, writer = yield from connect
header = 'GET / HTTP/1.0\r\nHost:%s\r\n\r\n' % host
writer.write(header.encode('utf-8'))
yield from writer.drain()
while True:
line = yield from reader.readline()
if line == b'\r\n':
break
print('%s header > %s' % (host, line.decode('utf-8').rstrip()))
# Ignore the body, close the socket
writer.close()
loop = asyncio.get_event_loop()
tasks = [wget(host) for host in ['www.sina.com.cn', 'www.sohu.com', 'www.163.com']]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
执行结果如下:
wget www.sina.com.cn ...
wget www.163.com ...
wget www.sohu.com ...
www.sina.com.cn header > HTTP/1.1 302 Moved Temporarily
www.sina.com.cn header > Server: nginx
www.sina.com.cn header > Date: Tue, 14 Aug 2018 17:42:47 GMT
www.sina.com.cn header > Content-Type: text/html
www.sina.com.cn header > Content-Length: 154
www.sina.com.cn header > Connection: close
www.sina.com.cn header > Location: https://www.sina.com.cn/
www.sina.com.cn header > X-Via-CDN: f=edge,s=cnc.beixian.ha2ts4.214.nb.sinaedge.com,c=124.64.17.205;
www.sina.com.cn header > X-Via-Edge: 1534268567481cd11407cde9d7e7b06c32819
www.163.com header > HTTP/1.0 302 Moved Temporarily
www.163.com header > Server: Cdn Cache Server V2.0
www.163.com header > Date: Tue, 14 Aug 2018 17:42:47 GMT
www.163.com header > Content-Length: 0
www.163.com header > Location: http://www.163.com/special/0077jt/error_isp.html
www.163.com header > Connection: close
www.sohu.com header > HTTP/1.1 200 OK
www.sohu.com header > Content-Type: text/html;charset=UTF-8
www.sohu.com header > Connection: close
www.sohu.com header > Server: nginx
www.sohu.com header > Date: Tue, 14 Aug 2018 17:42:20 GMT
www.sohu.com header > Cache-Control: max-age=60
www.sohu.com header > X-From-Sohu: X-SRC-Cached
www.sohu.com header > Content-Encoding: gzip
www.sohu.com header > FSS-Cache: HIT from 4373726.6995176.5466374
www.sohu.com header > FSS-Proxy: Powered by 2669764.3587278.3762386
可见3个链接由一个线程通过coroutine并发完成。
小结
- asyncio提供了完善的异步IO支持;
- 异步操作需要在coroutine中通过yield from来完成,它能让你在一个生成器函数中调用另一个生成器函数,完成异步IO的操作;
- 多个coroutine可以封装成一组Task后,再使用asyncio.wait()等待所有任务列表里的Futures任务完成。