版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/qq_26437925/article/details/80689324
环境
* CPU 2个核
* python 2.7.10
CPU密集型
实践代码
# -*- coding:utf-8 -*-
import time
from threading import Thread
from multiprocessing import Process
def count():
'''
# CPU密集操作
'''
c = 0
x = 1
y = 2
while c < 100000:
c += 1
x += x
y += y
def countn(n):
for x in range(n):
count()
def multi_thread(n):
counts = []
t = time.time()
tcount = 8
tmp = tcount / n
for i in range(n):
thread = Thread(target=countn, args=(tmp,))
counts.append(thread)
thread.start()
for i in range(n):
counts[i].join()
print("thread count=%d" % n, time.time() - t)
def multi_process(n):
counts = []
t = time.time()
tcount = 8
tmp = tcount / n
for i in range(n):
process = Process(target=countn, args=(tmp,))
counts.append(process)
process.start()
for i in range(n):
counts[i].join()
print("process count=%d" % n, time.time() - t)
if __name__ == '__main__':
li = [1, 2, 4, 8]
for x in li:
multi_thread(x)
for x in li:
multi_process(x)
运行结果
('thread count=1', 2.6109211444854736)
('thread count=2', 2.642864942550659)
('thread count=4', 2.6333298683166504)
('thread count=8', 2.6398651599884033)
('process count=1', 2.594637155532837)
('process count=2', 1.3601601123809814)
('process count=4', 1.0247399806976318)
('process count=8', 0.9624059200286865)
结论
python中GIL的存在,进程是有单独的GIL的,线程没有。对于CPU密集型的程序,多线程运行起来跟单线程基本没有区别,因为同一时刻,GIL的存在只能使得一个线程能够执行,所以效率基本不会提高,反而可能下降(线程的创建,切换等都要耗时)。
IO密集型
实践代码
# -*- coding:utf-8 -*-
import time
from threading import Thread
from multiprocessing import Process
def io():
'''
# IO密集操作
'''
x = 1
y = 2
time.sleep(1)
x += x
time.sleep(1)
y += y
def countn(n):
for x in range(n):
io()
def multi_thread(n):
counts = []
t = time.time()
tcount = 8
tmp = tcount / n
for i in range(n):
thread = Thread(target=countn, args=(tmp,))
counts.append(thread)
thread.start()
for i in range(n):
counts[i].join()
print("thread count=%d" % n, time.time() - t)
def multi_process(n):
counts = []
t = time.time()
tcount = 8
tmp = tcount / n
for i in range(n):
process = Process(target=countn, args=(tmp,))
counts.append(process)
process.start()
for i in range(n):
counts[i].join()
print("process count=%d" % n, time.time() - t)
if __name__ == '__main__':
li = [1, 2, 4, 8]
for x in li:
multi_thread(x)
for x in li:
multi_process(x)
运行结果
('thread count=1', 16.05018401145935)
('thread count=2', 8.026233196258545)
('thread count=4', 4.011896848678589)
('thread count=8', 2.004067897796631)
('process count=1', 16.032613039016724)
('process count=2', 8.01167893409729)
('process count=4', 4.009658098220825)
('process count=8', 2.012108087539673)
结论
IO密集型操作,如文件处理、网络爬虫等涉及文件读写的操作,其特点是CPU消耗很少,任务的大部分时间都在等待IO操作完成(因为IO的速度远远低于CPU和内存的速度)。
多线程,多进程都能在IO密集型操作下体现优势。因为可以利用IO阻塞等待时的空闲时间去执行其它的线程(或进程)。在某个特定的时刻,可以有多个线程(或)进程运行(例如某些线程阻塞,某个线程执行CPU计算),这显然能提高效率。
参考
CPU-bound(计算密集型) 和I/O bound(I/O密集型).剑西楼[EB/OL]https://blog.csdn.net/q_l_s/article/details/51538039
Python中单线程、多线程和多进程的效率对比实验.饒木陽 [EB/OL]http://python.jobbole.com/86822/