一:
1、GIL:全局解释器锁
GIL本质就是一把互斥锁,是夹在解释器身上的,同一个进程内的所有线程都需要先抢到GIL锁,才能执行代码!
from threading import Thread,current_thread import time def task(): print('%s is running' %current_thread().name) time.sleep(1) print('%s is ending' %current_thread().name) if __name__ == '__main__': t1 = Thread(target=task) t2 = Thread(target=task) t3 = Thread(target=task) t1.start() t2.start() t3.start()
2、GIL的优缺点:
优点:
保证Cpython解释器内存管理的线程安全
缺点:
同一进程内所有的线程同一时刻只能有一个执行,也就是说Cpython解释器的多线程无法实现并行;
二:Cpython解释器并发效率验证
1、计算密集型应该使用多进程;
1 from multiprocessing import Process 2 from threading import Thread 3 4 import os 5 import time 6 def task1(): 7 res = 0 8 for i in range(100000000): 9 res +=i 10 11 def task2(): 12 res = 0 13 for i in range(100000000): 14 res += i 15 16 def task3(): 17 res = 0 18 for i in range(100000000): 19 res += i 20 21 def task4(): 22 res = 0 23 for i in range(100000000): 24 res += i 25 26 27 28 if __name__ == '__main__': 29 p1 = Process(target=task1) 30 p2 = Process(target=task2) 31 p3 = Process(target=task3) 32 p4 = Process(target=task4) 33 34 35 start = time.time() 36 37 p1.start() 38 p2.start() 39 p3.start() 40 p4.start() 41 p1.join() 42 p2.join() 43 p3.join() 44 p4.join() 45 print('ending') 46 print(time.time()-start)
2、IO密集型应该使用多线程;
1 from threading import Thread 2 import time 3 4 def task1(): 5 time.sleep(3) 6 7 if __name__ == '__main__': 8 t_l = [] 9 start = time.time() 10 for i in range(500): 11 t = Thread(target=task1) 12 t_l.append(t) 13 14 t.start() 15 16 for i in t_l: 17 i.join() 18 19 20 print(time.time() - start)
三:
1、线程互斥锁与GIL对比;
1 from threading import Thread,Lock 2 import time 3 4 mutex=Lock() 5 count=0 6 7 def task(): 8 global count 9 mutex.acquire() 10 temp=count 11 time.sleep(0.1) 12 count=temp+1 13 mutex.release() 14 15 16 17 if __name__ == '__main__': 18 t_l=[] 19 for i in range(2): 20 t=Thread(target=task) 21 t_l.append(t) 22 t.start() 23 for t in t_l: 24 t.join() 25 26 print('主',count)
线程互斥锁是如果线程1抢到了就不会因为遇到IO或者阻塞而释放掉,而GIL会因为线程遇到IO后就释放掉;
四:
1、基于多线程实现并发套字节通信;
1 from socket import * 2 from threading import Thread 3 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor 4 pool = ThreadPoolExecutor(3) 5 6 def communicate(conn,client_addr): 7 while True: 8 try: 9 data = conn.recv(1024) 10 if not data:break 11 conn.send(data.upper()) 12 except ConnectionResetError: 13 break 14 conn.close() 15 16 def server(): 17 server = socket(AF_INET,SOCK_STREAM) 18 server.bind(('127.0.0.1',8080)) 19 server.listen(5) 20 21 while True: 22 conn,client_addr = server.accept() 23 print(client_addr) 24 pool.submit(communicate,conn,client_addr) 25 server.close() 26 27 if __name__ == '__main__': 28 server()
1 from socket import * 2 3 client=socket(AF_INET,SOCK_STREAM) 4 client.connect(('127.0.0.1',8080)) 5 6 while True: 7 msg=input('>>>: ').strip() 8 if not msg:continue 9 client.send(msg.encode('utf-8')) 10 data=client.recv(1024) 11 print(data.decode('utf-8')) 12 13 client.close()
五:
1、进程池vs线程池
为什么要用“池”
1)池子使用来限制并发的任务数目,限制我们的计算机在一个自己可承受的范围内去并发地执行任务;
2)池子内什么时候装进程:并发的任务属于计算密集型;
1 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor 2 import time,os,random 3 4 def task(x): 5 print('%s 接客' %os.getpid()) 6 time.sleep(random.randint(2,5)) 7 return x**2 8 9 if __name__ == '__main__': 10 p=ProcessPoolExecutor() # 默认开启的进程数是cpu的核数 11 12 # alex,武佩奇,杨里,吴晨芋,张三 13 14 for i in range(20): 15 p.submit(task,i)
3)池子内什么时候装线程:并发的任务属于IO密集型;
1 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor 2 import time,os,random 3 4 def task(x): 5 print('%s 接客' %x) 6 time.sleep(random.randint(2,5)) 7 return x**2 8 9 if __name__ == '__main__': 10 p=ThreadPoolExecutor(4) # 默认开启的线程数是cpu的核数*5 11 12 # alex,武佩奇,杨里,吴晨芋,张三 13 14 for i in range(20): 15 p.submit(task,i)
六:
1、阻塞与非阻塞的是程序的两种运行状态:
1)阻塞:遇到IO就发生阻塞,程序一旦遇到阻塞操作就会停在原地,并且立即释放CPU资源;
2)非阻塞(就绪态或运行态):没有遇到IO操作,或者通过某种手段让程序即便是遇到IO操作也不会停在原地执行其他操作,力求尽可能多的占有CPU;
2、同步与异步指的是提交任务的两种方式;
1)同步调用:提交完任务后,就在原地等待,直到任务运行完毕后,拿到任务的返回值,才继续执行下一行代码
1 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor 2 import time,os,random 3 4 def task(x): 5 print('%s 接客' %x) 6 time.sleep(random.randint(1,3)) 7 return x**2 8 9 if __name__ == '__main__': 10 # 同步调用 11 p=ThreadPoolExecutor(4) # 默认开启的线程数是cpu的核数*5 12 13 # alex,武佩奇,杨里,吴晨芋,张三 14 15 for i in range(10): 16 res=p.submit(task,i).result() 17 18 print('主')
2)异步调用:提交完任务后,不在原地等待,直接执行下一行代码,结果:
1 from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor 2 import time,os,random 3 4 def task(x): 5 print('%s 接客' %x) 6 time.sleep(random.randint(1,3)) 7 return x**2 8 9 if __name__ == '__main__': 10 # 异步调用 11 p=ThreadPoolExecutor(4) # 默认开启的线程数是cpu的核数*5 12 13 # alex,武佩奇,杨里,吴晨芋,张三 14 15 obj_l=[] 16 for i in range(10): 17 obj=p.submit(task,i) 18 obj_l.append(obj) 19 20 # p.close() 21 # p.join() 22 p.shutdown(wait=True) 23 24 print(obj_l[3].result()) 25 print('主')