如果直接使用 multiprocess.Pool
多进程执行任务函数,若任务函数中抛出异常,则主进程中无法获得异常信息。例如:
import multiprocessing
def task():
print("test print")
raise Exception()
if __name__ == "__main__":
pool = multiprocessing.Pool()
pool.apply_async(task)
pool.close()
pool.join()
print("finish")
运行结果如下:
test print
finish
Process finished with exit code 0
1. 在主进程中获取 apply_async()
方法返回的 ApplyResult
实例,并调用其 get()
获取异常
我们可以在 task
函数中嵌套异常捕获逻辑,但是无法捕获 task
函数的实参和形参不匹配的问题。因此,为了在主进程中获取各种异常信息,我们可以将上述逻辑改写如下:
import multiprocessing
def task():
print("test print")
raise Exception()
if __name__ == "__main__":
pool = multiprocessing.Pool()
t = pool.apply_async(task)
t.get()
pool.close()
pool.join()
运行结果如下:
test print
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "test.py", line 6, in task
raise Exception()
Exception
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "test.py", line 12, in <module>
t.get()
File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 771, in get
raise self._value
Exception
Process finished with exit code 1
这种处理导致主进程抛出异常被停止。
2. 在主进程捕获异常并打印信息
于是在主进程捕获异常,并打印异常信息即可:
import multiprocessing
import traceback
def task():
print("test print")
raise Exception()
if __name__ == "__main__":
pool = multiprocessing.Pool()
t = pool.apply_async(task)
try:
t.get()
except Exception as e:
traceback.print_exc()
pool.close()
pool.join()
print("finish")
运行结果如下:
test print
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "E:\brick\test\多进程报错样例.py", line 7, in task
raise Exception()
Exception
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "E:\brick\test\多进程报错样例.py", line 14, in <module>
t.get()
File "C:\Program Files\Python39\lib\multiprocessing\pool.py", line 771, in get
raise self._value
Exception
finish