multiprocessing模块实现了对多进程编程的封装,让我们可以非常方便的使用多进程进行编程。它的使用方法非常类似threading模块。
1.创建一个进程
import multiprocessing def worker(): """worker function""" print ‘Worker‘ return if __name__ == ‘__main__‘: jobs = [] for i in range(5): p = multiprocessing.Process(target=worker) jobs.append(p) p.start()输出结果:
$ python multiprocessing_simple.py Worker Worker Worker Worker Worker
2.传递参数给子进程
import multiprocessing def worker(num): """thread worker function""" print 'Worker:', num return if __name__ == '__main__': jobs = [] for i in range(5): p = multiprocessing.Process(target=worker, args=(i,)) jobs.append(p) p.start()运行结果:
$ python multiprocessing_simpleargs.py Worker: 0 Worker: 1 Worker: 2 Worker: 3 Worker: 4注意:所传递的参数必需是可以序列化的
3.后台进程
默认情况下,主进程会等待所有子进程运行完毕后退出。如果需要让子进程独立在后台运行,可以如下操作。
import multiprocessing import time import sys def daemon(): p = multiprocessing.current_process() print 'Starting:', p.name, p.pid sys.stdout.flush() time.sleep(2) print 'Exiting :', p.name, p.pid sys.stdout.flush() def non_daemon(): p = multiprocessing.current_process() print 'Starting:', p.name, p.pid sys.stdout.flush() print 'Exiting :', p.name, p.pid sys.stdout.flush() if __name__ == '__main__': d = multiprocessing.Process(name='daemon', target=daemon) d.daemon = True n = multiprocessing.Process(name='non-daemon', target=non_daemon) n.daemon = False d.start() time.sleep(1) n.start()运行结果:
$ python multiprocessing_daemon.py Starting: daemon 13866 Starting: non-daemon 13867 Exiting : non-daemon 13867
可以看到,daemon并没有exiting输出。
4.等待进程
import multiprocessing import time import sys def daemon(): print 'Starting:', multiprocessing.current_process().name time.sleep(2) print 'Exiting :', multiprocessing.current_process().name def non_daemon(): print 'Starting:', multiprocessing.current_process().name print 'Exiting :', multiprocessing.current_process().name if __name__ == '__main__': d = multiprocessing.Process(name='daemon', target=daemon) d.daemon = True n = multiprocessing.Process(name='non-daemon', target=non_daemon) n.daemon = False d.start() time.sleep(1) n.start() d.join() n.join()
join方法会等待对应的进程执行完毕,然后进行下一步操作。
运行结果:
$ python multiprocessing_daemon_join.py Starting: non-daemon Exiting : non-daemon Starting: daemon Exiting : daemon
join方法还可以指定timeout参数,等待一段时间,如果子进程还没有执行完毕,那么就会返回。
import multiprocessing import time import sys def daemon(): print 'Starting:', multiprocessing.current_process().name time.sleep(2) print 'Exiting :', multiprocessing.current_process().name def non_daemon(): print 'Starting:', multiprocessing.current_process().name print 'Exiting :', multiprocessing.current_process().name if __name__ == '__main__': d = multiprocessing.Process(name='daemon', target=daemon) d.daemon = True n = multiprocessing.Process(name='non-daemon', target=non_daemon) n.daemon = False d.start() n.start() d.join(1) print 'd.is_alive()', d.is_alive() n.join()
5.关闭子进程
import multiprocessing import time def slow_worker(): print 'Starting worker' time.sleep(0.1) print 'Finished worker' if __name__ == '__main__': p = multiprocessing.Process(target=slow_worker) print 'BEFORE:', p, p.is_alive() p.start() print 'DURING:', p, p.is_alive() p.terminate() print 'TERMINATED:', p, p.is_alive() p.join() print 'JOINED:', p, p.is_alive()
输出:
$ python multiprocessing_terminate.py BEFORE: <Process(Process-1, initial)> False DURING: <Process(Process-1, started)> True TERMINATED: <Process(Process-1, started)> True JOINED: <Process(Process-1, stopped[SIGTERM])> False
6.进程返回状态
0,没有错误产生
>0,进程产生了一个错误
<0,进程被杀死
import multiprocessing import sys import time def exit_error(): sys.exit(1) def exit_ok(): return def return_value(): return 1 def raises(): raise RuntimeError('There was an error!') def terminated(): time.sleep(3) if __name__ == '__main__': jobs = [] for f in [exit_error, exit_ok, return_value, raises, terminated]: print 'Starting process for', f.func_name j = multiprocessing.Process(target=f, name=f.func_name) jobs.append(j) j.start() jobs[-1].terminate() for j in jobs: j.join() print '%s.exitcode = %s' % (j.name, j.exitcode)
输出:
$ python multiprocessing_exitcode.py Starting process for exit_error Starting process for exit_ok Starting process for return_value Starting process for raises Starting process for terminated Process raises: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python 2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python 2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "multiprocessing_exitcode.py", line 24, in raises raise RuntimeError('There was an error!') RuntimeError: There was an error! exit_error.exitcode = 1 exit_ok.exitcode = 0 return_value.exitcode = 0 raises.exitcode = 1 terminated.exitcode = -15