Supervisord是用Python实现的一款非常实用的进程管理工具,类似于monit,monit和supervisord的一个比较大的差异是supervisord管理的进程必须由supervisord来启动,monit可以管理已经在运行的程序;supervisord还要求管理的程序是非daemon程序,supervisord会帮你把它转成daemon程序,因此如果用supervisord来管理nginx的话,必须在nginx的配置文件里添加一行设置daemon off让nginx以非daemon方式启动。(此段话引用链接:http://feilong.me/2011/03/monitor-processes-with-supervisord)
注意:如果用supervisorctlstart nginx启动nginx显示错误nginx: ERROR(abnormal termination),查看日志说是nginx已经启动,这就是因为nginx本身是daemon程序,要在nginx的配置文件中把它转为非daemon程序.
1.Supervisor安装:
# 安装
easy_install supervisor
也可采用离线安装的方式:
安装python
安装meld3-0.6.8.tar.gz
安装elementtree-1.2.6-20050316.tar.gz
安装supervisor-3.0a12.tar.gz
官方安装文档:
http://supervisord.org/installing.html
依赖软件:
setuptools (latest) from http://pypi.python.org/pypi/setuptools.
meld3 (latest) from http://www.plope.com/software/meld3/.
elementtree (latest) from http://effbot.org/downloads#elementtree.
2.生成默认配置文件:
# 生成默认配置文件
echo_supervisord_conf > /etc/supervisord.conf
mkdir /etc/supervisord.conf.d
3.修改配置文件
include区段修改为:
[include]
files = /etc/supervisord.conf.d/*.conf
每个需要管理的进程分别写在一个文件里面,放在/etc/supervisord.conf.d/目录下,便于管理。例如:test.conf
[program:sqlparse]
directory = /var/local/python
command = /use/bin python test.py
4.把supervisord服务设为开机自启动,把下面程序复制下来生成supervisord文件放到/etc/init.d/目录下面:
CentOS环境下:
#!/bin/sh
#
# /etc/rc.d/init.d/supervisord
#
# Supervisor is a client/server system that
# allows its users to monitor and control a
# number of processes on UNIX-like operating
# systems.
#
# chkconfig: -
# description: Supervisor Server
# processname: supervisord # Source init functions
. /etc/init.d/functions RETVAL=
prog="supervisord"
pidfile="/tmp/supervisord.pid"
lockfile="/var/lock/subsys/supervisord" start()
{
echo -n $"Starting $prog: "
daemon --pidfile $pidfile supervisord -c /etc/supervisord.conf
RETVAL=$?
echo
[ $RETVAL -eq ] && touch ${lockfile}
} stop()
{
echo -n $"Shutting down $prog: "
killproc -p ${pidfile} /usr/bin/supervisord
RETVAL=$?
echo
if [ $RETVAL -eq ] ; then
rm -f ${lockfile} ${pidfile}
fi
} case "$1" in start)
start
;; stop)
stop
;; status)
status $prog
;; restart)
stop
start
;; *)
echo "Usage: $0 {start|stop|restart|status}"
;; esac
Ubuntu环境下:
#! /bin/sh
### BEGIN INIT INFO
# Provides: supervisord
# Required-Start: $remote_fs
# Required-Stop: $remote_fs
# Default-Start:
# Default-Stop:
# Short-Description: Example initscript
# Description: This file should be used to construct scripts to be
# placed in /etc/init.d.
### END INIT INFO # Author: Dan MacKinlay <danielm@phm.gov.au>
# Based on instructions by Bertrand Mathieu
# http://zebert.blogspot.com/2009/05/installing-django-solr-varnish-and.html # Do NOT "set -e" # PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="Description of the service"
NAME=supervisord
DAEMON=/usr/local/bin/supervisord
DAEMON_ARGS=""
PIDFILE=/tmp/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME # Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0 # Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME # Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh # Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions #
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
|| return 1
start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
$DAEMON_ARGS \
|| return 2
# Add code here, if necessary, that waits for the process to be ready
# to handle requests from services started subsequently which depend
# on this one. As a last resort, sleep for some time.
} #
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
RETVAL="$?"
[ "$RETVAL" = 2 ] && return 2
# Wait for children to finish too if this is a daemon that forks
# and if the daemon is only ever run from this initscript.
# If the above conditions are not satisfied then add some other code
# that waits for the process to drop all resources that could be
# needed by services started subsequently. A last resort is to
# sleep for some time.
start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
[ "$?" = 2 ] && return 2
# Many daemons don't delete their pidfiles when they exit.
rm -f $PIDFILE
return "$RETVAL"
} #
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
#
# If the daemon can reload its configuration without
# restarting (for example, when it is sent a SIGHUP),
# then implement that here.
#
start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
return 0
} case "$1" in
start)
[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
do_start
case "$?" in
0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
esac
;;
stop)
[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
do_stop
case "$?" in
0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
esac
;;
#reload|force-reload)
#
# If do_reload() is not implemented then leave this commented out
# and leave 'force-reload' as an alias for 'restart'.
#
#log_daemon_msg "Reloading $DESC" "$NAME"
#do_reload
#log_end_msg $?
#;;
restart|force-reload)
#
# If the "reload" option is implemented then remove the
# 'force-reload' alias
#
log_daemon_msg "Restarting $DESC" "$NAME"
do_stop
case "$?" in
0|1)
do_start
case "$?" in
0) log_end_msg 0 ;;
1) log_end_msg 1 ;; # Old process is still running
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
#echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
exit 3
;;
esac
用root用户执行下面命令把supervisord服务设为开机自启动:
CentOS系统下:
chmod +x /etc/init.d/supervisord
chkconfig supervisord on
service supervisord start
Ubuntu系统下:
chmod +x /etc/init.d/supervisord
update-rc.d supervisord defaults
5.supervisord管理
Supervisord安装完成后有两个可用的命令行supervisor和supervisorctl,命令使用解释如下:
- supervisord,初始启动Supervisord,启动、管理配置中设置的进程。
- supervisorctl stop programxxx,停止某一个进程(programxxx),programxxx为
[program:chatdemon]
里配置的值,这个示例就是chatdemon。 - supervisorctl start programxxx,启动某个进程
- supervisorctl restart programxxx,重启某个进程
- supervisorctl stop groupworker: ,重启所有属于名为groupworker这个分组的进程(start,restart同理)
- supervisorctl stop all,停止全部进程,注:start、restart、stop都不会载入最新的配置文件。
- supervisorctl reload,载入最新的配置文件,停止原有进程并按新的配置启动、管理所有进程。
- supervisorctl update,根据最新的配置文件,启动新配置或有改动的进程,配置没有改动的进程不会受影响而重启。
- 注意:显示用stop停止掉的进程,用reload或者update都不会自动重启。
6.supervisord的配置文件(/etc/supervisord.conf)
; Sample supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
;
; Note: shell expansion ("~" or "$HOME") is not supported. Environment
; variables can be expanded using this syntax: "%(ENV_HOME)s". [unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
;chmod= ; socket file mode (default )
;chown=nobody:nogroup ; socket file uid:gid owner
;username=user ; (default is no username (open server))
;password= ; (default is no password (open server)) ;[inet_http_server] ; inet (TCP) server disabled by default
;port=127.0.0.1: ; (ip_address:port specifier, *:port for all iface)
;username=user ; (default is no username (open server))
;password= ; (default is no password (open server)) [inet_http_server] ; inet (TCP) server disabled by default
port=0.0.0.0: ; (ip_address:port specifier, *:port for all iface)
username=supervisor ; (default is no username (open server))
password=supervisor ; (default is no password (open server)) [supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups= ; (num of main logfile rotation backups;default )
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds= ; (min. avail startup file descriptors;default )
minprocs= ; (min. avail process descriptors;default )
;umask= ; (process file creation umask;default )
;user=chrism ; (default is current user, required if root)
;identifier=supervisor ; (supervisord identifier, default is 'supervisor')
;directory=/tmp ; (default is not to cd during start)
;nocleanup=true ; (don't clean up tempfiles at start;default false)
;childlogdir=/tmp ; ('AUTO' child log dir, default $TEMP)
;environment=KEY="value" ; (key value pairs to add to environment)
;strip_ansi=false ; (strip ansi escape codes in logs; def. false) ; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface [supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
;username=chris ; should be same as http_username if set
;password= ; should be same as http_password if set
;prompt=mysupervisor ; cmd line prompt (default "supervisor")
;history_file=~/.sc_history ; use readline history if available ; The below sample program section shows all possible program subsection values,
; create one or more 'real' program: sections to be able to control them under
; supervisor. ;[program:theprogramname]
;command=/bin/cat ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs= ; number of processes copies to start (def )
;directory=/tmp ; directory to cwd to before exec (def no cwd)
;umask= ; umask for process (default None)
;priority= ; the relative start priority (default )
;autostart=true ; start at supervisord start (default: true)
;autorestart=unexpected ; whether/when to restart (default: unexpected)
;startsecs= ; number of secs prog must stay running (def. )
;startretries= ; max # of serial start failures (default )
;exitcodes=, ; 'expected' exit codes for process (default ,)
;stopsignal=QUIT ; signal used to kill process (default TERM)
;stopwaitsecs= ; max num secs to wait b4 SIGKILL (default )
;stopasgroup=false ; send stop signal to the UNIX process group (default false)
;killasgroup=false ; SIGKILL the UNIX process group (def false)
;user=chrism ; setuid to this UNIX account to run the program
;redirect_stderr=true ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups= ; # of stdout logfile backups (default )
;stdout_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default )
;stdout_events_enabled=false ; emit events on stdout writes (default false)
;stderr_logfile=/a/path ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups= ; # of stderr logfile backups (default )
;stderr_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default )
;stderr_events_enabled=false ; emit events on stderr writes (default false)
;environment=A="",B="" ; process environment additions (def no adds)
;serverurl=AUTO ; override serverurl computation (childutils) ; The below sample eventlistener section shows all possible
; eventlistener subsection values, create one or more 'real'
; eventlistener: sections to be able to handle event notifications
; sent by supervisor. ;[eventlistener:theeventlistenername]
;command=/bin/eventlistener ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs= ; number of processes copies to start (def )
;events=EVENT ; event notif. types to subscribe to (req'd)
;buffer_size= ; event buffer queue size (default )
;directory=/tmp ; directory to cwd to before exec (def no cwd)
;umask= ; umask for process (default None)
;priority=- ; the relative start priority (default -)
;autostart=true ; start at supervisord start (default: true)
;autorestart=unexpected ; whether/when to restart (default: unexpected)
;startsecs= ; number of secs prog must stay running (def. )
;startretries= ; max # of serial start failures (default )
;exitcodes=, ; 'expected' exit codes for process (default ,)
;stopsignal=QUIT ; signal used to kill process (default TERM)
;stopwaitsecs= ; max num secs to wait b4 SIGKILL (default )
;stopasgroup=false ; send stop signal to the UNIX process group (default false)
;killasgroup=false ; SIGKILL the UNIX process group (def false)
;user=chrism ; setuid to this UNIX account to run the program
;redirect_stderr=true ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups= ; # of stdout logfile backups (default )
;stdout_events_enabled=false ; emit events on stdout writes (default false)
;stderr_logfile=/a/path ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups ; # of stderr logfile backups (default )
;stderr_events_enabled=false ; emit events on stderr writes (default false)
;environment=A="",B="" ; process environment additions
;serverurl=AUTO ; override serverurl computation (childutils) ; The below sample group section shows all possible group values,
; create one or more 'real' group: sections to create "heterogeneous"
; process groups. ;[group:thegroupname]
;programs=progname1,progname2 ; each refers to 'x' in [program:x] definitions
;priority= ; the relative start priority (default ) ; The [include] section can just contain the "files" setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files *cannot*
; include files themselves. ;[include]
;files = relative/directory/*.ini [include]
files = /etc/supervisord.conf.d/*.conf
7.管理实例
7.1.测试一个简单的python程序:
在/etc/supervisord.conf.d/目录下生成testsupervisor.conf文件
[program:testsupervisor]
directory = /opt/testsupervisor
command = /usr/local/bin/python testsupervisor.py
priority=
numprocs=
autostart=true
autorestart=true
directory表示testsupervisor.py所在的工作目录
command表示执行的命令,注意python要用绝对路径:/usr/local/bin/python
priority表示优先级,用supervisord管理的程序会按优先级从高到低依次启动,数字越多,优先级越高
autostart表示自动启动
autorestart表示自动重启
7.2.用supervisord管理 redis程序:
在/etc/supervisord.conf.d/目录下生成redis.conf文件
[program:redis]
directory=/usr/local/src/redis-2.6.
command=/usr/local/src/redis-2.6./src/redis-server redis.conf stdout_logfile=/tmp/redis_access.log
stderr_logfile=/tmp/redis_error.log
autostart=true
autorestart=true
startsecs= ; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = ; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=
7.3.用supervisord管理nginx程序,因为nginx默认就是以daemon方式启动的,所以如果用supervisord来管理nginx的话,必须在nginx的配置文件里添加一行设置daemon off让nginx以非daemon方式启动。(supervisord还要求管理的程序是非daemon程序,supervisord会帮你把它转成daemon程序)
在/usr/local/nginx/conf/目录下面修改nginx.conf文件:
worker_processes ;
daemon off;
events {
#use epoll;
worker_connections ;
}
http{
#gzip on;
#gzip_min_length ;
#gzip_buffers 16k;
#gzip_http_version 1.0;
#gzip_comp_level ;
#gzip_types text/plain application/x-javascript text/css application/xml;
#gzip_vary on; upstream 192.168.56.11 {
#这里指定多个源服务器,ip:端口,80端口的话可写可不写
server 127.0.0.1:;
server 127.0.0.1:;
server 127.0.0.1:;
server 127.0.0.1:; }
server {
listen ;
location / {
#rewrite ^/smmp/servletsendmoremsg\.do$ /send last;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass http://192.168.56.11; } } # server {
# listen ;
# ssl on;
# ssl_certificate server.crt;
# ssl_certificate_key server.key;
#
# ssl_session_timeout 5m;
#
# ssl_protocols SSLv2 SSLv3 TLSv1;
# ssl_ciphers HIGH:!aNULL:!MD5;
# ssl_prefer_server_ciphers on;
# location / {
# proxy_redirect off;
# proxy_set_header Host $host;
# proxy_set_header X-Forwarded-For $remote_addr;
# proxy_pass http://192.168.56.11;
#
# }
#
# }
}
在/etc/supervisord.conf.d/目录下生成nginx.conf文件
[program:nginx]
command=/usr/local/nginx/sbin/nginx stdout_logfile=/tmp/nginx.log
stderr_logfile=/tmp/nginx.log
autostart=true
autorestart=true
startsecs= ; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = ; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=
7.4.用supervisord管理celery服务(python环境下):
celery官方链接:https://github.com/celery/celery/tree/3.0/extra/supervisord
在/etc/supervisord.conf.d/目录下生成celeryd.conf文件,启了两个worker,配置文件中要注意celeryconfig.py的路径:
; ============================
; celeryd supervisor example
; ============================ ; NOTE: If you're using Django, you shouldn't use this file.
; Use
; http://github.com/celery/django-celery/tree/master/extra/supervisord/celeryd.conf
; instead! [program:celery]
command=/usr/local/bin/celery worker --loglevel INFO --queues=low_send_task_queue,high_send_task_queue process_name = %(program_name)s-%(process_num)d ; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH=/usr/local/workspace_eclipse3.7.2/venus directory=/usr/local/workspace_eclipse3.7.2/venus numprocs=
stdout_logfile=/tmp/celery_sendsms_access.log
stderr_logfile=/tmp/celery_sendsms_error.log
autostart=true
autorestart=true
startsecs= ; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = ; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=
7.5.用supervisord管理celery服务(django环境下):
在/etc/supervisord.conf.d/目录下生成celeryd_worker.conf文件
; ==============================================
; celery worker supervisor example for Django
; ============================================== [program:celeryworker]
command=/usr/local/bin/python manage.py celery worker --loglevel=INFO --queues=db_write_back_queue
directory=/usr/local/workspace_eclipse3.7.2/sms_web numprocs=
stdout_logfile=/tmp/celeryworker.log
stderr_logfile=/tmp/celeryworker.log
autostart=true
autorestart=true
startsecs= ; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = ; if rabbitmq is supervised, set its priority higher
; so it starts first
priority= ;process_name = %(program_name)s-%(process_num)d
;numprocs=
在/etc/supervisord.conf.d/目录下生成celeryd_beat.conf文件,django环境下的心跳服务
; ==============================================
; celery worker supervisor example for Django
; ============================================== [program:celerybeat]
command=/usr/local/bin/python manage.py celery beat -s /var/log/sms_web/celerybeat-schedule --loglevel=INFO
directory=/usr/local/workspace_eclipse3.7.2/sms_web numprocs=
stdout_logfile=/tmp/celerybeat.log
stderr_logfile=/tmp/celerybeat.log
autostart=true
autorestart=true
startsecs= ; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = ; if rabbitmq is supervised, set its priority higher
; so it starts first
priority= ;process_name = %(program_name)s-%(process_num)d
;numprocs=
7.6.用supervisord管理venus服务(启了四个进程):
在/etc/supervisord.conf.d/目录下生成venux.conf文件
[program:venus-]
directory=/usr/local/workspace_eclipse3.7.2/venus
command=/usr/local/bin/python main.py --port=
stdout_logfile=/tmp/venus_8003.log
stderr_logfile=/tmp/venus_8003.log
autostart=true
autorestart=true
priority= [program:venus-]
directory=/usr/local/workspace_eclipse3.7.2/venus
command=/usr/local/bin/python main.py --port=
stdout_logfile=/tmp/venus_8004.log
stderr_logfile=/tmp/venus_8004.log
autostart=true
autorestart=true
priority= [program:venus-]
directory=/usr/local/workspace_eclipse3.7.2/venus
command=/usr/local/bin/python main.py --port=
stdout_logfile=/tmp/venus_8005.log
stderr_logfile=/tmp/venus_8005.log
autostart=true
autorestart=true
priority= [program:venus-]
directory=/usr/local/workspace_eclipse3.7.2/venus
command=/usr/local/bin/python main.py --port=
stdout_logfile=/tmp/venus_8006.log
stderr_logfile=/tmp/venus_8006.log
autostart=true
autorestart=true
priority=
~
7.7.用supervisord管理接收短信的服务:
在/etc/supervisord.conf.d/目录下生成receiver_cmpp.conf文件
[program:receiver-cmpp]
directory=/usr/local/workspace_eclipse3.7.2/sms_tool
command=/usr/local/bin/python receiver_cmpp.py stdout_logfile=/tmp/receiver_cmpp.log
stderr_logfile=/tmp/receiver_cmpp.log
autostart=true
autorestart=true
startsecs= ; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = ; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=
在/etc/supervisord.conf.d/目录下添加新的配置文件后,执行命令:
supervisorctl update
就会把新添加的服务启动起来,且不会影响正在运行的 服务。