通过gunicorn + supervisor管理python项目 Gunicorn(Green Unicorn)是一个Python WSGI HTTP Server for UNIX,来源于Ruby的unicorn项目。Gunicorn使用pre-fork master-worker模型(在gunicorn中,master被称为arbiter),能够与各种wsgi web框架协作。
Supervisor是用Python开发的一个client/server服务,是Linux/Unix系统下的一个进程管理工具。它可以很方便的监听、启动、停止、重启一个或多个进程。能将一个普通的命令行进程变为后台daemon,并监控进程状态,异常退出时能自动重启,很方便的做到进程自动恢复。
一、安装python组件 1 2 3 4 5 6 7 8 9 10 11 12 13 $ yum install python-pip $ pip install --upgrade pip $ pip install gunicorn supervisor gevent $ pip list --format=legacy gevent (1.2.2) gunicorn (19.7.1) pip (9.0.3) setuptools (0.9.8) supervisor (3.3.4)
二、配置python项目 配置app1,app2,app3三个项目,这里以显示“hello,world”信息为例
1 2 3 4 5 6 7 8 9 10 11 12 13 $ cd /home/www $ mkdir -p app1 app2 app3 supervisor/log $ cd app1 && vim app1/run.py //添加如下内容 def app(environ, start_response): data = b"app1 Hello, World!\n" start_response("200 OK" , [ ("Content-Type" , "text/plain" ), ("Content-Length" , str(len(data))) ]) return iter([data])
复制run.py到app2和app3目录中,并分别更改data行的输出名称为app2和app3,作为区分不同的项目data = b"app2 Hello, World!\n"
data = b"app3 Hello, World!\n"
命令运行测试,注意需要在项目目录下运行命令,通过curl测试访问
1 2 $ gunicorn –w 2 –b 127.0.0.1:8001 run:app & $ curl 127.0.0.1:8001
三、创建gunicorn配置文件 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 $ vim app1.py import loggingimport logging.handlersfrom logging.handlers import WatchedFileHandlerimport osimport multiprocessingbind = '127.0.0.1:8001' backlog = 2048 chdir = '/home/www/app1' timeout = 30 worker_class = 'gevent' workers = 2 threads = 2 debug = True pidfile = '/tmp/app1.pid' loglevel = 'info' access_log_format = '%(t)s %(p)s %(h)s "%(r)s" %(s)s %(L)s %(b)s %(f)s" "%(a)s"' accesslog = " ../supervisor/log/app1_access.log" errorlog = " ../supervisor/log/app1_error.log"
使用配置文件启动gunicorn脚本$ gunicorn -c app1.py run:app &
结束运行$ pkill gunicorn
同理配置app2和app3,更改对应的目录和文件名,端口号分别为8002和8003
四、创建supervisor配置文件 相关组件 supervisord — 服务端程序,启动supervisor服务 supervisorctl — 客户端工具,通过命令行来管理进程 echo_supervisord_conf — 生成默认的配置文件 web server — 支持web方式管理,提供简单的web管理页面
1 2 3 4 5 6 7 8 # 生成默认supervisor配置 $ cd /home/www/supervisor echo_supervisord_conf > supervisord.conf # 更改最后的子配置文件目录 $ vim supervisord.conf ... [include] ;取消注释 files = /home/www/supervisor/*.conf ;指定子配置文件路径
五、配置supervisor脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 $ vim app.conf [program:app1] command=/usr/bin/gunicorn –c app1.py run:app directory=/home/www/app1 ;注意这里也必须切换到gunicorn启动脚本目录 ;单进程配置 process_name=%(program_name)s numprocs=1 ;多进程配置 ;process_name=%(program_name)s_%(process_name)02d ;numprocs=2 autostart=true startsecs=10 autorestart=true startretries=3 user=www priority=1 redirect_stderr=true stdout_logfile_maxbytes=20MB stdout_logfile_backups=10 stdout_logfile= log/app1_super.log stopasgroup=false killasgroup=false [program:app2] #配置省略... [program:app3] #配置省略...
启动supervisor$ supervisord -c supervisord.conf
查看进程状态
1 2 3 4 $ supervisorctl status app1:app1_00 RUNNING pid 67885, uptime 0:04:48 app2:app2_00 RUNNING pid 67886, uptime 0:04:48 app3:app3_00 RUNNING pid 67897, uptime 0:04:47
管理命令 supervisorctl help //查看命令帮助 supervisorctl update //更新supervisor配置并重新加载 supervisorctl status //查看进程状态 supervisorctl stop app1: //停止单个进程,如果是多进程的话注意后面加冒号 supervisorctl stop all //关闭所有进程,但是保留supervisor进程 supervisorctl shutdown //关闭supervisor进程
如果出现以下错误,是因为找不到默认的配置文件导致的
1 2 $ supervisorctl status http://localhost:9001 refused connection
可以执行以下命令生成一个软连接$ ln -sv /home/www/supervisor/supervisord.conf /etc/supervisord.conf
查看进程树
1 2 3 4 5 6 7 $ pstree -p ├─supervisord(117512)─┬─gunicorn(67885)─┬─gunicorn(67895) │ │ └─gunicorn(67899) │ ├─gunicorn(67886)─┬─gunicorn(67896) │ │ └─gunicorn(67909) │ └─gunicorn(67897)─┬─gunicorn(67914) │ └─gunicorn(67915)
查看监听端口
1 2 3 4 $ ss -tnl | grep 800 LISTEN 0 128 127.0.0.1:8001 *:* LISTEN 0 128 127.0.0.1:8002 *:* LISTEN 0 128 127.0.0.1:8003 *:*
目录结构
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 $ tree ├── app1 │ ├── app1.py │ ├── run.py │ └── run.pyc ├── app2 │ ├── app2.py │ ├── run.py │ └── run.pyc ├── app3 │ ├── app3.py │ ├── run.py │ └── run.pyc ├── supervisor │ ├── app.conf │ ├── log │ │ ├── app1_access.log │ │ ├── app1_error.log │ │ ├── app1_super.log │ │ ├── app2_access.log │ │ ├── app2_error.log │ │ ├── app2_super.log │ │ ├── app3_access.log │ │ ├── app3_error.log │ │ └── app3_super.log │ └── supervisord.conf
进程终止测试,查看相关进程PID,比如第一条93828,是app3的一个进程
1 2 3 4 5 $ ps aux | grep gunicorn www 93828 0.1 1.3 213940 13552 ? S 15:21 0:00 /usr/bin/python2 /usr/bin/gunicorn -w 2 -b 127.0.0.1:8083 run:app www 93829 0.2 1.3 213936 13572 ? S 15:21 0:00 /usr/bin/python2 /usr/bin/gunicorn -w 2 -b 127.0.0.1:8082 run:app www 93830 0.2 1.3 213940 13552 ? S 15:21 0:00 /usr/bin/python2 /usr/bin/gunicorn -w 2 -b 127.0.0.1:8081 run:app ...
查看进程树
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 $ pstree -p | grep gunicorn |-supervisord(93815)-+-gunicorn(93828)-+-gunicorn(93835) | | `-gunicorn(93846) | |-gunicorn(93829)-+-gunicorn(93845) | | `-gunicorn(93848) | `-gunicorn(93830)-+-gunicorn(93842) | `-gunicorn(93847) # 结束进程93828 $ kill 93828 $ pstree -p | grep gunicorn |-supervisord(93815)-+-gunicorn(93829)-+-gunicorn(93845) | | `-gunicorn(93848) | |-gunicorn(93830)-+-gunicorn(93842) | | `-gunicorn(93847) | `-gunicorn(95571)---gunicorn(95576)
查看app3的supervisor日志,显示93828进程异常退出,重启后进程号为95571
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 $ tail -20 log/app3_super.log [2018-03-30 15:21:03 +0000] [93828] [INFO] Starting gunicorn 19.7.1 [2018-03-30 15:21:03 +0000] [93828] [INFO] Listening at: http://127.0.0.1:8083 (93828) [2018-03-30 15:21:03 +0000] [93828] [INFO] Using worker: sync [2018-03-30 15:21:03 +0000] [93835] [INFO] Booting worker with pid: 93835 [2018-03-30 15:21:03 +0000] [93846] [INFO] Booting worker with pid: 93846 [2018-03-30 15:23:29 +0000] [93828] [INFO] Handling signal: term [2018-03-30 15:23:29 +0000] [93846] [INFO] Worker exiting (pid: 93846) [2018-03-30 15:23:29 +0000] [93835] [INFO] Worker exiting (pid: 93835) [2018-03-30 15:23:29 +0000] [93828] [INFO] Shutting down: Master [2018-03-30 15:23:30 +0000] [95571] [INFO] Starting gunicorn 19.7.1 [2018-03-30 15:23:30 +0000] [95571] [INFO] Listening at: http://127.0.0.1:8083 (95571) [2018-03-30 15:23:30 +0000] [95571] [INFO] Using worker: sync [2018-03-30 15:23:30 +0000] [95576] [INFO] Booting worker with pid: 95576 [2018-03-30 15:23:30 +0000] [95579] [INFO] Booting worker with pid: 95579