nginx+uwsgi+django+supervisor+mysql+redis

目录

1. 概述 3

2. 安装与配置 3

2.1 django项目与应用创建 3

2.2 uwsgi安装与配置 6

2.3 supervisor安装与配置 8

2.4 nginx安装与作为反向代理服务器的配置 10

3. nginx+uwsgi优化 16

3.1   uwsgi进程线程数配置优化 16

3.2 Django数据库连接参数CONN_MAX_AGE优化 20

3.3 nginx中worker_rlimit_nofile与worker_connections优化 21

1. 概述

基于CentOS7.4的Nginx+UWsgi+Django+Supervisor是常见的web项目部署方式。为方便项目部署,了解各个软件配置,性能调优提供参考,以及高并发环境下一些异常修复与规避。

2. 安装与配置

PtCloud可根据环境的不同进行灵活性的搭建和配置,但都需要使用PtCloud发行版光盘进行角色的选择,并至少选择一台物理机作为计算节点。根据根据安装的不同情况做以下的安装说明。

2.1  Django项目与app创建

2.1.1 给python2.7安装django1.10的库

[root@bogon my_projects]# pip install django==1.10

2.1.2 创建django项目helloworld

[root@bogon my_projects]# django-admin.py startproject hellowold

2.1.3 创建第一个应用 application1

[root@bogon my_projects]# cd hellowold

[root@bogon hellowold]# django-admin.py startapp application1

2.1.4 修改setting.py中的ALLOWED_HOSTS

说明:ALLOWED_HOSTS后面所跟的属性值是一个字符串列表值,这个字符串列表值表示当下这个Django站点可以提供的host/domain(主机/域名)。*通配符去配置,另外当DEBUG设置为False的时候必须配置这个配置。否则会抛出异常。

将/home/my_projects/helloworld/helloworld/setting.py中的ALLOWED_HOSTS修改为以下内容

ALLOWED_HOSTS = ['*']

2.1.5 运行django项目

[root@bogon hellowold]# ./manage.py runserver 0.0.0.0:1001

在另一个终端执行以下命令,进行测试

[root@bogon my_projects]# curl http://127.0.0.1:1001

<!DOCTYPE html>

<html lang="en"><head>

<meta http-equiv="content-type" content="text/html; charset=utf-8">

<meta name="robots" content="NONE,NOARCHIVE"><title>Welcome to Django</title>

<style type="text/css">

html * { padding:0; margin:0; }

body * { padding:10px 20px; }

body * * { padding:0; }

body { font:small sans-serif; }

body>div { border-bottom:1px solid #ddd; }

h1 { font-weight:normal; }

h2 { margin-bottom:.8em; }

h2 span { font-size:80%; color:#666; font-weight:normal; }

h3 { margin:1em 0 .5em 0; }

h4 { margin:0 0 .5em 0; font-weight: normal; }

table { border:1px solid #ccc; border-collapse: collapse; width:100%; background:white; }

tbody td, tbody th { vertical-align:top; padding:2px 3px; }

thead th {

padding:1px 6px 1px 3px; background:#fefefe; text-align:left;

font-weight:normal; font-size:11px; border:1px solid #ddd;

}

tbody th { width:12em; text-align:right; color:#666; padding-right:.5em; }

#summary { background: #e0ebff; }

#summary h2 { font-weight: normal; color: #666; }

#explanation { background:#eee; }

#instructions { background:#f6f6f6; }

#summary table { border:none; background:transparent; }

</style>

</head>

<body>

<div id="summary">

<h1>It worked!</h1>

<h2>Congratulations on your first Django-powered page.</h2>

</div>

<div id="instructions">

<p>

Of course, you haven't actually done any work yet. Next, start your first app by running <code>python manage.py startapp [app_label]</code>.

</p>

</div>

<div id="explanation">

<p>

You're seeing this message because you have <code>DEBUG = True</code> in your Django settings file and you haven't configured any URLs. Get to work!

</p>

</div>

</body></html>

2.2 uwsgi安装与配置

2.2.1安装uwsgi

[root@bogon hellowold]# yum -y install uwsgi

2.2.2 django项目hellowold中uwsgi.ini配置

[root@bogon hellowold]# pwd

/home/my_projects/hellowld

[root@bogon hellowold]# cat uwsgi.ini

[uwsgi]

chdir           = /home/my_projects/helloworld

module         = helloworld.wsgi:application

reload-mercy    = 10

user            = root

uid             = root

master          = True

harakiri-verbose = true

post-buffering   = 65536

buffer-size      = 65536

Harakiri        = 30

threads         = 30  # 与进程的乘积不大于数据库允许的最大连接数

processes       = 8  # 二倍于CPU核数

# socket        = 127.0.0.1:7001  # nginx和 uwsgi间走的是 wsgi 协议,对应的nginx那边也需要配置uwsgi的一些参数

http             = 127.0.0.1:7001  # uwsgi将本端口的流按照http协议解析

chmod-socket    = 664

vacuum          = true

2.2.3 用uwsgi启动django项目

如果步骤2.1.5启动的程序还未停止,请用ctrl+c先停止,然后执行以下命令

[root@bogon hellowold]# uwsgi --ini /home/my_projects/helloworld/uwsgi.ini

在另一个终端中查看结果,执行

[root@bogon hellowold]# curl http://127.0.0.1:1001 |grep Congratulations

<h2>The install worked successfully! Congratulations!</h2>

2.3 supervisor安装与配置

2.3.1 supervisor安装

[root@bogon hellowold]# yum install supervisor

2.3.2 supervisor配置

将/etc/supervisor/supervisor.conf修改为以下内容

[unix_http_server]

file=/var/run/supervisor/supervisor.sock   ; (the path to the socket file)

[supervisord]

logfile=/var/log/supervisor/supervisord.log  ; (main log file;default $CWD/supervisord.log)

logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)

logfile_backups=10          ; (num of main logfile rotation backups;default 10)

loglevel=info               ; (log level;default info; others: debug,warn,trace)

pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)

nodaemon=false              ; (start in foreground if true;default false)

minfds=1024                 ; (min. avail startup file descriptors;default 1024)

minprocs=200                ; (min. avail process descriptors;default 200)

[rpcinterface:supervisor]

supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]

serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL  for a unix socket

[include]

files = /etc/supervisor/conf.d/*.conf

在/etc/supervisor/supervisor.d目录下创建helloworld.conf内容如下

[program:helloworld]

directory = /home/my_projects/helloworld

command = uwsgi --ini /home/my_projects/helloworld/uwsgi.ini

user = root

autostart = true

autorestart = true

stopsignal = QUIT

redirect_stderr = true

loglevel = error

stdout_logfile = /var/log/uwsgi/client_uwsgi_out.log

stderr_logfile = /var/log/uwsgi/client_uwsgi_err.log

logfile_maxbytes = 10M

aogfile_maxbytes = 2M

如果步骤2.2.2启动的程序还在运行,请先ctrl+c停止,然后执行以下命令

[root@bogon hellowold]# supervisord -c /etc/supervisor/supervisord.conf  # 启动supervisor服务

helloworld                       RUNNING   pid 28899, uptime 0:00:03

其它supervisor相关命令

supervisorctl status  # 查看supervisor管理的程序

supervisorctl stop all  # 停止supervisor守护的所有进程

supervisorctl start all  # 启动supervisor守护的所有进程

supervisorctl stop helloworld  # 停止supervisor守护的helloworld进程(也可以是其它被守护的进程,该名称在/etc/supervisor/conf.d/***.conf文件中[program:后配置)。

supervisorctl start helloworld  # 启动supervisor守护的helloworld进程

supervisorctl restart helloworld  # 重启supervisor守护的helloworld进程

2.4  Nginx安装与作为反向代理服务器的配置

2.4.1安装nginx

[root@bogon my_projects]# yum -y install nginx

安装好nginx后,nginx也为自己创建好了用户。在/etc/nginx/nginx.conf中user后面就是nginx执行时所使用的用户。如:“user nginx;”

2.4.2配置nginx

除原有niginx.conf中的用户不修改外,其余修改为以下内容。

# user nginx;  # nginx 用户为原文件中的nginx用户。

worker_processes auto;

pid /run/nginx.pid;

events {

worker_connections 768;

}

http {

sendfile on;

tcp_nopush on;

tcp_nodelay on;

keepalive_timeout 65;

types_hash_max_size 4096;

include /etc/nginx/mime.types;

default_type application/octet-stream;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE

ssl_prefer_server_ciphers on;

access_log /var/log/nginx/access.log;

error_log /var/log/nginx/error.log;

gzip on;

gzip_disable "msie6";

include /etc/nginx/conf.d/*.conf;

}

给helloworld项目设置反向代理规则

[root@bogon my_projects]# vim /etc/nginx/conf.d/helloworld.conf

server {

listen       80;

server_name  127.0.0.1;  # ext-sandbox.51zhuan.com;

charset      utf-8;

access_log /var/log/nginx/helloworld/nginx.access.log;

error_log /var/log/nginx/helloworld/nginx.error.log;

# Load configuration files for the default server block.

location / {

proxy_pass_header Server;

proxy_set_header Host $http_host;

proxy_redirect off;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://127.0.0.1:1001;

}

error_page 404 /404.html;

location = /40x.html {

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

}

}

检查配置文件nginx.conf的正确性:

[root@bogon my_projects]# nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

2.1.3启动 Nginx

Nginx 重新启动命令如下:

[root@bogon my_projects]# service nginx restart

访问站点

[root@bogon my_projects]# curl http://127.0.01

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

<style>

body {

width: 35em;

margin: 0 auto;

font-family: Tahoma, Verdana, Arial, sans-serif;

}

</style>

</head>

<body>

<h1>Welcome to nginALLOWED_HOSTS = ['*']

x!</h1>

<p>If you see this page, the nginx web server is successfully installed and

working. Further configuration is required.</p>

<p>For online documentation and support please refer to

<a href="http://nginx.org/">nginx.org</a>.<br/>

Commercial support is available at

<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>

</body>

</html>

1.2.4 Nginx 其他命令

以下包含了 Nginx 常用的几个命令:

nginx -s reload            # 重新载入配置文件

nginx -s reopen            # 重启 Nginx

nginx -s stop              # 停止 Nginx

service nginx status       # 使用service查看nginx的状态

service nginx start        # 使用service启动nginx

service nginx stop         # 使用service停止nginx

service nginx restart      # 使用service重启nginx

3.nginx+uwsgi优化

3.1 http与socket路由方式

3.1.1 http路由方式

使用http路由方式,uwsgi与nginx之间也会以http协议进行通信。在uwsgi内部会生成一个监听已配置端口的服务器,转发请求到由master进程管理的4个uWSGI worker组成的池中。

3.1.2 socket路由方式

nginx与uwsgi间使用wsgi协议进行socket通信,uwsgi接受到nginx传输的数据后直接由workers进程对数据进行处理,然后返回给nginx,nginx再解析成http返回给客户端。

修改uwsgi启动方式为socket,将uwsgi.ini中的http = 127.0.0.1:1001修改为socket = 127.0.0.1:1001

[root@bogon my_projects]# cd /home/my_projects/helloworld

[root@bogon helloworld]# cat uwsgi.ini

[uwsgi]

chdir           = /home/agu/temp/helloworld

module          = helloworld.wsgi:application

reload-mercy    = 10

user            = root

uid             = root

master          = True

harakiri-verbose = true

post-buffering   = 65536

buffer-size      = 65536

harakiri        = 30

threads         = 30

processes       = 8

socket          = 127.0.0.1:1001

# http            = 127.0.0.1:1001

chmod-socket    = 664

vacuum          = true

重启服务

[root@bogon helloworld]# supervisorctl restart helloworld

在使用socket方式时nginx需要与uwsgi适配,在/etc/nginx/nginx.conf文件http模块中加入以下两行

include uwsgi_params;

uwsgi_read_timeout 120;

创建/etc/nginx/uwsgi_params该文件内容如下

uwsgi_param  QUERY_STRING       $query_string;

uwsgi_param  REQUEST_METHOD     $request_method;

uwsgi_param  CONTENT_TYPE       $content_type;

uwsgi_param  CONTENT_LENGTH     $content_length;

uwsgi_param  REQUEST_URI        $request_uri;

uwsgi_param  PATH_INFO          $document_uri;

uwsgi_param  DOCUMENT_ROOT      $document_root;

uwsgi_param  SERVER_PROTOCOL    $server_protocol;

uwsgi_param  REQUEST_SCHEME     $scheme;

uwsgi_param  HTTPS              $https if_not_empty;

uwsgi_param  REMOTE_ADDR        $remote_addr;

uwsgi_param  REMOTE_PORT        $remote_port;

uwsgi_param  SERVER_PORT        $server_port;

uwsgi_param  SERVER_NAME        $server_name;

nginx重载配置文件或重启nginx

[root@bogon helloworld]# nginx -s reload

在另一个终端中查看结果,执行

[root@bogon hellowold]# curl http://127.0.0.1:1001 |grep successfully

<h2>The install worked successfully! Congratulations!</h2>

3.1 uwsgi进程线程数配置优化

3.1.1 uwsgi进程线程数配置

配置文件路径/home/my_projects/helloworld/uwsgi.ini

修改processes进程数与threads进程数

3.1.2测试服务器配置与结果报表

 

3.1.2测试结论

当进程数为4线程数为60时平均吞吐量最高,错误率也相对较低。所以得出结论当进程数为CPU核数2倍时,效率最高。

3.2 Django数据库连接参数CONN_MAX_AGE优化

3.2.1 CONN_MAX_AGE参数配置

路径/home/my_projects/helloworld/helloworld/settings.py database参数修改为以下内容

DATABASES = {

"default": {

"ENGINE": "django.db.backends.mysql",

"NAME": "integral_wall_sandbox",

"USER": "root",

"PASSWORD": "ztwl",

"HOST": "127.0.0.1",

"PORT": "3306",

"CONN_MAX_AGE": 28800, # 8 hours

},

}

测试报表

测试结论

1.当django数据库连接中配置CONN_MAX_AGE参数时,只有views业务中从sql执行开始到本次请求结束的时间超过CONN_MAX_AGE时会立即断开与数据库的连接。如果执行时间少于CONN_MAX_AGE不会断开连接。

2.当django数据库连接中不配置CONN_MAX_AGE参数时,views业务中执行sql时会建立起与数据库的连接,数据库操作结束后立即断开连接。

3.配置了CONN_MAX_AGE参数省去了频繁连接与断开数据的时间,所以服务器吞吐量略大于不配置CONN_MAX_AGE时的吞吐量。

4.uwsgi实际启动进程数=1个主进程+uwsgi配置的进程数;主进程始终1个线程,子进程中的线程始终为uwsgi中配置的线程数。

3.3 nginx中worker_rlimit_nofile与worker_connections优化

3.3.1 worker_rlimit_nofile解释

本参数为nginx工作进程改变打开最多文件描述符数目的限制。用来在不重启主进程的情况下增加限制,小于等于unix系统打开文件最大进程数的配置

3.3.2 当worker_rlimit_nofile不配置或者配置量过小,在高并发下可能出现的异常

异常日志路径/var/log/nginx/helloworld/nginx.access.log

异常日志:socket() failed (24: Too many open files) while connecc ting to upstream

3.3.3 worker_connections含义

每一个worker进程能并发处理(发起)的最大连接数。应小于等于worker_rlimit_nofile值。

3.3.4 当worker_connections配置量过小,在高并发下可能出现的异常

异常日志路径/var/log/nginx/helloworld/nginx.access.log

异常日志:socket() failed (24: Too many open files) while connecc ting to upstream

 

 

 

 

 

 

 

上一篇:【session】


下一篇:Python学习笔记(四)函数式编程