搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

LVS-master:192.168.0.210

LVS-backup:192.168.0.211

LVS-VIP:192.168.0.209

nginx+tomcat:192.168.0.212

nginx+tomcat:192.168.0.227

安装nginx所需包:

Nginx-1.6.0.tar.gzpcre-8.35.zip

一.安装pcre-8.35

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #unzip pcre-8.35.zip
2 #cd pcre-8.35
3 #./configure
4 #make
5 #make install
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

二.安装nginx

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #tar -zxvf nginx-1.6.0.tar.gz
2 #cd nginx-1.6.0
3 #./configure --with-http_stub_status_module --with-http_ssl_module
4 #make
5 #make install
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

在第三部检查编译安装环境,提示缺少什么包就安装什么包

例:提示错误:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 SSL modules require the OpenSSL library
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

解决办法为安装openssl

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 yum –y install openssl openssl-devel
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

nginx安装成功后安装目录为/usr/local/nginx

三.为nginx单独配置用户,并为tomcat设置单独的目录。安全性和可维护性高

1.tomcat的目录设置为/home/www/web(没有该目录就新建)

更改tomcat的server.xml文件,在Host name="localhost"处将appBase=""的指向路径改为/home/www/web

2.创建用户www和用户组www为该文件目录的使用权限者

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #/usr/sbin/groupadd www
2 #/usr/sbin/useradd -g www www -s /sbin/nologin
3 #mkdir -p /home/www
4 #chmod +w /home/www
5 #chown -R www:www/home/www
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

四.配置nginx

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #cd /usr/local/nginx
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

在conf文件夹中新建proxy.conf,用于配置一些代理参数,内容如下:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
 1 # proxy.conf
 2 proxy_redirect            off;
 3 proxy_set_header          Host $host;
 4 proxy_set_header          X-Real-IP $remote_addr;  
 5 client_max_body_size      10m;
 6 client_body_buffer_size   128k;
 7 proxy_connect_timeout     90;
 8 proxy_send_timeout        90;
 9 proxy_read_timeout        90;
10 proxy_buffer_size         4k;
11 proxy_buffers             4 32k;
12 proxy_busy_buffers_size   64k;
13 proxy_temp_file_write_size 64k;
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

修改conf中的nginx.conf,配置如下

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
  1 user  www www;
  2 worker_processes  8;
  3 
  4 error_log   /usr/local/nginx/logs/error.log;
  5 error_log   /usr/local/nginx/logs/error.log  notice;
  6 error_log   /usr/local/nginx/logs/error.log  info;
  7 
  8 pid         /usr/local/nginx/logs/nginx.pid;
  9 
 10 #工作模式及连接数上限 
 11 events {
 12     use epoll;
 13     
 14     worker_connections  65535;
 15 }
 16 
 17 #设定http服务器,利用它的反向代理功能提供负载均衡支持 
 18 http {
 19     #设定mime类型   
 20     include       mime.types;
 21     default_type  application/octet-stream;
 22     include       /usr/local/nginx/conf/proxy.conf;
 23     #log_format  main  ‘$remote_addr - $remote_user [$time_local] "$request" ‘
 24     #                  ‘$status $body_bytes_sent "$http_referer" ‘
 25     #                  ‘"$http_user_agent" "$http_x_forwarded_for"‘;
 26 
 27     #access_log  logs/access.log  main;
 28 
 29     #设定请求缓冲
 30     server_names_hash_bucket_size  128;
 31     client_header_buffer_size   32K;
 32     large_client_header_buffers  4 32k;
 33     # client_max_body_size   8m;
 34 
 35     sendfile        on;
 36     tcp_nopush      on;
 37     tcp_nodelay     on;
 38 
 39     #keepalive_timeout  0;
 40     keepalive_timeout  65;
 41 
 42     gzip  on;
 43     gzip_min_length  1k;
 44     gzip_buffers   4 16k;
 45     gzip_http_version  1.1;
 46     gzip_comp_level  2;
 47     gzip_types  text/plain application/x-javascript text/css  application/xml;
 48     gzip_vary on;    
 49 
 50      server {
 51       server_name _;
 52       return 404;
 53      }
 54     #此处为你tomcat的地址,可以写多个tomcat地址
 55     upstream tomcat_pool {
 56       # server tomcat地址:端口号 weight=4 max_fails=2 fail_timeout=30s;
 57        server 192.168.0.212:8080 weight=4 max_fails=2 fail_timeout=30s;
 58     }    
 59 
 60     server {
 61         listen       80;
 62         server_name  www.yourdomain.com;#此处替换为你自己的网址,如有多个中间用空格
 63         index jump.jsp index.htm index.html  index.do;#设定访问的默认首页地址     
 64         root /home/www/web/ROOT; #设定网站的资源存放路径 
 65 
 66         #charset koi8-r;
 67 
 68         #access_log  logs/host.access.log  main;
 69 
 70        # location / {
 71        #     root   html;
 72        #     index  index.html index.htm;
 73        # }
 74 
 75         location ~ \.(jsp|jspx|dp)?$ #所有JSP的页面均交由tomcat处理
 76         {
 77             proxy_set_header  Host $host;
 78             proxy_set_header  X-Real-IP $remote_addr; 
 79             proxy_pass http://tomcat_pool;#转向tomcat处理
 80         }

 81         #设定访问静态文件直接读取不经过tomcat
 82         location ~ .*\.(htm|html|gif|jpg|jpeg|png|bmp|swf|ioc|rar|zip|txt|flv|mid|doc|ppt|pdf|xls|mp3|wma)$ 
 83          {
 84              expires  30d;
 85          }
 86 
 87          location ~ .*\.(js|css)?$
 88          {
 89               expires  1h;
 90          }
 91         log_format  access  ‘$remote_addr - $remote_user [$time_local] "$request" ‘
 92               ‘$status $body_bytes_sent "$http_referer" ‘
 93               ‘"$http_user_agent" $http_x_forwarded_for‘;
 94     access_log  /usr/local/nginx/logs/ubitechtest.log access;#设定访问日志的存放路径     
 95 
 96 
 97 
 98         error_page   500 502 503 504  /50x.html;
 99         location = /50x.html {
100             root   html;
101         }
102 
103 
104         # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
105         #
106         #location ~ \.php$ {
107         #    root           html;
108         #    fastcgi_pass   127.0.0.1:9000;
109         #    fastcgi_index  index.php;
110         #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
111         #    include        fastcgi_params;
112         #}
113 
114         # deny access to .htaccess files, if Apache‘s document root
115         # concurs with nginx‘s one
116         #
117         #location ~ /\.ht {
118         #    deny  all;
119         #}
120     }
121 
122     server {
123         listen       80;
124         server_name  bbs.yourdomain.com;
125         location / {
126             root /home/www/web/springmvc; #设定网站的资源存放路径 
127             index index.jsp index.htm index.html  index.do welcome.jsp;#设定访问的默认首页地址   
128         }
129 
130         location ~ \.(jsp|jspx|dp)?$ #所有JSP的页面均交由tomcat处理
131         {
132             proxy_set_header  Host $host;
133             proxy_set_header  X-Real-IP $remote_addr; 
134             proxy_pass http://tomcat_pool;#转向tomcat处理
135         }
136         #设定访问静态文件直接读取不经过tomcat
137         location ~ .*\.(htm|html|gif|jpg|jpeg|png|bmp|swf|ioc|rar|zip|txt|flv|mid|doc|ppt|pdf|xls|mp3|wma)$ 
138          {
139              expires  30d;
140          }
141 
142          location ~ .*\.(js|css)?$
143          {
144               expires  1h;
145          }
146         log_format  jsp  ‘$remote_addr - $remote_user [$time_local] "$request" ‘
147               ‘$status $body_bytes_sent "$http_referer" ‘
148               ‘"$http_user_agent" $http_x_forwarded_for‘;
149         access_log  /usr/local/nginx/logs/ubitechztt.log jsp;#设定访问日志的存放路径     
150 
151 
152 
153         error_page   500 502 503 504  /50x.html;
154         location = /50x.html {
155             root   html;
156         }
157 
158     }
159 
160     # another virtual host using mix of IP-, name-, and port-based configuration
161     #
162     #server {
163     #    listen       8000;
164     #    listen       somename:8080;
165     #    server_name  somename  alias  another.alias;
166 
167     #    location / {
168     #        root   html;
169     #        index  index.html index.htm;
170     #    }
171     #}
172 
173 }
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

nginx启动时提示错误:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 /usr/local/nginx/sbin/nginx -t
2 /usr/local/nginx/sbin/nginx: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

解决办法为:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
ln -s /usr/local/lib/libpcre.so.1 /lib64
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

32位系统则:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 ln -s /usr/local/lib/libpcre.so.1 /lib
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

附nginx常用命令

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
 1 #检测配置文件
 2 #/usr/local/nginx/sbin/nginx -t 
 3 #启动nginx
 4  /usr/local/nginx/sbin/nginx 
 5  echo "/usr/local/nginx/sbin/nginx" >> /etc/rc.local
 6 #查看nginx主进程号
 7 #ps -ef | grep "nginx: master process" | grep -v "grep" | awk -F ‘ ‘ ‘{print $2}‘
 8 #停止nginx
 9 #/usr/local/nginx/sbin/nginx -s stop
10 #平滑重启
11 #/usr/local/nginx/sbin/nginx -s reload
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

测试时为在/home/www/web中新建一个index.jsp

启动tomcat,启动nginx

通过访问ip地址即可看到,nginx+tomcat配置完毕。

 

 

五.配置lvs+keepalived

1.安装LVS前系统需要安装popt-static,kernel-devel,make,gcc,openssl-devel,lftp,libnl*,popt*

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #yum –y install popt-static,kernel-devel,make,gcc,openssl-devel,lftp,libnl*,popt*
2 #ln -s /usr/src/kernels/2.6.32-431.17.1.el6.x86_64/ /usr/src/linux
3 #tar -zxvf ipvsadm-1.26.tar.gz
4 #cd ipvsadm-1.26
5 #make && make install
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

2.安装keepalived

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
 1 #wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz
 2 #tar –zxvf keepalived-1.2.13.tar.gz
 3 #cd keepalived-1.2.13
 4 #./configure
 5 #make && make install
 6 ######### 将keepalived做成启动服务,方便管理##########
 7 # cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
 8 # cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
 9 # mkdir /etc/keepalived/
10 # cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
11 # cp /usr/local/sbin/keepalived /usr/sbin/
12 # service keepalived start | stop
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

3.开启路由转发

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #vi /etc/sysctl.conf
2 #sysctl –p
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

4.配置keepalived

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #vi /etc/keepalived/keepalived.conf
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

keepalive.conf具体如下:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
 1 ! Configuration File for keepalived
 2 
 3 global_defs {
 4    notification_email {
 5       cong.chen@czubitech.com
 6    }
 7    notification_email_from cong.chen@czubitech.com
 8    smtp_server send.one.com
 9    smtp_connect_timeout 30
10    router_id LVS_MASTER #备份服务器上将MASTER改为BACKUP 
11 }
12 
13 vrrp_instance VI_1 {
14     state MASTER #备份服务器上将MASTER改为BACKUP 
15     interface eth0  #该网卡名字需要查看具体服务器的网口
16     virtual_router_id 51
17     priority 100 # 备份服务上将100改为90
18     advert_int 1
19     authentication {
20         auth_type PASS
21         auth_pass 1111
22     }
23     virtual_ipaddress {
24         192.168.0.209
25          #(如果有多个VIP,继续换行填写.)
26     }
27 }
28 
29 virtual_server 192.168.0.209 80 {
30     delay_loop 6   #(每隔6秒查询realserver状态)
31     lb_algo rr   #(rr 算法)
32     lb_kind DR      #(Direct Route)
33     nat_mask 255.255.255.0
34    # persistence_timeout 50   #(同一IP的连接60秒内被分配到同一台realserver)
35     protocol TCP    #(用TCP协议检查realserver状态)
36 
37     real_server 192.168.0.212 80 {
38         weight 1   #(权重)
39         TCP_CHECK {
40             connect_timeout 10    #(10秒无响应超时)
41             nb_get_retry 3
42             delay_before_retry 3
43             connect_port 80
44         }
45      }
46      real_server 192.168.0.227 80 {
47         weight 1
48         TCP_CHECK {
49             connect_timeout 10
50             nb_get_retry 3
51             delay_before_retry 3
52             connect_port 80
53         }
54      }
55     
56 }
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

需要注意的是{前面需要有空格,我在配置时TCP_CHECK没有空格导致无法找到real_server

5.配置realserver(即tomcat+nginx那物理机)

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #vi /etc/init.d/lvs_real.sh
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

lvs_real.sh具体配置如下:

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
 1 # description: Config realserver lo and apply noarp
 2  
 3 SNS_VIP=192.168.0.209
 4  
 5 /etc/rc.d/init.d/functions
 6  
 7 case "$1" in
 8 start)
 9        ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
10        /sbin/route add -host $SNS_VIP dev lo:0
11        echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
12        echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
13        echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
14        echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
15        sysctl -p >/dev/null 2>&1
16        echo "RealServer Start OK"
17  
18        ;;
19 stop)
20        ifconfig lo:0 down
21        route del $SNS_VIP >/dev/null 2>&1
22        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
23        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
24        echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
25        echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
26        echo "
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群
1 #chmod +x /roo/lvs_real.sh
2 #/etc/init.d/lvs_real.sh start
搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

启动提示权限不够,赋权即可

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

查看网络ifconfig命令

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

6.测试lvs+keepalived

Master上输入命令ip add

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

VIP绑定成功,ipvsadm可以查看212和227这2台物理机连接情况

Backup上输入命令

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

 

解析域名,测试访问,LVS转发,在master上停掉keepalived

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

经历延迟后继续可以访问,同时在backup上输入命令ip add可以查看vip已经转至backup机器上

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

重启Master上的服务,又重新切换回来

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

Jsp集群中session共享使用memcached,具体配置可参考

http://www.cnblogs.com/cc1688/p/3772156.html

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群,布布扣,bubuko.com

搭建LVS+Keepalived+nginx+tomcat高可用性,高性能jsp集群

上一篇:LAMP编译安装之Apache+php+mysql


下一篇:谷歌Web中文开发手册:3响应式