jboss eap 6.3 集群(cluster)配置

上一篇继续,Domain模式解决了统一管理多台jboss的问题,今天我们来学习如何利用mod_cluster来实现负载均衡、容错。

mod_cluster是jboss的一个开源集群模块(基于apache 2.2.x),官网地址为http://mod-cluster.jboss.org/ ,下面是使用步骤:

一、从官网下载binary文件

http://mod-cluster.jboss.org/downloads/1-2-6-Final-bin 这上面有各种OS的版本,本文以windows为试验环境,应该下载http://downloads.jboss.org/mod_cluster//1.2.6.Final/windows/mod_cluster-1.2.6.Final-windows-amd64.zip 这个版本

(注:如果是linux环境,应该下载http://downloads.jboss.org/mod_cluster//1.2.6.Final/linux-x86_64/mod_cluster-1.2.6.Final-linux2-x64.tar.gz)

二、解压安装

将下载下来的文件解压到jboss master server机器上的某个目录(假定master server的IP为172.21.129.181),比如C:\Apache2.2\ ,mod_cluster其实就是一个包含集群模块的apache server,如果你的机器上之前有安装过apache server,且自动安装成了windows服务,强烈建议先卸载,否则启动mod_cluster对应的apache时,有可能启动的是之前的版本。

注:如果是linux环境安装,强烈建议将压缩包中的httpd目录,解压到/opt/jboss/目录下,因为linux版本的很多脚本和配置里,都是写死这个目录,如果修改了目录,apachectl、http.conf这些文件里的路径全都要改,十分麻烦

2.1 先执行C:\Apache2.2\bin\installconf.bat 生成默认配置 (注:linux环境下安装时,无需执行该步骤,直接进入2.2改http.conf文件即可)

2.2 修改C:\Apache2.2\conf\httpd.conf

a)

#Listen 12.34.56.78:80

Listen 80

40行,把原来的端口8000,改成80,这样方便直接用 http://172.21.129.181/xxx 来访问应用

b)

164行改成

ServerName 172.21.129.181:80

c)

最后的mod_cluster部分,参考下面修改

# MOD_CLUSTER_ADDS
# Adjust to you hostname and subnet.
<IfModule manager_module>
Listen 172.21.129.181:6666
ManagerBalancerName mycluster
<VirtualHost 172.21.129.181:6666>
<Location />
Order deny,allow
#Deny from all
Allow from all
</Location> KeepAliveTimeout 300
MaxKeepAliveRequests 0
#ServerAdvertise on http://127.0.0.1:6666
AdvertiseFrequency 5
#AdvertiseSecurityKey secret
#AdvertiseGroup 224.0.1.105:23364
EnableMCPMReceive <Location /mod_cluster_manager>
SetHandler mod_cluster-manager
Order deny,allow
#Deny from all
Allow from all
</Location> </VirtualHost>
</IfModule>

主要是把127.0.0.1,改成master server对应的真实IP,另外把Deny from all注释掉,把紧贴着的一行改成Allow from all,去掉这些访问上的限制

下面是完整的http.conf参考内容:

 ServerRoot "C:/Apache2.2"
Listen 80
LoadModule actions_module modules/mod_actions.so
LoadModule alias_module modules/mod_alias.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule dir_module modules/mod_dir.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so ServerAdmin jimmy@localhost.localdomain
ServerName 172.21.129.181:80
DocumentRoot "C:/Apache2.2/htdocs" <Directory />
Options FollowSymLinks
AllowOverride None
Order deny,allow
Allow from all
</Directory> <Directory "C:/Apache2.2/htdocs">
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory> <IfModule dir_module>
DirectoryIndex index.html
</IfModule> <FilesMatch "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</FilesMatch> ErrorLog "logs/error_log"
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common <IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog "logs/access_log" common
</IfModule> <IfModule alias_module>
ScriptAlias /cgi-bin/ "C:/Apache2.2/cgi-bin/"
</IfModule> <Directory "C:/Apache2.2/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory> DefaultType text/plain <IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
</IfModule> LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule manager_module modules/mod_manager.so
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule advertise_module modules/mod_advertise.so # MOD_CLUSTER_ADDS
<IfModule manager_module>
Listen 172.21.129.181:6666
ManagerBalancerName mycluster
<VirtualHost 172.21.129.181:6666>
<Location />
Order deny,allow
Allow from all
</Location>
KeepAliveTimeout 300
MaxKeepAliveRequests 0
AdvertiseFrequency 5
EnableMCPMReceive
<Location /mod_cluster_manager>
SetHandler mod_cluster-manager
Order deny,allow
Allow from all
</Location>
</VirtualHost>
</IfModule>

2.3 将httpd安装成windows服务

C:\Apache2.2\bin\httpd -k install

再启动apache

C:\Apache2.2\bin\httpd -k start

顺利的话,浏览http://172.21.129.181:6666/mod_cluster_manager 应该能看到类似以下界面:(注:不要用google chrome浏览器,chrome打不开这个地址,不知道是啥原因,换成IE或Firefox就可以了)

jboss eap 6.3 集群(cluster)配置

这是很关键的一个界面,用来管理/监控所有节点,如果没成功,请仔细检查前面的步骤是否有误。

三、以domain模式启动所有jboss master/slave server

参考上一篇的做法,启动jboss master server / slave server,完成后,刷新 http://172.21.129.181:6666/mod_cluster_manager,会看到多出了好多节点

jboss eap 6.3 集群(cluster)配置

如果没有新的节点显示在该界面,检查jboss master server中创建的group,对应的profile是否为full-ha或ha

点击相关的链接,还能看到更详细的信息

jboss eap 6.3 集群(cluster)配置

下面是点击 show INFO output后的内容:

Node: [1],Name: d2c49277-af22-32b1-95be-b70e3ff20237,Balancer: mycluster,LBGroup: ,Host: 172.21.129.128,Port: 8010,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [2],Name: 708b3ee3-fd4d-39eb-b24c-3384bd11958c,Balancer: mycluster,LBGroup: ,Host: 172.21.129.128,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [3],Name: c4051fb6-9cc9-30aa-967b-6ba1294026b8,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8011,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [4],Name: 17181717-8b5c-3289-b6ea-fedcee2a27cb,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8012,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 3,Read: 3355,Transfered: 0,Connected: 0,Load: 100
Node: [5],Name: 2465cf8d-9cb0-3248-b335-09c02f13b6fe,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1
Node: [6],Name: 68a1e90f-6645-38e0-8157-219ab2004148,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8010,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1
Node: [7],Name: b403bcec-5eb5-37a2-a956-51d273b87548,Balancer: mycluster,LBGroup: ,Host: 172.21.129.57,Port: 8011,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Node: [8],Name: 6e5ac603-5f19-3c90-8f97-d0bf01124c94,Balancer: mycluster,LBGroup: ,Host: 172.21.129.57,Port: 8012,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100
Vhost: [2:1:1], Alias: example.com
Vhost: [2:1:2], Alias: localhost
Vhost: [2:1:3], Alias: default-host
Vhost: [4:1:4], Alias: default-host
Vhost: [4:1:5], Alias: localhost
Vhost: [4:1:6], Alias: example.com
Vhost: [1:1:7], Alias: example.com
Vhost: [1:1:8], Alias: localhost
Vhost: [1:1:9], Alias: default-host
Vhost: [3:1:10], Alias: default-host
Vhost: [3:1:11], Alias: localhost
Vhost: [3:1:12], Alias: example.com
Vhost: [6:1:13], Alias: localhost
Vhost: [6:1:14], Alias: default-host
Vhost: [7:1:15], Alias: example.com
Vhost: [8:1:16], Alias: example.com
Vhost: [8:1:17], Alias: localhost
Vhost: [8:1:18], Alias: default-host
Vhost: [7:1:19], Alias: localhost
Vhost: [7:1:20], Alias: default-host
Context: [2:1:1], Context: /SSO, Status: ENABLED
Context: [4:1:2], Context: /SSO, Status: ENABLED
Context: [1:1:3], Context: /SSO, Status: ENABLED
Context: [3:1:4], Context: /SSO, Status: DISABLED
Context: [7:1:5], Context: /SSO, Status: ENABLED
Context: [8:1:6], Context: /SSO, Status: ENABLED

注意一下:虽然我们有8台jboss server,但是mod_cluster貌似最多只能支持6个node的负载均衡,有2个node的转发因子是-1,即未生效(但大多数情况下,已经够用了)。mod_cluster还有其它一些限制,详情可参考 http://docs.jboss.org/mod_cluster/1.2.0/html_single/#d0e85

四、负载均衡、容错验证

在上一篇里,已经在my-group-1、my-group-2上部署了一个名为SSO的应用,现在可以直接用 http://172.21.129.181/SSO/ 来访问了,apache会智能转发到6个节点中的某一个,如果访问正常,说明负载均衡已经生效。

继续把其它server停掉几台,再继续访问http://172.21.129.181/SSO/ ,如果仍然正常,说明某几个node损坏,并不影响正常使用,即容错成功。

所有server全shutdown后,http://172.21.129.181/SSO/ 将无法访问。

最后附一张:架构图

jboss eap 6.3 集群(cluster)配置

参考文章:

[1] mod_cluster Documentation

[2] JBoss AS7 cluster with mod_cluster setup howto

[3] 使用mod_cluster进行apache httpd server和jboss eap 6.1集群配置

上一篇:python 从SocketServer到 WSGIServer 源码分析、


下一篇:day30 python学习 struct模块和 subprocess 模块