jboss eap 6.3 集群(cluster)配置

2022-12-31 15:29:18

接上一篇继续，Domain模式解决了统一管理多台jboss的问题，今天我们来学习如何利用mod_cluster来实现负载均衡、容错。

mod_cluster是jboss的一个开源集群模块(基于apache 2.2.x)，官网地址为http://mod-cluster.jboss.org/ ，下面是使用步骤：

一、从官网下载binary文件

http://mod-cluster.jboss.org/downloads/1-2-6-Final-bin 这上面有各种OS的版本，本文以windows为试验环境，应该下载http://downloads.jboss.org/mod_cluster//1.2.6.Final/windows/mod_cluster-1.2.6.Final-windows-amd64.zip 这个版本

(注：如果是linux环境，应该下载http://downloads.jboss.org/mod_cluster//1.2.6.Final/linux-x86_64/mod_cluster-1.2.6.Final-linux2-x64.tar.gz)

二、解压安装

将下载下来的文件解压到jboss master server机器上的某个目录(假定master server的IP为172.21.129.181)，比如C:\Apache2.2\ ，mod_cluster其实就是一个包含集群模块的apache server，如果你的机器上之前有安装过apache server，且自动安装成了windows服务，强烈建议先卸载，否则启动mod_cluster对应的apache时，有可能启动的是之前的版本。

注：如果是linux环境安装，强烈建议将压缩包中的httpd目录，解压到/opt/jboss/目录下，因为linux版本的很多脚本和配置里，都是写死这个目录，如果修改了目录，apachectl、http.conf这些文件里的路径全都要改,十分麻烦

2.1 先执行C:\Apache2.2\bin\installconf.bat 生成默认配置 (注：linux环境下安装时，无需执行该步骤，直接进入2.2改http.conf文件即可)

2.2 修改C:\Apache2.2\conf\httpd.conf

a)

#Listen 12.34.56.78:80

Listen 80

40行，把原来的端口8000，改成80，这样方便直接用 http://172.21.129.181/xxx 来访问应用

b)

164行改成

ServerName 172.21.129.181:80

c)

最后的mod_cluster部分，参考下面修改

# MOD_CLUSTER_ADDS

# Adjust to you hostname and subnet.

<IfModule manager_module>

  Listen 172.21.129.181:6666

  ManagerBalancerName mycluster

  <VirtualHost 172.21.129.181:6666>

    <Location />

     Order deny,allow

     #Deny from all

     Allow from all

    </Location>

    KeepAliveTimeout 300

    MaxKeepAliveRequests 0

    #ServerAdvertise on http://127.0.0.1:6666

    AdvertiseFrequency 5

    #AdvertiseSecurityKey secret

    #AdvertiseGroup 224.0.1.105:23364

    EnableMCPMReceive

    <Location /mod_cluster_manager>

       SetHandler mod_cluster-manager

       Order deny,allow

       #Deny from all

       Allow from all

    </Location>

  </VirtualHost>

</IfModule>

主要是把127.0.0.1，改成master server对应的真实IP，另外把Deny from all注释掉，把紧贴着的一行改成Allow from all，去掉这些访问上的限制

下面是完整的http.conf参考内容:

 ServerRoot "C:/Apache2.2"

 Listen 80

 LoadModule actions_module modules/mod_actions.so

 LoadModule alias_module modules/mod_alias.so

 LoadModule auth_basic_module modules/mod_auth_basic.so

 LoadModule authz_host_module modules/mod_authz_host.so

 LoadModule autoindex_module modules/mod_autoindex.so

 LoadModule dir_module modules/mod_dir.so

 LoadModule log_config_module modules/mod_log_config.so

 LoadModule mime_module modules/mod_mime.so

 ServerAdmin jimmy@localhost.localdomain

 ServerName 172.21.129.181:80

 DocumentRoot "C:/Apache2.2/htdocs"

 <Directory />

     Options FollowSymLinks

     AllowOverride None

     Order deny,allow

     Allow from all

 </Directory>

 <Directory "C:/Apache2.2/htdocs">

     Options Indexes FollowSymLinks

     AllowOverride None

     Order allow,deny

     Allow from all

 </Directory>

 <IfModule dir_module>

     DirectoryIndex index.html

 </IfModule>

 <FilesMatch "^\.ht">

     Order allow,deny

     Deny from all

     Satisfy All

 </FilesMatch>

 ErrorLog "logs/error_log"

 LogLevel warn

 <IfModule log_config_module>

     LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

     LogFormat "%h %l %u %t \"%r\" %>s %b" common

     <IfModule logio_module>

       LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

     </IfModule>

     CustomLog "logs/access_log" common

 </IfModule>

 <IfModule alias_module>

     ScriptAlias /cgi-bin/ "C:/Apache2.2/cgi-bin/"

 </IfModule>

 <Directory "C:/Apache2.2/cgi-bin">

     AllowOverride None

     Options None

     Order allow,deny

     Allow from all

 </Directory>

 DefaultType text/plain

 <IfModule mime_module>

     TypesConfig conf/mime.types

     AddType application/x-compress .Z

     AddType application/x-gzip .gz .tgz

 </IfModule>

 LoadModule proxy_module modules/mod_proxy.so

 LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

 LoadModule proxy_http_module modules/mod_proxy_http.so

 LoadModule proxy_cluster_module modules/mod_proxy_cluster.so

 LoadModule manager_module modules/mod_manager.so

 LoadModule slotmem_module modules/mod_slotmem.so

 LoadModule advertise_module modules/mod_advertise.so

 # MOD_CLUSTER_ADDS

 <IfModule manager_module>

   Listen 172.21.129.181:6666

   ManagerBalancerName mycluster

   <VirtualHost 172.21.129.181:6666>

     <Location />

      Order deny,allow

      Allow from all

     </Location>

     KeepAliveTimeout 300

     MaxKeepAliveRequests 0

     AdvertiseFrequency 5

     EnableMCPMReceive

     <Location /mod_cluster_manager>

        SetHandler mod_cluster-manager

        Order deny,allow

        Allow from all

     </Location>

   </VirtualHost>

 </IfModule>

2.3 将httpd安装成windows服务

C:\Apache2.2\bin\httpd -k install

再启动apache

C:\Apache2.2\bin\httpd -k start

顺利的话，浏览http://172.21.129.181:6666/mod_cluster_manager 应该能看到类似以下界面：(注：不要用google chrome浏览器，chrome打不开这个地址，不知道是啥原因，换成IE或Firefox就可以了)

jboss eap 6.3 集群(cluster)配置

这是很关键的一个界面，用来管理/监控所有节点，如果没成功，请仔细检查前面的步骤是否有误。

三、以domain模式启动所有jboss master/slave server

参考上一篇的做法，启动jboss master server / slave server，完成后，刷新 http://172.21.129.181:6666/mod_cluster_manager，会看到多出了好多节点

jboss eap 6.3 集群(cluster)配置

如果没有新的节点显示在该界面，检查jboss master server中创建的group，对应的profile是否为full-ha或ha

点击相关的链接，还能看到更详细的信息

jboss eap 6.3 集群(cluster)配置

下面是点击 show INFO output后的内容：

Node: [1],Name: d2c49277-af22-32b1-95be-b70e3ff20237,Balancer: mycluster,LBGroup: ,Host: 172.21.129.128,Port: 8010,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100

Node: [2],Name: 708b3ee3-fd4d-39eb-b24c-3384bd11958c,Balancer: mycluster,LBGroup: ,Host: 172.21.129.128,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100

Node: [3],Name: c4051fb6-9cc9-30aa-967b-6ba1294026b8,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8011,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100

Node: [4],Name: 17181717-8b5c-3289-b6ea-fedcee2a27cb,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8012,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 3,Read: 3355,Transfered: 0,Connected: 0,Load: 100

Node: [5],Name: 2465cf8d-9cb0-3248-b335-09c02f13b6fe,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8009,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1

Node: [6],Name: 68a1e90f-6645-38e0-8157-219ab2004148,Balancer: mycluster,LBGroup: ,Host: 172.21.129.181,Port: 8010,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: -1

Node: [7],Name: b403bcec-5eb5-37a2-a956-51d273b87548,Balancer: mycluster,LBGroup: ,Host: 172.21.129.57,Port: 8011,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100

Node: [8],Name: 6e5ac603-5f19-3c90-8f97-d0bf01124c94,Balancer: mycluster,LBGroup: ,Host: 172.21.129.57,Port: 8012,Type: ajp,Flushpackets: Off,Flushwait: 10,Ping: 10,Smax: 65,Ttl: 60,Elected: 0,Read: 0,Transfered: 0,Connected: 0,Load: 100

Vhost: [2:1:1], Alias: example.com

Vhost: [2:1:2], Alias: localhost

Vhost: [2:1:3], Alias: default-host

Vhost: [4:1:4], Alias: default-host

Vhost: [4:1:5], Alias: localhost

Vhost: [4:1:6], Alias: example.com

Vhost: [1:1:7], Alias: example.com

Vhost: [1:1:8], Alias: localhost

Vhost: [1:1:9], Alias: default-host

Vhost: [3:1:10], Alias: default-host

Vhost: [3:1:11], Alias: localhost

Vhost: [3:1:12], Alias: example.com

Vhost: [6:1:13], Alias: localhost

Vhost: [6:1:14], Alias: default-host

Vhost: [7:1:15], Alias: example.com

Vhost: [8:1:16], Alias: example.com

Vhost: [8:1:17], Alias: localhost

Vhost: [8:1:18], Alias: default-host

Vhost: [7:1:19], Alias: localhost

Vhost: [7:1:20], Alias: default-host

Context: [2:1:1], Context: /SSO, Status: ENABLED

Context: [4:1:2], Context: /SSO, Status: ENABLED

Context: [1:1:3], Context: /SSO, Status: ENABLED

Context: [3:1:4], Context: /SSO, Status: DISABLED

Context: [7:1:5], Context: /SSO, Status: ENABLED

Context: [8:1:6], Context: /SSO, Status: ENABLED

注意一下：虽然我们有8台jboss server，但是mod_cluster貌似最多只能支持6个node的负载均衡，有2个node的转发因子是-1，即未生效(但大多数情况下，已经够用了)。mod_cluster还有其它一些限制，详情可参考 http://docs.jboss.org/mod_cluster/1.2.0/html_single/#d0e85

四、负载均衡、容错验证

在上一篇里，已经在my-group-1、my-group-2上部署了一个名为SSO的应用，现在可以直接用 http://172.21.129.181/SSO/ 来访问了，apache会智能转发到6个节点中的某一个，如果访问正常，说明负载均衡已经生效。

继续把其它server停掉几台，再继续访问http://172.21.129.181/SSO/ ，如果仍然正常，说明某几个node损坏，并不影响正常使用，即容错成功。

所有server全shutdown后，http://172.21.129.181/SSO/ 将无法访问。

最后附一张:架构图

jboss eap 6.3 集群(cluster)配置

参考文章：

[1] mod_cluster Documentation

[2] JBoss AS7 cluster with mod_cluster setup howto

[3] 使用mod_cluster进行apache httpd server和jboss eap 6.1集群配置

上一篇：python 从SocketServer到 WSGIServer 源码分析、


下一篇：day30 python学习 struct模块和 subprocess 模块