kxcontroller1 kxcontroller2 controller1
这3台作角色
确保服务器的域名解析访问正常
确保NTP时间同步
配置HAproxy主备访问，其中的一项数据库3306端口项，参考Haproxy配置文档。
每台安装如下包
----------------------------------------------------------------------
201511月的安装包如下
yum install -y mariadb-galera-server xinetd rsync mariadb mariadb-galera-common galera MySQL-python
-----------------------------------------------------------------------
20160620 数据库集群安装方式变了，缺省只需要2个包：mariadb mariadb-galera-common
新版的常规安装只需要用到2个包，以及方法如下
vi /etc/yum.repos.d/Galera.repo
[mariadb]
name = MariaDB Galera Cluster
baseurl = http://yum.mariadb.org/5.5-galera/centos7-amd64/
gpgkey = https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck = 1
--------------------------------------------------------------------
yum install galera MariaDB-Galera-server xinetd MySQL-python -y
但我们还需要用到xinetd MySQL-python ，第1个包是用来心跳检测，第2个是python访问mysql的插件，我们知道，心跳检测不是简单的telnet 3306端口就完事了，数据库集群异常的判断是你能能否登录数据库；
配置数据之间的心跳参数,三台服务器都配，配置的方式都是自己检测自己，所以下文会提及到一个普通登录检查的用户clustercheck
# vi /etc/sysconfig/clustercheck
MYSQL_USERNAME="clustercheck"
MYSQL_PASSWORD="venicchina"
MYSQL_HOST="localhost"
MYSQL_PORT="3306"
# vi /etc/xinetd.d/galera-monitor
service galera-monitor
{
port = 9200
disable = no
socket_type = stream
protocol = tcp
wait = no
user = root
group = root
groups = yes
server = /usr/bin/clustercheck
type = UNLISTED
per_source = UNLIMITED
log_on_success =
log_on_failure = HOST
flags = REUSE
}
初始守护进程，以及启动xinetd的9200端口
# systemctl daemon-reload
# systemctl enable xinetd
# systemctl start xinetd
# systemctl enable xinetd
# systemctl start xinetd
配置三台服务器数据配置文件
# vi /etc/my.cnf
[mysqld]
datadir=/home/local/mysql/data
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
max_connections = 9999
[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid
#
# include all files from the config directory
#
!includedir /etc/my.cnf.d
新版本的mysql+centos7，要支持max_connections = 9999 参数还需要额外改多一个地方来支持外部访问并发数
vi /usr/lib/systemd/system/mariadb.service
在以下增加红色字体部份,意在修改普通用户的文件句柄数和可使用的进程数
[Service]
Type=simple
User=mysql
Group=mysql
LimitNOFILE=65535
LimitNPROC=65535
创建三台服务器的目录
# mkdir -p /home/local/mysql/data/
# mkdir -p /home/local/mysql/log/
# chown -R mysql:mysql /home/local/mysql
配置三台服务器数据配置文件
# vi /etc/my.cnf.d/galera.cnf
[mysqld]
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8
#根据每台服务器进行修改
bind-address=10.40.42.1
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_name="my_wsrep_cluster"
wsrep_slave_threads=1
wsrep_certify_nonPK=1
wsrep_max_ws_rows=131072
wsrep_max_ws_size=1073741824
wsrep_debug=0
wsrep_convert_LOCK_to_trx=0
wsrep_retry_autocommit=1
wsrep_auto_increment_control=1
wsrep_drupal_282555_workaround=0
wsrep_causal_reads=0
wsrep_notify_cmd=
wsrep_sst_method=rsync
wsrep_cluster_address = "gcomm://kxcontroller1,kxcontroller2,controller1"
#根据每台服务器进行修改
wsrep_node_name = controller1
#根据每台服务器进行修改
wsrep_node_address=10.40.42.1
此时集群配置后，无需按*版本提供的配置文件/etc/my.cnf.d/mariadb_openstack.cnf，因为它提到的配置，以上两个文件均有体现
启动第一台kxcontroller1数据库
(如第一步有报错，请检查/home/local/mysql目录的权限所有者是否为mysql,改之
/usr/libexec/mysqld --wsrep-new-cluster --user=root &
观察日志：
[root@node4 ~]# tail -f /var/log/mariadb/mariadb.log
150701 19:54:17 [Note] WSREP: wsrep_load(): loading provider library 'none'
150701 19:54:17 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.40-MariaDB-wsrep' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server, wsrep_25.11.r4026
150701 19:54:17 [Note] WSREP: wsrep_load(): loading provider library 'none'
150701 19:54:17 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.5.40-MariaDB-wsrep' socket: '/var/lib/mysql/mysql.sock' port: 3306 MariaDB Server, wsrep_25.11.r4026
出现ready for connections,证明我们启动成功
再初始第一台数据库
# mysql_secure_installation （初始化过程中，设置root密码venic8888 开启远程访问）
接下来另外两台（kxcontroller2 controller1）进行
systemctl start mariadb
可以查看/var/log/mariadb/mariadb.log，在日志可以看到节点均加入了集群中。
警告⚠：--wsrep-new-cluster这个参数只能在初始化集群使用，且只能在一个节点使用，且一般使用一次即可完成。
MariaDB [(none)]> show status like 'wsrep_%';
+------------------------------+---------------------------------------------------+
| Variable_name | Value |
+------------------------------+---------------------------------------------------+
| wsrep_local_state_uuid | de6a8351-b5fa-11e5-a450-cebe9a9fd835 |
| wsrep_protocol_version | 5 |
| wsrep_last_committed | 2 |
| wsrep_replicated | 0 |
| wsrep_replicated_bytes | 0 |
| wsrep_repl_keys | 0 |
| wsrep_repl_keys_bytes | 0 |
| wsrep_repl_data_bytes | 0 |
| wsrep_repl_other_bytes | 0 |
| wsrep_received | 6 |
| wsrep_received_bytes | 1146 |
| wsrep_local_commits | 0 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.000000 |
| wsrep_local_cached_downto | 18446744073709551615 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_cert_deps_distance | 0.000000 |
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 0.000000 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 0.000000 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_cert_index_size | 0 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.000000 |
| wsrep_incoming_addresses | 10.120.42.1:3306,10.120.42.2:3306,10.40.42.1:3306 | #关注这个加入集群的主机IP，就是对应的这3台机器的IP
| wsrep_cluster_conf_id | 16 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | de6a8351-b5fa-11e5-a450-cebe9a9fd835 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 2 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 3.5(rXXXX) |
| wsrep_ready | ON |
| wsrep_thread_count | 2 |
+------------------------------+---------------------------------------------------+
测试，数据库stop其中1台观察集群show status like 'wsrep_%';状态。
接下可测试数据库创建等操作。
三台服务器加入开机自启。
systemctl enable mariadb
使用HA proxy网页服务器，http://10.40.42.10:8008/admin-venic 观察代理是否正常
还差一步，进入其中一台数据库。执行。mysql -u root -p
创建一个专用于心跳的用户，参考以上 vi /etc/sysconfig/clustercheck配置的用户名和密码
CREATE USER 'clustercheck'@'localhost' IDENTIFIED BY 'venicchina';
flush privileges;
其它两个数据库查看是否创建成功这个用户
SELECT DISTINCT CONCAT('User: ''',user,'''@''',host,''';') AS query FROM mysql.user;
再使用HA proxy网页服务器，http://10.40.42.10:8008/admin-venic观察状态是否变绿。
--------------------------------------------------------------------------------------------
重点
服务器重启时，不建议全部同时重启数据库。不然，
systemctl start mariadb会命令全部失效
如发生此情况。应按以上的方法先启用第一台kxcontroller
#/usr/libexec/mysqld --wsrep-new-cluster --user=root &
再其它台
systemctl start mariadb
状态同步后，你可以保持这个状态，如果你跟我一样有强迫症的话，可选择在第1台kxcontroller1，kill -9 杀死mysql进程，第1台就可以使用命令，命令就相同了
systemctl start mariadb
-------------------------------------------------------------------------------------------
其它额外配置。赋予远程登录的权限和密码
grant all privileges on *.* to root@'%' identified by '高强度密码';
flush privileges;
这是是查询数据库当前设置的最大连接数（每台数据的并发数是独立的，所以上面max_connections = 9999 在一开始都配好，因为openstack数据库并发非常高）
mysql> show variables like '%max_connections%';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 1000 |----默认1000
+-----------------+-------+