四十.创建Redis集群 管理集群
环境准备 准备 6台(51-56) redis服务器 以默认配置运行redis服务即可
一.创建Redis集群
1.启用集群功能( 51-56 都要配置)
]# netstat -antupl |grep :6379（要有东西）
]# /etc/init.d/redis_6379 stop
*一一对应，以52为例*
]# vim /etc/redis/6379.conf
70 bind 192.168.4.52
93 port 6352
815 cluster-enabled yes
823 cluster-config-file nodes-6352.conf
829 cluster-node-timeout 5000
:wq
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis-server
tcp 0 0 192.168.4.52:6352 0.0.0.0:* LISTEN 7469/redis-server 1
tcp 0 0 192.168.4.52:16352 0.0.0.0:* LISTEN 7469/redis-server 1
]# cat /var/lib/redis/6379/nodes-6352.conf 查看集群信息文件
ee0ff6941c920977e98966a80a6b1835ec4944f6 :0@0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0
2 创建集群
2.1 部署ruby脚本执行环境 (192.168.4.51)
]# cd nosql/
]# yum -y install ruby rubygems
]# rpm -ivh ruby-devel-2.0.0.648-30.el7.x86_64.rpm
]# gem install redis-3.2.1.gem
2.2 创建管理集群的ruby脚本
]# echo $PATH
]# mkdir /root/bin
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
]# cp redis-4.0.8/src/redis-trib.rb /root/bin/
]# chmod +x /root/bin/redis-trib.rb
]# redis-trib.rb help
2.3 创建集群
]# redis-trib.rb create --replicas 1 \
> 192.168.4.51:6351 192.168.4.52:6352 \
> 192.168.4.53:6353 192.168.4.54:6354 \
> 192.168.4.55:6355 192.168.4.56:6356
...
Can I set the above configuration? (type 'yes' to accept): yes(输入yes)
...
[OK] All 16384 slots covered.
3 查看集群中主机状态信息
]# cat /var/lib/redis/6379/nodes-6354.conf
]# redis-cli -h 192.168.4.54 -p 6354
> cluster info
> cluster nodes
集群创建失败的解决办法
]# redis-cli -h x.x.x.x -p xxxx shutdown
]# rm -rf /var/lib/redis/6379/*
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis_server
再次执行创建集群命令
4 测试集群 在192.168.4.50 访问集群存取数据(可以连接任意一台主库的ip地址)
]# redis-cli -c -h 192.168.4.51 -p 6351
192.168.4.51:6351> keys *
(empty list or set)
192.168.4.51:6351>
192.168.4.51:6351> set x 101
192.168.4.53:6353> keys *
1) "x"
主库对应的从库，会自动把数据同步到本机。
在管理主机51上 执行如下操作
]# redis-trib.rb info 192.168.4.51:6351 查看数据存储情况
192.168.4.51:6351 (aa919b76...) -> 0 keys | 5461 slots | 1 slaves.
192.168.4.52:6352 (83d43aaf...) -> 0 keys | 5462 slots | 1 slaves.
192.168.4.53:6353 (46a1d2ec...) -> 1 keys | 5461 slots | 1 slaves.
]# redis-trib.rb check 192.168.4.51:6351 查看集群信息
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots: (0 slots) slave
replicates 83d43aaf83697e4f76947c3b4b534912a02b9048
M: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
主库 从库
51 -- 55
52 -- 56
53 -- 54
主库 宕机后，对应的从库会自动升级为主库(52宕机，56主库）
]# redis-cli -h 192.168.4.52 -p 6352 shutdown
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:5461-10922 (5462 slots) master
0 additional replica(s)
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
开启52,变为56的从库
l52 utils]# /etc/init.d/redis_6379 start
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots: (0 slots) slave
replicates 8252253a0e18bdc7151027080e01477967ed1364
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
二.管理集群
准备2台新的虚拟机ip 地址分别为 192.168.4.57 192.168.4.58
运行redis服务并启用的集群配置
]# /etc/init.d/redis_6379 stop
]# vim /etc/redis/6379.conf(52为例子）
70 bind 192.168.4.52
93 port 6352
815 cluster-enabled yes
823 cluster-config-file nodes-6352.conf
829 cluster-node-timeout 5000
:wq
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis-server
2.1 向集群里添加新主机
2.1.1 添加master角色主机 (添加 192.168.4.57，在51操作)
]# redis-trib.rb add-node 192.168.4.57:6357 192.168.4.51:6351
]# redis-trib.rb check 192.168.4.51:6351 查看
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 7ac349e23d72f46967eeb0ef6b022c27396a4388 192.168.4.57:6357
slots: (0 slots) master
0 additional replica(s)
S: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots: (0 slots) slave
replicates 8252253a0e18bdc7151027080e01477967ed1364
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
]# redis-trib.rb reshard 192.168.4.51:6351 重新分片
指定移出hash槽个数 4096
指定接收hash槽主机ID 57主机的ID值
指定移出hash槽主机ID all
同意这样分片 yes
查看分片
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:1365-5460 (4096 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:6827-10922 (4096 slots) master
1 additional replica(s)
M: 7ac349e23d72f46967eeb0ef6b022c27396a4388 192.168.4.57:6357
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
0 additional replica(s)
S: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots: (0 slots) slave
replicates 8252253a0e18bdc7151027080e01477967ed1364
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
客户端连接主库57 可以存取数据
]# redis-cli -c -h 192.168.4.57 -p 6357
192.168.4.57:6357> ping
PONG
192.168.4.57:6357> set key 111
-> Redirected to slot [12539] located at 192.168.4.53:6353
OK
192.168.4.53:6353> get key
"111"
192.168.4.53:6353> keys *
1) "key"
2) "x"
2.1.2 添加slave角色主机 (添加slave 192.168.4.58 51上操作）
]# redis-trib.rb add-node --slave 192.168.4.58:6358 192.168.4.51:6351
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:1365-5460 (4096 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:6827-10922 (4096 slots) master
1 additional replica(s)
M: 7ac349e23d72f46967eeb0ef6b022c27396a4388 192.168.4.57:6357
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
1 additional replica(s)
S: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots: (0 slots) slave
replicates 8252253a0e18bdc7151027080e01477967ed1364
S: a28a8ad17defa60a4169262a909a17e46d5ce699 192.168.4.58:6358
slots: (0 slots) slave
replicates 7ac349e23d72f46967eeb0ef6b022c27396a4388
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
自动分配给57
客户端连接主库58 可以查看到数据
]# redis-cli -c -h 192.168.4.58 -p 6358
192.168.4.58:6358> set t2 333
-> Redirected to slot [4748] located at 192.168.4.51:6351
OK
192.168.4.51:6351> keys *
1) "t2"
192.168.4.51:6351> set t3 444
-> Redirected to slot [685] located at 192.168.4.57:6357
OK
192.168.4.57:6357> keys *
1) "t3"
192.168.4.57:6357> set t4 444
-> Redirected to slot [12874] located at 192.168.4.53:6353
OK
192.168.4.53:6353> keys *
1) "t4"
2) "key"
3) "x"
2.2 从集群里移除主机
2.2.1 移除slave角色主机 (移除192.168.4.58，51操作)
redis-trib.rb del-node 任意主机 被移除主机ID(slave主机没有haxi槽，可直接移除）
]# redis-trib.rb del-node 192.168.4.51:6351 a28a8ad17defa60a4169262a909a17e46d5ce699
]# redis-trib.rb check 192.168.4.51:6351
此时没有58的slave信息
2.2.2 移除master角色主机 (移除 192.168.4.57 51操作)
释放占用的hash槽
]# redis-trib.rb reshard 192.168.4.51:6351
指定移出hash槽个数 4096
指定接收hash槽主机ID 写任意一个主库ID值
Source node #1: 写57主机的ID值
Source node #2: done
同意这样分片 yes
移除master主机
]# redis-trib.rb del-node 192.168.4.51:6351 7ac349e23d72f46967eeb0ef6b022c27396a4388
检测集群
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:0-6826,10923-12287 (8192 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:6827-10922 (4096 slots) master
1 additional replica(s)
S: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots: (0 slots) slave
replicates 8252253a0e18bdc7151027080e01477967ed1364
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
重新平均分配hash槽
]# redis-trib.rb rebalance 192.168.4.51:6351
]# redis-trib.rb check 192.168.4.51:6351 查看
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: aa919b76cfddf094256a228b84f63087dc2b6af3 192.168.4.51:6351
slots:2731-6826,10923-12287 (5461 slots) master
1 additional replica(s)
M: 8252253a0e18bdc7151027080e01477967ed1364 192.168.4.56:6356
slots:0-1365,6827-10922 (5462 slots) master
1 additional replica(s)
S: 83d43aaf83697e4f76947c3b4b534912a02b9048 192.168.4.52:6352
slots: (0 slots) slave
replicates 8252253a0e18bdc7151027080e01477967ed1364
M: 46a1d2ecee4ac309c3557295900182a43dd611d4 192.168.4.53:6353
slots:1366-2730,12288-16383 (5461 slots) master
1 additional replica(s)
S: 1b2535c616fcdc4fec50369e3e0fdc0f998ab041 192.168.4.55:6355
slots: (0 slots) slave
replicates aa919b76cfddf094256a228b84f63087dc2b6af3
S: ca21191e7430151953267f46fc9dc310f7e08b0b 192.168.4.54:6354
slots: (0 slots) slave
replicates 46a1d2ecee4ac309c3557295900182a43dd611d4
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
把redis集群里的主机 恢复为独立的redis服务器(51-58)
]# redis-cli -h 192.168.4.51 -p 6351 shutdown
]# rm -rf /var/lib/redis/6379/*
]# vim /etc/redis/6379.conf (把与集群配置相关的 3项 注释)
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis-server