简介
为何要搭建Redis集群?Redis是在内存中保存数据的,而我们的电脑一般内存都不大,这也就意味着Redis不适合存储大数据,适合存储大数据的是Hadoop生态系统的Hbase或者是MogoDB。Redis更适合处理高并发,一台设备的存储能力是很有限的,但是多台设备协同合作,就可以让内存增大很多倍,这就需要用到集群。
Redis集群搭建的方式有多种,例如使用客户端分片、Twemproxy、Codis等,但从redis 3.0之后版本支持redis-cluster集群,它是Redis官方提出的解决方案,Redis-Cluster采用无中心结构,每个节点保存数据和整个集群状态,每个节点都和其他所有节点连接。其redis-cluster架构图如下:
客户端与 redis 节点直连,不需要中间 proxy 层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可。
所有的 redis 节点彼此互联(PING-PONG 机制),内部使用二进制协议优化传输速度和带宽.
解决的问题:
- 1、单击版redis内存容量有限的问题
- 2、高并发场景下,单击版redis性能不高
- 3、单击版redis不具有高可用性,不能应用于生产环境
分布存储机制-槽
Normal
0
7.8 磅
0
2
false
false
false
EN-US
ZH-CN
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:普通表格;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.5pt;
mso-bidi-font-size:11.0pt;
font-family:"Times New Roman",serif;
mso-font-kerning:1.0pt;}
1,redis-cluster 把所有的物理节点映射到[0-16383]slot 上,cluster 负责维护
node<->slot<->value
2,Redis 集群中内置了 16384 个哈希槽,当需要在 Redis 集群中放置一个 key-value 时,redis 先对 key 使用 crc16 算法算出一个结果,然后把结果对 16384 求余数,这样每个key 都会对应一个编号在 0-16383 之间的哈希槽,redis 会根据节点数量大致均等的将哈希槽映射到不同的节点。
例如三个节点:槽分布的值如下:
- SERVER1: 0-5460
- SERVER2: 5461-10922
- SERVER3: 10923-16383
键到slot的映射算法:HASH_SLOT=CRC16(key)mod16384
Normal
0
7.8 磅
0
2
false
false
false
EN-US
ZH-CN
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:普通表格;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.5pt;
mso-bidi-font-size:11.0pt;
font-family:"Times New Roman",serif;
mso-font-kerning:1.0pt;}
Normal
0
7.8 磅
0
2
false
false
false
EN-US
ZH-CN
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:普通表格;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.5pt;
mso-bidi-font-size:11.0pt;
font-family:"Times New Roman",serif;
mso-font-kerning:1.0pt;}
容错机制-投票
Normal
0
7.8 磅
0
2
false
false
false
EN-US
ZH-CN
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:普通表格;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.5pt;
mso-bidi-font-size:11.0pt;
font-family:"Times New Roman",serif;
mso-font-kerning:1.0pt;}
1,选举过程是集群中所有master参与,如果半数以上master节点与故障节点通信超过(cluster-node-timeout),认为该节点故障,自动触发故障转移操作. 故障节点对应的从节点自动升级为主节点
2,什么时候整个集群不可用(cluster_state:fail)?
如果集群任意master挂掉,且当前master没有slave,集群进入fail状态,也可以理解成集群的slot映射[0-16383]不完整时进入fail状态.
搭建Redis-Cluster
redis cluster 为了保证数据的高可用性,加入了主从模式,一个主节点对应一个或多个从节点,主节点提供数据存取,从节点则是从主节点拉取数据备份,当这个主节点挂掉后,就会有这个从节点选取一个来充当主节点,从而保证集群不会挂掉。所以redis-cluster是需要至少6个redis组成,三个Master(主)A B C,三个Slave(从)A1 B1 C1。严格来说每个redis需要一台机器,但是一般6个reids使用3台机器也是可以的,只要A和A1,B和B1,C和C1不在同一台机器上即可。
伪集群(所有redis安装到同一台机器上)准备工作:
1,安装gcc,ruby
Redis 是 c 语言开发的。安装 redis 需要 c 语言的编译环境。如果没有 gcc 需要在线安装。
yum install gcc-c++ 使用yum命令安装 ruby (我们需要使用ruby脚本来实现集群搭建)
yum install ruby
yum install rubygems
2,把文件放到服务器上并解压
mkdir redis-cluster
cd /usr/local/redis-cluster/
rz 上传redis的安装包
tar -zxvf redis-3.0.0.tar.gz
3,编译redis
cd /usr/local/redis-cluster/redis-3.0.0
make
4,安装6个reids实例
cd /usr/local/redis-cluster/redis-3.0.0
make install PREFIX=/usr/local/redis-cluster/redis-1
make install PREFIX=/usr/local/redis-cluster/redis-2
make install PREFIX=/usr/local/redis-cluster/redis-3
make install PREFIX=/usr/local/redis-cluster/redis-4
make install PREFIX=/usr/local/redis-cluster/redis-5
make install PREFIX=/usr/local/redis-cluster/redis-6
5,复制redis配置文件到6个redis实例中
cd /usr/local/redis-cluster/redis-3.0.0 cp redis.conf /usr/local/redis-cluster/redis-1/bin/
cp redis.conf /usr/local/redis-cluster/redis-2/bin/
cp redis.conf /usr/local/redis-cluster/redis-3/bin/
cp redis.conf /usr/local/redis-cluster/redis-4/bin/
cp redis.conf /usr/local/redis-cluster/redis-5/bin/
cp redis.conf /usr/local/redis-cluster/redis-6/bin/
配置集群
1,修改每个redis实例的配置文件,修改端口以及开启集群能力
vi /usr/local/redis-cluster/redis-1/bin/redis.conf
daemonize yes 后台运行
port 7001 redis实例的端口号
cluster-enabled yes 开启集群能力 vi /usr/local/redis-cluster/redis-2/bin/redis.conf
daemonize yes 后台运行
port 7002 redis实例的端口号
cluster-enabled yes 开启集群能力 vi /usr/local/redis-cluster/redis-3/bin/redis.conf
daemonize yes 后台运行
port 7003 redis实例的端口号
cluster-enabled yes 开启集群能力 vi /usr/local/redis-cluster/redis-4/bin/redis.conf
daemonize yes 后台运行
port 7004 redis实例的端口号
cluster-enabled yes 开启集群能力 vi /usr/local/redis-cluster/redis-5/bin/redis.conf
daemonize yes 后台运行
port 7005 redis实例的端口号
cluster-enabled yes 开启集群能力 vi /usr/local/redis-cluster/redis-6/bin/redis.conf
daemonize yes 后台运行
port 7006 redis实例的端口号
cluster-enabled yes 开启集群能力
2,启动6个redis实例
cd /usr/local/redis-cluster/redis-1/bin/
./redis-server redis.conf cd /usr/local/redis-cluster/redis-2/bin/
./redis-server redis.conf cd /usr/local/redis-cluster/redis-3/bin/
./redis-server redis.conf cd /usr/local/redis-cluster/redis-4/bin/
./redis-server redis.conf cd /usr/local/redis-cluster/redis-5/bin/
./redis-server redis.conf cd /usr/local/redis-cluster/redis-6/bin/
./redis-server redis.conf
3,安装ruby用户搭建redis集群的脚本
cd /usr/local/redis-cluster
rz 上传redis-3.0.0.gem
gem install redis-3.0.0.gem
4,使用ruby脚本搭建redis集群(每个reids保证启动状态)
cd /usr/local/redis-cluster/redis-3.0.0/src
./redis-trib.rb create --replicas 1 192.168.127.137:7001 192.168.127.137:7002 192.168.127.137:7003 192.168.127.137:7004 192.168.127.137:7005 192.168.127.137:7006
>>> Creating cluster
Connecting to node 192.168.25.140:7001: OK
Connecting to node 192.168.25.140:7002: OK
Connecting to node 192.168.25.140:7003: OK
Connecting to node 192.168.25.140:7004: OK
Connecting to node 192.168.25.140:7005: OK
Connecting to node 192.168.25.140:7006: OK
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.25.140:7001
192.168.25.140:7002
192.168.25.140:7003
Adding replica 192.168.25.140:7004 to 192.168.25.140:7001
Adding replica 192.168.25.140:7005 to 192.168.25.140:7002
Adding replica 192.168.25.140:7006 to 192.168.25.140:7003
M: 1800237a743c2aa918ade045a28128448c6ce689 192.168.25.140:7001
slots:0-5460 (5461 slots) master
M: 7cb3f7d5c60bfbd3ab28800f8fd3bf6de005bf0d 192.168.25.140:7002
slots:5461-10922 (5462 slots) master
M: 436e88ec323a2f8bb08bf09f7df07cc7909fcf81 192.168.25.140:7003
slots:10923-16383 (5461 slots) master
S: c2a39a94b5f41532cd83bf6643e98fc277c2f441 192.168.25.140:7004
replicates 1800237a743c2aa918ade045a28128448c6ce689
S: b0e38d80273515c84b1a01820d8ecee04547d776 192.168.25.140:7005
replicates 7cb3f7d5c60bfbd3ab28800f8fd3bf6de005bf0d
S: 03bf6bd7e3e6eece5a02043224497c2c8e185132 192.168.25.140:7006
replicates 436e88ec323a2f8bb08bf09f7df07cc7909fcf81
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 192.168.25.140:7001)
M: 1800237a743c2aa918ade045a28128448c6ce689 192.168.25.140:7001
slots:0-5460 (5461 slots) master
M: 7cb3f7d5c60bfbd3ab28800f8fd3bf6de005bf0d 192.168.25.140:7002
slots:5461-10922 (5462 slots) master
M: 436e88ec323a2f8bb08bf09f7df07cc7909fcf81 192.168.25.140:7003
slots:10923-16383 (5461 slots) master
M: c2a39a94b5f41532cd83bf6643e98fc277c2f441 192.168.25.140:7004
slots: (0 slots) master
replicates 1800237a743c2aa918ade045a28128448c6ce689
M: b0e38d80273515c84b1a01820d8ecee04547d776 192.168.25.140:7005
slots: (0 slots) master
replicates 7cb3f7d5c60bfbd3ab28800f8fd3bf6de005bf0d
M: 03bf6bd7e3e6eece5a02043224497c2c8e185132 192.168.25.140:7006
slots: (0 slots) master
replicates 436e88ec323a2f8bb08bf09f7df07cc7909fcf81
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
执行结果
5,向集群中添加数据
cd /usr/local/redis-cluster/redis-1/bin ./redis-cli -h 192.168.127.137 -p 7001 –c set name abc get name
后续补充:(向集群中添加新节点、master、slave、指定master的slave)
添加master 前面是新添加的redis 后面是原来集群中的任意一个redis节点
./redis-trib.rb add-node 193.168.10.103:7000 193.168.10.101:7000
添加slave
./redis-trib.rb add-node --slave 193.168.10.103:7000 193.168.10.102:7001
//为哪个master添加slave(master-id be4a31b6fcab6295dd14799b3b67ec128fbbd7f9 为masterid)
./redis-trib.rb add-node --slave --master-id be4a31b6fcab6295dd14799b3b67ec128fbbd7f9 193.168.10.103:7000 193.168.10.102:7001
ps:
添加或者删除节 需要考虑redis的槽的问题,保证槽不丢失,每个master上都分配一定的槽
一个集群中一共有16383个槽,里面是redis的数据,这个要分配好了。
可使用以下命令重新分配槽:
./redis-trib.rb reshard 193.168.10.101:7000
Normal
0
7.8 磅
0
2
false
false
false
EN-US
ZH-CN
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:普通表格;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.5pt;
mso-bidi-font-size:11.0pt;
font-family:"Times New Roman",serif;
mso-font-kerning:1.0pt;}