主从复制
原理
This system works using three main mechanisms:
(1)When a master and a slave instances are well-connected, the master keeps the slave updated by sending a stream of commands to the slave, in order to replicate the effects on the dataset happening in the master side due to: client writes, keys expired or evicted, any other action changing the master dataset.
(2)When the link between the master and the slave breaks, for network issues or because a timeout is sensed in the master or the slave, the slave reconnects and attempts to proceed with a partial resynchronization: it means that it will try to just obtain the part of the stream of commands it missed during the disconnection.
(3)When a partial resynchronization is not possible, the slave will ask for a full resynchronization. This will involve a more complex process in which the master needs to create a snapshot of all its data, send it to the slave, and then continue sending the stream of commands as the dataset changes.
大致意思:
(1)当master与slave连接是好的时候,master会发送流命令给slave,也会更新slave的数据,保持和master一致;
(2)当master与slave连接由于网络问题或者其他原因断开,slave会重新连接,并尝试局部恢复在断开之后丢失的数据;
(3)当局部恢复不了,slave会做个全同步,这将会调用更加复杂的进程去做快照,发送给slave等等;
redis复制特点
redis是异步复制;
redis复制支持一主多从和级联复制;
当salve在初始化同步或局部同步的时候,master不会阻塞,仍然可以处理请求;
Slave大部分是不会阻塞的,你可以访问老的数据当salve在初始化同步或局部同步的时候;
redis的slave 默认read only;
redis建议master不要关闭数据持久化RDB和AOF,虽然不会影响复制本身,但是一旦master宕机,如果设置的自动重启master,那么master的所有数据都会丢失,更可怕的是会复制给slave,slave的所有数据也会清空,所以master关闭数据持久化这是一件非常危险的事情;
slaveof no one可以关闭复制结构,并不会丢失数据,这样就可以把从变成主,替换掉故障的master;
主从复制相关的参数
slaveof <masterip> <masterport>
#当本机为slave时,指定master服务的IP地址及端口,在Redis启动时,它会自动从master进行数据同步
masterauth <master-password>
#当master服务设置了密码保护时,slave服务连接master的密码
slave-read-only yes
#从库只读
slave-serve-stale-data yes
#当slave与master连接断开的时候,如果参数值是yes,slave仍然会响应client的请求,但是数据可能是过期的;
如果参数值是no,slave会返回error:"SYNC with master in progress"
repl-diskless-sync no
#这个参数控制master如何把数据到slave
有两种方式:
(1)disk-backed: master创建一个新进程做个RDB快照,然后父进程把这个RDB增量传输到slave上,当磁盘性能比较好,而网络带宽一般,可以使用disk-backed策略;
(2)diskless: master创建一个新进程直接写RDB文件到slave socket,不需要接触disk, 当磁盘性能比较差,而网络带宽比较好的情况,可以使用diskless策略;
repl-ping-slave-period 10
#slave发送ping给master的间隔时间,
repl-timeout 60
#复制超时的时间,这个参数必须要大于repl-ping-slave-period,否者会报错:复制一直超时
repl-disable-tcp-nodelay no
#控制是否关闭tcp不延时,如果值为yes, master则用较少的资源发送数据到slave,但是复制会有较大的延时;
如果值为no,master则用较多的资源发送数据到slave,同时复制会减少延时;
slave-priority 100
#这个参数被用来redis sentinel架构中,用来把slave提升为master的优先级,数字越小越优先;
min-slaves-to-write 3
min-slaves-max-lag 10
#这两个参数的含义是如果slave数量低于3个,并且延迟大于10,master则不能写入;
repl-backlog-size 1mb
#当slave断开连接,backlog会积累slave数据,以便slave重新连接后,只需要部分数据同步即可;
安装配置
redis复制比较简单,在配置文件中添加一个参数slaveof <masterip> <masterport>,然后执行命令slaveof <masterip> <masterport>动态生效即可;
如果master设置的密码访问,需要在添加一个参数masterauth <master-password>
master:
[root@sht-sgmhadoopcm-01 redis]# vim redis.conf
bind 172.16.101.54
slave:
[root@sht-sgmhadoopnn-01 redis]# vim redis.conf
bind 172.16.101.55
slaveof 172.16.101.54 6379
[root@sht-sgmhadoopnn-01 redis]# src/redis-cli -h 172.16.101.55
172.16.101.55:6379> slaveof 172.16.101.54 6379
OK
172.16.101.55:6379> config get *slaveof*
1) "slaveof"
2) "172.16.101.54 6379"
172.16.101.55:6379> config get *slave-read-only*
1) "slave-read-only"
2) "yes"
172.16.101.55:6379> set key3 3
(error) READONLY You can't write against a read only slave.
#查看主从服务信息
172.16.101.54:6379> client list
id=344 addr=172.16.101.55:42590 fd=5 name= age=339 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=replconf
172.16.101.54:6379> role
1) "master"
2) (integer) 8919
3) 1) 1) "172.16.101.55"
2) "6379"
3) "8919"
172.16.101.54:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.16.101.55,port=6379,state=online,offset=9255,lag=1
master_repl_offset:9255
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:9254
172.16.101.55:6379> client list
id=5 addr=172.16.101.54:6379 fd=6 name= age=84 idle=4 flags=M db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
172.16.101.55:6379> role
1) "slave"
2) "172.16.101.54"
3) (integer) 6379
4) "connected"
5) (integer) 8919
172.16.101.55:6379> info replication
# Replication
role:slave
master_host:172.16.101.54
master_port:6379
master_link_status:up
master_last_io_seconds_ago:7
master_sync_in_progress:0
slave_repl_offset:9325
slave_priority:100
slave_read_only:0
connected_slaves:0
master_repl_offset:119130015
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:119120678
repl_backlog_histlen:9338
#关闭从库的复制功能,但是数据仍然保留
172.16.101.55:6379> slaveof no one
OK
172.16.101.55:6379> config get *slaveof*
1) "slaveof"
2) ""
172.16.101.55:6379> slaveof no one
OK
172.16.101.55:6379> keys *
1) "key1"
2) "key2"
3) "counter:__rand_int__"
4) "key:__rand_int__"
5) "mylist"
参考链接
https://redis.io/topics/replication