redis 消息队列（发布订阅）、持久化(RDB、AOF)、集群(cluster)

2022-10-18 08:20:12

一：订阅：

192.168.10.205:6379> SUBSCRIBE test

Reading messages... (press Ctrl-C to quit)

1) "subscribe"

2) "test"

3) (integer) 1

在另一个终端向订阅test发布消息：

127.0.0.1:6379> PUBLISH test1  hello

(integer) 0

上一个终端的订阅就收到了消息：

1) "message"

2) "test"

3) "hello"

二：redis持久化：

redis支持两种数据持久化，一种是rdb，会在指定的数据间隔内把内存的数据快照到文件，aof将所有服务端执行的命令备查，在服务重启的时候全部执行以此

rdb在断电的时候会丢失部分内网，aof不会

1、使用rdb持久化redis数据：

配置文件的快照时间设置：

快照是fork出一个当前的进程在后台进行快照，不会影响当前的进程，先写一个临时文件，写完以后将临时文件替换旧的文件。这个文件任何时候都是一个完整的副本

save 900 1 #900秒以内有1个key发生变化就快照
save 300 10 #300秒以内有10个key发生变化就快照
save 60 10000 #60秒内有10000个key变化就快照

rdbcompression yes #持久化到RDB文件时，是否压缩，“yes”为压缩，“no”则反之

rdbchecksum yes #读取和写入的时候是否支持CRC64校验，默认是开启的

dbfilename dump.rdb #保存的文件的名称

dir /usr/local/redis #快照文件的保存路径

SAVE 与 BGSAVE:

SAVE:阻塞保存

BGSAVE：在后台保存，不阻塞

KILL -9 会丢失自上次保存以后到现在的数据

2、使用AOF进行持久化：

appendonly yes #默认为no，改为yes

appendfilename "appendonly.aof" #保存的文件名，路径为rdb指定的file目录

关闭服务在启动即可

三：redis主从：

一个主服务器可以有多个从服务器，而且从服务器也可以有主服务器，主从复制时非阻塞的，通过fork主进程

主服务器收到从服务器的sync命令，就在后台执行bgsave，执行完毕后将保持的rdb文件发给客户端，客户端收到后将rdb载入到内存，新版本的支持增量备份

通过info命令查看默认配置：

# Replication

role:master    #默认较色为主服务器

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

在从服务器执行命令：

127.0.0.1:6379> SLAVEOF 192.168.10.205 6379 #主服务器的地址和端口

OK

查看从服务器的info信息：

# Replication

role:slave #角色为从服务器

master_host:192.168.10.205 #主服务器的地址

master_port:6379 #主服务器的端口

master_link_status:up

master_last_io_seconds_ago:8

master_sync_in_progress:0

slave_repl_offset:29

slave_priority:100

slave_read_only:1

connected_slaves:0

master_repl_offset:0

repl_backlog_active:0

repl_backlog_size:1048576

repl_backlog_first_byte_offset:0

repl_backlog_histlen:0

主服务器的状态信息：

# Replication

role:master #角色为主服务器

connected_slaves:1

slave0:ip=192.168.10.206,port=6379,state=online,offset=471,lag=0 #从服务器的地址和端口

master_repl_offset:471

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:470

从服务器无法写,如下：

127.0.0.1:6379> set a x

(error) READONLY You can't write against a read only slave.

三：redis集群：

四种大方案：

1、客户端分片，优势是比较灵活，不存在单点故障，缺点是添加节点需要重新配置，数据要手动同步

2、代理：代理分片，有proxy代理取数据，可以为proxy配置算法，如Twemproxy

3、Redis Cluster:在3.0版本以后支持，无中心，在某种情况下会造成数据丢失

4、Coodis：豌豆荚的开源方案

使用redis cluster：

需要至少6个机器，3主3从，下面创建8个客户端，另外2个做备用

先将配置文件复制到/opt，然后创建8个目录并生成8个不同端口、不同pid和不同rdb文件的redis.conf配置文件：

[root@node5 ~]# cd /opt/

[root@node5 opt]# mkdir `seq 7001 7008`

[root@node5 opt]# cp /etc/redis/6379.conf  .

[root@node5 opt]# sed 's/6379/7001/g' 6379.conf >> 7001/redis.conf

[root@node5 opt]# sed 's/7001/7002/g' 6379.conf >> 7002/redis.conf

[root@node5 opt]# sed 's/7002/7003/g' 6379.conf >> 7003/redis.conf

[root@node5 opt]# sed 's/7003/7004/g' 6379.conf >> 7004/redis.conf

[root@node5 opt]# sed 's/7004/7005/g' 6379.conf >> 7005/redis.conf

[root@node5 opt]# sed 's/7005/7006/g' 6379.conf >> 7006/redis.conf

[root@node5 opt]# sed 's/7006/7007/g' 6379.conf >> 7007/redis.conf

[root@node5 opt]# sed 's/7007/7008/g' 6379.conf >> 7008/redis.conf

批量启动redis：

[root@node5 opt]# for i in `seq 7001 7008`;do cd /opt/$i && /usr/local/redis/bin/redis-server  /opt/$i/redis.conf; done

确认端口都已经启动成功：

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7001

192.168.10.205:7001>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7002

192.168.10.205:7002>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7003

192.168.10.205:7003>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7004

192.168.10.205:7004>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7005

192.168.10.205:7005>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7006

192.168.10.205:7006>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7007

192.168.10.205:7007>

[root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7008

192.168.10.205:7008>

安装ruby管理工具：

[root@node5 opt]# yum install ruby rubygems -y

[root@node5 opt]# gem install redis

Successfully installed redis-3.2.2

1 gem installed

Installing ri documentation for redis-3.2.2...

Installing RDoc documentation for redis-3.2.2..

复制ruby的管理脚本：

[root@node5 ~]# cp /root/redis-3.0.7/src/redis-trib.rb  /usr/local/bin/redis-trib #是一个ruby脚本，方便redis管理

功能介绍：

root@node5 ~]# redis-trib  help

Usage: redis-trib <command> <options> <arguments ...>

  help            (show this help)

  del-node        host:port node_id #删除节点

  reshard         host:port #重新分片

                  --timeout <arg>

                  --pipeline <arg>

                  --slots <arg>

                  --to <arg>

                  --yes

                  --from <arg>

  fix             host:port

                  --timeout <arg>

  create          host1:port1 ... hostN:portN #创建集群

                  --replicas <arg>

  rebalance       host:port

                  --timeout <arg>

                  --simulate

                  --pipeline <arg>

                  --threshold <arg>

                  --use-empty-masters

                  --auto-weights

                  --weight <arg>

  call            host:port command arg arg .. arg

  add-node        new_host:new_port existing_host:existing_port #添加节点

                  --slave

                  --master-id <arg>

  check           host:port #检测节点

  import          host:port

                  --replace

                  --copy

                  --from <arg>

  set-timeout     host:port milliseconds

  info            host:port

创建集群：

[root@node5 ~]# redis-trib  create --replicas 1 192.168.10.205:7001 192.168.10.205:7002 192.168.10.205:7003 192.168.10.205:7004 192.168.10.205:7005 192.168.10.205:7006

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

192.168.10.205:7001

192.168.10.205:7002

192.168.10.205:7003

Adding replica 192.168.10.205:7004 to 192.168.10.205:7001

Adding replica 192.168.10.205:7005 to 192.168.10.205:7002

Adding replica 192.168.10.205:7006 to 192.168.10.205:7003

M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001

   slots:0-5460 (5461 slots) master

M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002

   slots:5461-10922 (5462 slots) master

M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003

   slots:10923-16383 (5461 slots) master

S: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004

   replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4

S: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005

   replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6

S: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006

   replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join..

>>> Performing Cluster Check (using node 192.168.10.205:7001)

M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 #ID及IP和端口

   slots:0-5460 (5461 slots) master #本机对应的分片位置，0-5460

M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002

   slots:5461-10922 (5462 slots) master #本机对应的分片位置，5461-10922 #ID及IP和端口

M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003

   slots:10923-16383 (5461 slots) master #本机对应的分片位置，10923-16383 #ID及IP和端口

M: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004

   slots: (0 slots) master

   replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 #master的ID

M: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005

   slots: (0 slots) master

   replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 #master的ID

M: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006

   slots: (0 slots) master

   replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35 #master的ID

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

连接到集群：

[root@node5 ~]# redis-cli  -h 192.168.10.205 -c -p 7001

查看集群信息：

# Replication

role:master

connected_slaves:1

slave0:ip=192.168.10.205,port=7004,state=online,offset=407,lag=0

master_repl_offset:407

repl_backlog_active:1

repl_backlog_size:1048576

repl_backlog_first_byte_offset:2

repl_backlog_histlen:406

操作集群：

192.168.10.205:7001> set key1 value1

-> Redirected to slot [9189] located at 192.168.10.205:7002

OK

192.168.10.205:7002> set key2 value2

-> Redirected to slot [4998] located at 192.168.10.205:7001

OK

192.168.10.205:7001> set key3 value3

OK

192.168.10.205:7001> set key4 value4

-> Redirected to slot [13120] located at 192.168.10.205:7003

OK

192.168.10.205:7003> set key5 value5

-> Redirected to slot [9057] located at 192.168.10.205:7002

OK

192.168.10.205:7002> set key6 value6

-> Redirected to slot [4866] located at 192.168.10.205:7001

OK

192.168.10.205:7001> set key7 value7

OK

192.168.10.205:7001> set key8 value8

-> Redirected to slot [13004] located at 192.168.10.205:7003 #插入的数据是轮训写入到各个redis server

OK

cluster nodes：列出当前的主从

192.168.10.205:7003> CLUSTER  nodes

8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 myself,master - 0 0 3 connected 10923-16383

5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458558870837 2 connected 5461-10922

32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458558869829 4 connected

840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458558868820 6 connected

19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458558870334 5 connected

9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458558869829 1 connected 0-5460

cluster info：当前的状态：

192.168.10.205:7003> CLUSTER INFO

cluster_state:ok

cluster_slots_assigned:16384

cluster_slots_ok:16384

cluster_slots_pfail:0

cluster_slots_fail:0

cluster_known_nodes:6

cluster_size:3

cluster_current_epoch:6

cluster_my_epoch:3

cluster_stats_messages_sent:2911

cluster_stats_messages_received:2911

向集群中添加节点：

[root@node5 ~]# redis-trib add-node 192.168.10.205:7007 192.168.10.205:7001

　　　　　　　　　　　　　　　　　　　　要添加的节点和端口添加到的目的节点和端口

[root@node5 ~]# redis-trib  add-node 192.168.10.205:7007 192.168.10.205:7001

>>> Adding node 192.168.10.205:7007 to cluster 192.168.10.205:7001

>>> Performing Cluster Check (using node 192.168.10.205:7001)

M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006

   slots: (0 slots) slave

   replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35

M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004

   slots: (0 slots) slave

   replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4

S: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005

   slots: (0 slots) slave

   replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

>>> Send CLUSTER MEET to node 192.168.10.205:7007 to make it join the cluster.

[OK] New node added correctly.

添加主机以后需要重新分片：

[root@node5 ~]# redis-trib reshard 192.168.10.205:7007

连接到新添加的节点查询信息：

[root@node5 ~]# redis-cli -h 192.168.10.205 -p  7007

192.168.10.205:7007> KEYS *

(empty list or set)

192.168.10.205:7007> CLUSTER NODES

5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458560205983 2 connected 5728-10922

8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 master - 0 1458560204471 3 connected 11189-16383

19a5bb651f4b182a0e3dd49154323d31c1524167 192.168.10.205:7007 myself,master - 0 0 7 connected 0-265 5461-5727 10923-11188 #已经有了分片信息

19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458560204977 2 connected

840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458560205482 3 connected

9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458560204471 1 connected 266-5460

32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458560204977 1 connected

为新添加的主节点添加从节点：

[root@node5 ~]# redis-trib  add-node 192.168.10.205:7008 192.168.10.205:7001 #添加一个节点

#登陆到从节点进行配置

[root@node5 ~]# redis-cli -h 192.168.10.205 -c -p 7008 #登陆到要备用的从节点

192.168.10.205:7008> CLUSTER replicate 19a5bb651f4b182a0e3dd49154323d31c1524167 #指定主节点的ID即可

OK

再次查看集群信息：

192.168.10.205:7008> CLUSTER  nodes

19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458560717276 2 connected

ab146fb5881064a6375da76f861e205d2d702140 192.168.10.205:7008 myself,slave 19a5bb651f4b182a0e3dd49154323d31c1524167 0 0 0 connected   #已经是从节点并连接了

32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458560716772 1 connected

8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 master - 0 1458560715764 3 connected 11189-16383

19a5bb651f4b182a0e3dd49154323d31c1524167 192.168.10.205:7007 master - 0 1458560717276 7 connected 0-265 5461-5727 10923-11188

9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458560717276 1 connected 266-5460

5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458560716267 2 connected 5728-10922

840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458560717780 3 connected

码农公寓

相关文章