使用docker-compose搭建redis集群环境

2021-10-04 16:12:00

主题：
主机只有一台需要搭建三主三从的redis集群

开盘：

**第一步：**创建目录分发redis.conf; redis目录里创建data、conf目录，将redis.conf文件分别拷贝到conf目录里；修改redis.conf文件的port（从7001-7006）

redis.conf

#Redis configuration for testing.

always-show-logo yes
notify-keyspace-events KEA
daemonize no
pidfile /var/run/redis.pid
port {按需添加端口}
timeout 0
bind 0.0.0.0
loglevel verbose
logfile ''
databases 16
latency-monitor-threshold 1

save 900 1
save 300 10
save 60 10000

rdbcompression yes
dbfilename dump.rdb
dir ./

# 持久化配置
slave-serve-stale-data yes
appendonly yes
appendfilename  appendonly.aof
appendfsync everysec
no-appendfsync-on-rewrite no
activerehashing yes

# 集群配置
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000

**第二步，**便携docker-compose的yml文件；
docker-compose.yml

version: '3.1'
services:
  redis-1:
    restart: always
    image: redis:latest
    container_name: redis-cluster-1
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7001/data:/data:rw
      - /data02/xxxx/middleware/redis/7001/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-2:
    restart: always
    image: redis:latest
    container_name: redis-cluster-2
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7002/data:/data:rw
      - /data02/xxxx/middleware/redis/7002/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-3:
    restart: always
    image: redis:latest
    container_name: redis-cluster-3
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7003/data:/data:rw
      - /data02/xxxx/middleware/redis/7003/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-4:
    restart: always
    image: redis:latest
    container_name: redis-cluster-4
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7004/data:/data:rw
      - /data02/xxxx/middleware/redis/7004/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-5:
    restart: always
    image: redis:latest
    container_name: redis-cluster-5
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7005/data:/data:rw
      - /data02/xxxx/middleware/redis/7005/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]
  redis-6:
    restart: always
    image: redis:latest
    container_name: redis-cluster-6
    environment:
      - TZ=Asia/Shanghai
    network_mode: "host"
    volumes:
      - /data02/xxxx/middleware/redis/7006/data:/data:rw
      - /data02/xxxx/middleware/redis/7006/conf/redis.conf:/etc/redis/redis.conf:rw
    command: ["redis-server","/etc/redis/redis.conf"]

命令： docker-compose -f docker-compose.yml up -d #后台启动所有的容器

[root@host-192-168-0-4 redis]# docker-compose up -d 
Creating redis-1 ... done
Creating redis-2 ... done
Creating redis-3 ... done
Creating redis-4 ... done
Creating redis-6 ... done
Creating redis-5 ... done

**第三步：**创建集群

docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 192.168.0.4:7001 192.168.0.4:7002 192.168.0.4:7003 192.168.0.4:7004 192.168.0.4:7005 192.168.0.4:7006 --cluster-replicas 1

中途有个yes

[root@host-192-168-0-4 redis]# docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 192.168.0.4:7001 192.168.0.4:7002 192.168.0.4:7003 192.168.0.4:7004 192.168.0.4:7005 192.168.0.4:7006 --cluster-replicas 1
.........
>>> Trying to optimize slaves allocation for anti-affinity
.........
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
.........
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

有个问题需要注意和讨论：
我在第一次安装的时候，自建了网络使用bridge的模式；暴露端口7001->6379,17001->16379; 这样的话配置文件都指向同一个redis.conf即可；启动容器的时候可以正常启动，
默认的情况redis是在业务端口+10000，再启动一个端口用于redis集群的底层gossip协议通讯，按照理论来讲，按照宿主机地址创建集群即可；但是在添加的时候会卡在“Waiting for the cluster to join…”；网络规划指定了每个容器启动的ip地址，后将创建集群的ip地址改成容器地址，并制定6379端口；例如：

docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 172.17.0.4:7001 172.17.0.5:7002 172.17.0.6:7003 172.17.0.7:7004 172.17.0.8:7005 172.17.0.9:7006 --cluster-replicas 1

结果是集群顺利建立；但是在使用redis-ui测试的时候，出现了可以连接单机的redis，不能连接集群模式；至此问题暴露，即bridge模式下的redis不能建立集群模式；

经过一顿国内外搜索有两个观点：
1、redis的官网的说明，docker建立redis集群模式，需要配置网络模式为host，即yml里需要改成 network_mode: “host”
2、redis是通过gossip协议进行集群管理的，所有节点信息都将写入到node.conf里，如果配置了宿主机地址；docker容器不经修改防火墙规则，不能访问到宿主机的地址；即容器里不能访问到 192.168.0.4:17001、17002…17006

还有存有疑虑的事情，请路过的高手大侠施以援手：
1、当集群配置容器地址的时候，可以创建集群，但是业务系统不能访问以集群模式访问redis，这是为什么？
2、容器为什么不能访问到宿宿主机，我理解的是从容器发出来的包通过docker0网桥未NET地址，相当于内网访问，为什么不通呢？

码农公寓

相关文章