zookeeper集群搭建
下载zookeeper
zookeeper集群介绍
zookeeper集群模式：standalone与quorum模式。
zookeeper如何保证数据一致性：
1、主节点负责写请求和读请求，follower节点只负责读请求，如果follower收到写请求也会转发给主节点。主节点的写入采用全局线性化写入，先来先写入。
2、客户端FIFO顺序：来自给定客户端的请求按照发送顺序执行。
zookeeper配置
将zookeeper的安装包解压后复制三份，或者你准备三个配置文件即可，今天我们将搭建一个三节点的zookeeper集群：
配置如下：
节点1：
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# zookeeper数据存储目录
dataDir=/data/zk/quorum/node1
# zookeeper监听的客户端地址，因为我是在一台机子上，所以不同的节点监听的端口号必须不一样
clientPort=2181
# 集群信息
# 格式为：server.n = ip : serverPort : selectMasterPort
# serverPort 为服务节点的端口号，selectMasterPort为选主时的端口号
server.1=127.0.0.1:3333:3334
server.2=127.0.0.1:4444:4445
server.3=127.0.0.1:5555:5556
节点2：
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# zookeeper数据存储目录
dataDir=/data/zk/quorum/node2
# zookeeper监听的客户端地址，因为我是在一台机子上，所以不同的节点监听的端口号必须不一样
clientPort=2182
# 集群信息
# 格式为：server.n = ip : serverPort : selectMasterPort
# serverPort 为服务节点的端口号，selectMasterPort为选主时的端口号
server.1=127.0.0.1:3333:3334
server.2=127.0.0.1:4444:4445
server.3=127.0.0.1:5555:5556
节点3：
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# zookeeper数据存储目录
dataDir=/data/zk/quorum/node3
# zookeeper监听的客户端地址，因为我是在一台机子上，所以不同的节点监听的端口号必须不一样
clientPort=2183
# 集群信息
# 格式为：server.n = ip : serverPort : selectMasterPort
# serverPort 为服务节点的端口号，selectMasterPort为选主时的端口号
server.1=127.0.0.1:3333:3334
server.2=127.0.0.1:4444:4445
server.3=127.0.0.1:5555:5556
配置好之后保存，启动3个节点。
# 在zookeeper的根目录下,启动不同的节点请修改不同的配置文件
./bin/zkServer.sh start-foreground conf/zoo1.cfg
启动节点
此时启动节点1，报错
[root@iZ219es4jofbudZ apache-zookeeper-3.6.2-bin]# ./bin/zkServer.sh start-foreground conf/zoo1.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo1.cfg
2021-03-24 11:18:36,616 [myid:] - INFO [main:QuorumPeerConfig@174] - Reading configuration from: conf/zoo1.cfg
2021-03-24 11:18:36,628 [myid:] - WARN [main:VerifyingFileFactory@65] - conf/zoo1.cfg is relative. Prepend ./ to indicate that you're sure!
2021-03-24 11:18:36,642 [myid:] - INFO [main:QuorumPeerConfig@460] - clientPortAddress is 0.0.0.0:2181
2021-03-24 11:18:36,645 [myid:] - INFO [main:QuorumPeerConfig@464] - secureClientPort is not set
2021-03-24 11:18:36,645 [myid:] - INFO [main:QuorumPeerConfig@480] - observerMasterPort is not set
2021-03-24 11:18:36,645 [myid:] - INFO [main:QuorumPeerConfig@497] - metricsProvider.className is org.apache.zookeeper.metrics.impl.DefaultMetricsProvider
2021-03-24 11:18:36,652 [myid:] - ERROR [main:QuorumPeerMain@98] - Invalid config, exiting abnormally
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing conf/zoo1.cfg
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:198)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:124)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:90)
Caused by: java.lang.IllegalArgumentException: myid file is missing
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.checkValidity(QuorumPeerConfig.java:812)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.setupQuorumPeerConfig(QuorumPeerConfig.java:683)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:507)
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:194)
... 2 more
Invalid config, exiting abnormally
2021-03-24 11:18:36,654 [myid:] - INFO [main:ZKAuditProvider@42] - ZooKeeper audit is disabled.
2021-03-24 11:18:36,656 [myid:] - ERROR [main:ServiceUtils@42] - Exiting JVM with code 2
错误表明我们没有配置myid文件。
小贴士：
myid文件用来标识zookeeper节点的信息，配置在datadir的路径下,名为myid的文件
我们分别在三个节点的datadir下，创建了对应的myid文件，并分别写入，1、2、3。如图：
设置好myid文件之后，启动节点1后，报类似的错误，这个错误没问题，表明节点1无法与节点2、3建立连接。
/data/zk/quorum/node1/2021-03-24 11:31:46,265 [myid:1] - INFO [QuorumPeer[myid=1](plain=0.0.0.0:2181)(secure=disabled):FastLeaderElection@979] - Notification time out: 51200
2021-03-24 11:31:46,266 [myid:1] - WARN [QuorumConnectionThread-[myid=1]-2:QuorumCnxManager@400] - Cannot open channel to 2 at election address /127.0.0.1:4445
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-03-24 11:31:46,267 [myid:1] - WARN [QuorumConnectionThread-[myid=1]-3:QuorumCnxManager@400] - Cannot open channel to 3 at election address /127.0.0.1:5556
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.initiateConnection(QuorumCnxManager.java:383)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$QuorumConnectionReqThread.run(QuorumCnxManager.java:457)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
我们现在启动节点2、3，
我们看到，节点2成为了主节点，其余两个是从节点。
客户端连接集群
我们现在通过客户端连接zookeeper集群。
# 在zookeeper的根目录下,我们之前配置的zookeeper三个节点监听的端口号分别是2181,2182,2183
./bin/zkCli.sh -server 127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
我们看到，我们连接到了2182的主节点上。
集群高可用测试
我们在主节点上创建一个znode，
此时我们使主节点宕机，理论上，服务端会重新选举一个leader，客户端会主动连接到存活的节点上，且数据不丢失，我们来看看：
我们看到，当主节点宕机后，客户端会超时，同时收到Disconnected通知，且连接到了2183端口，在节点3（左下角）我们看到，节点3被选举为leader。
此时我们查看zookeeper的节点信息：
/test 临时节点仍然存在。
由此体现了zookeeper集群的高可用性。
参考：
Zookeeper集群报错：myid文件缺失导致zookeeper无法启动（myid file is missing）