ZooKeeper有三种安装模式:单机安装(standalone)、伪集群模式、集群模式:
- standalone模式:只在一台机器上安装ZooKeeper,并只启动一个ZooKeeper实例。
- 伪集群:在一台机器上开启多个ZooKeeper实例来模拟集群。通过加载不同配置文件,分配不同数据目录来实现这种模式。因为在同一机器上运行,所以机器宕机,整个ZooKeeper就挂了。基本上不会布置这样的ZooKeeper集群。
- 集群模式:在至少3台机器上安装ZooKeeper构成集群,只要故障节点数量在一定范围内就能保证ZooKeeper服务的可用性。
真正布置的基本上都是集群模式,standalone偶尔用来做些测试,它有单点故障,伪集群模式也有单点故障。
集群模式并非一定至少3台机器,但用2台机器比1台机器更危险,因为随便故障一台都无法满足"大多数"的要求,都会阻塞整个ZooKeeper服务。而2台机器的故障几率是1台机器的2倍。所以,也建议采用奇数台机器来部署ZooKeeper。
ZooKeeper下载:https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/stable/
我下载的是3.4.14版本。
1.安装jdk
ZooKeeper运行在java环境下,所以需要先安装jdk,且要求版本高于1.6。
jdk下载:http://www.oracle.com/technetwork/java/javase/downloads/index.html
源码包安装jdk
[root@master ~]# tar xf jdk-8u191-linux-x64.tar.gz -C /usr/local/ [root@master ~]# cd /usr/local/ [root@master local]# ln -sv jdk1.8.0_191/ jdk "jdk" -> "jdk1.8.0_191/"
配置环境变量:
[root@master local]# vi /etc/profile.d/jdk.sh JAVA_HOME=/usr/local/jdk PATH=$JAVA_HOME/bin:$PATH export JAVA_HOME PATH
[root@master local]# chmod o+x /etc/profile.d/jdk.sh [root@master local]# /etc/profile.d/jdk.sh
查看java版本。
[root@master local]# ln -sv /usr/local/jdk1.8.0_191/jre/bin/java /usr/local/bin "/usr/local/bin/java" -> "/usr/local/jdk1.8.0_191/jre/bin/java" [root@master local]# java -version java version "1.8.0_191" Java(TM) SE Runtime Environment (build 1.8.0_191-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
StandAlone模式
standalone模式是在单机上安装ZooKeeper。
先解压zookeeper-3.4.14.tar.gz。
[root@master ~]# tar xf zookeeper-3.4.14.tar.gz
再把它移到/usr/local/下方便管理。
[root@master ~]# mv zookeeper-3.4.14 /usr/local/zookeeper
在ZooKeeper的bin目录下提供了几个Windows和Linux下的脚本:
[root@master ~]# cd /usr/local/zookeeper [root@master zookeeper]# ls bin README.txt zkCleanup.sh zkCli.cmd zkCli.sh zkEnv.cmd zkEnv.sh zkServer.cmd zkServer.sh zkTxnLogToolkit.cmd zkTxnLogToolkit.sh
- zkServer:用于启动、停止ZooKeeper,且能查看ZooKeeper状态。
zkServer.sh {start|start-foreground|stop|restart|status|upgrade|print-cmd}
- zkEnv:设置ZooKeeper启动、关闭时的环境变量。在其余的每个脚本中都引用了这个脚本。
- zkCleanup:清除ZooKeeper的事务日志和快照。
- zkCli:ZooKeeper的一个命令行客户端。
设置ZooKeeper的环境变量,这不是必须的,只是为了方便操作上面的几个脚本
[root@master zookeeper]# echo 'ZOOKEEPER_HOME=/usr/local/zookeeper' >/etc/profile.d/zk.sh [root@master zookeeper]# echo 'PATH=$ZOOKEEPER_HOME/bin:$PATH' >> /etc/profile.d/zk.sh [root@master zookeeper]# chmod +x /etc/profile.d/zk.sh [root@master zookeeper]# source /etc/profile.d/zk.sh
把windows下的脚本删除:
[root@master zookeeper]# rm -rf /usr/local/zookeeper/bin/{*.cmd,README.txt} [root@master zookeeper]# ls bin/ zkCleanup.sh zkCli.sh zkEnv.sh zkServer.sh zkTxnLogToolkit.sh
为了启动ZooKeeper,先提供一个配置文件。默认配置文件的路径为$ZOOKEEPER_HOME下的conf目录。在这个目录下,有几个文件:
[root@master zookeeper]# ls conf configuration.xsl log4j.properties zoo_sample.cfg
-
configuration.xsl
:无视它。 -
log4f.properties
:是ZooKeeper的日志配置文件。 -
zoo_sample.cfg
:是ZooKeeper的示例配置文件。
zkServer.sh默认的配置文件名为zoo.cfg
。所以,在conf目录下创建一个zoo.cfg。写入几个保证ZooKeeper能正常运行的配置项:
[root@master zookeeper]# cd conf/ [root@master conf]# vi zoo.cfg tickTime=2000 dataDir=/usr/local/zookeeper/data1 clientPort=2181 ~
其中:
-
tickTime
:ZooKeeper中很多配置项(如心跳时间、连接超时时间等)都要用到的时间单位,这里配置为2000毫秒,即2秒。 -
dataDir
:ZooKeeper实例的数据目录。 -
clientPort
:ZooKeeper向外提供服务的端口。
然后用zkServer.sh来启动ZooKeeper。
[root@master zookeeper]# zkServer.sh start ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
也可以手动指定启动时读取的配置文件:
[root@master zookeeper]# zkServer.sh start
查看ZooKeeper的运行状态:
[root@master zookeeper]# zkServer.sh status ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg Mode: standalone
可以看到运行模式是"standalone"。
ZooKeeper启动后,就可以向外提供ZooKeeper服务了。这里用ZooKeeper提供的命令行客户端去连接一下ZooKeeper做个简单的测试。
连接ZooKeeper实例:
[root@master zookeeper]# zkCli.sh -server localhost:2181 Connecting to localhost:2181 2020-03-05 19:18:53,299 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2020-03-05 19:18:53,317 [myid:] - INFO [main:Environment@100] - Client environment:host.name=master 2020-03-05 19:18:53,318 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_191 2020-03-05 19:18:53,332 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2020-03-05 19:18:53,333 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/local/jdk1.8.0_191/jre 2020-03-05 19:18:53,333 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../zookeeper-server/target/classes:/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../zookeeper-server/target/lib/*.jar:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.7.25.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.7.25.jar:/usr/local/zookeeper/bin/../lib/netty-3.10.6.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.17.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../lib/audience-annotations-0.5.0.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.14.jar:/usr/local/zookeeper/bin/../zookeeper-server/src/main/resources/lib/*.jar:/usr/local/zookeeper/bin/../conf: 2020-03-05 19:18:53,334 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2020-03-05 19:18:53,335 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2020-03-05 19:18:53,335 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2020-03-05 19:18:53,335 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2020-03-05 19:18:53,336 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2020-03-05 19:18:53,336 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-693.el7.x86_64 2020-03-05 19:18:53,337 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2020-03-05 19:18:53,338 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2020-03-05 19:18:53,338 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/local/zookeeper 2020-03-05 19:18:53,342 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@799f7e29 2020-03-05 19:18:53,392 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) Welcome to ZooKeeper! JLine support is enabled 2020-03-05 19:18:53,725 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2020-03-05 19:18:53,856 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x100002d7ab00000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0]
例如,创建一个znode:
[zk: localhost:2181(CONNECTED) 0] create /zk_test mydata1 Created /zk_test [zk: localhost:2181(CONNECTED) 1] ls / [zookeeper, zk_test] [zk: localhost:2181(CONNECTED) 2] quit Quitting... 2020-03-05 19:24:42,992 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x100002d7ab00000 closed 2020-03-05 19:24:42,994 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x100002d7ab00000
配置伪集群模式
ZooKeeper的伪集群模式是在一个服务器上运行多个ZooKeeper实例来模拟ZooKeeper集群。伪集群模式下,每个实例的配置文件不同、数据目录不同、端口不同。一般来说,ZooKeeper集群至少需要3个服务器节点。
这里,我配置3个实例的ZooKeeper伪集群,它们的数据目录分别为$ZOOKEEPER_HOME
下的data1、data2和data3。因为稍后要向这些目录中写myid文件,所以先创建这3个目录:
mkdir /usr/local/zookeeper/data{1,2,3}
先提供3个实例的配置文件,分别为$ZOOKEEPER_HOME/conf/{zoo1.cfg,zoo2.cfg,zoo3.cfg}
。
以下是zoo1.cfg内容,各配置项稍后解释。
[root@master ~]# cd /usr/local/zookeeper/conf/
[root@master conf]# vi zoo2.cfg tickTime=2000 dataDir=/usr/local/zookeeper/data2 clientPort=2182 initLimit=5 syncLimit=2 server.1=localhost:2887:3887 server.2=localhost:2888:3888 server.3=localhost:2889:3889
[root@master conf]# vi zoo3.cfg tickTime=2000 dataDir=/usr/local/zookeeper/data3 clientPort=2183 initLimit=5 syncLimit=2 server.1=localhost:2887:3887 server.2=localhost:2888:3888 server.3=localhost:2889:3889
[root@master conf]# vi zoo2.cfg tickTime=2000 dataDir=/usr/local/zookeeper/data2 clientPort=2182 initLimit=5 syncLimit=2 server.1=localhost:2887:3887 server.2=localhost:2888:3888 server.3=localhost:2889:3889
上面的配置项中:
-
initLimit
:当非leader节点(即follower和observer)启动时,需要先从leader那里复制数据,以保证所有ZooKeeper节点数据都是同步的。这个选项设置非leader节点从启动到完成同步的超时时长,它以tickTime为时间单位,所以上面的超时时长为5*2=10
秒。一般来说,ZooKeeper保存的都是协调数据,数据量并不大,所以多数时候可以忽略这个参数,如果待同步数据真的很大,可以考虑增加这个超时时间。
-
syncLimit
:follower和leader之间数据延迟的最大时间长度。例如,有个节点的更新操作缓慢,它的数据已经严重落后于leader,ZooKeeper就会将它从ZooKeeper集群中踢出去。ZooKeeper使用时间来度量follower和leader之间数据的延迟,这个选项的值依赖于tickTime,例如tickTime=2000,syncLimit=2
表示follower比leader延迟了4秒。 -
server.X=[hostname]:port_A:port_B
:该选项用来指定ZooKeeper集群中的服务器节点。其中:- X:整数。是ZooKeeper中服务器的一个简单标识。这个数值需要和dataDir下的myid文件内容一致。在启动ZooKeeper集群中的每个实例时,需要读取数据目录中的myid文件,并将该文件中的数值和配置文件中的server.X做匹配,匹配到哪个就表示是哪个ZooKeeper服务器节点。
- hostname:ZooKeeper服务器节点的地址。
- port_A:这是第一个端口,用于Follower和Leader之间的数据同步和其它通信。
- port_B:这是第二个端口,用于Leader选举过程中投票通信。
所以,分别在每个实例的dataDir下创建对应的myid文件。
[root@master conf]# echo 1 >/usr/local/zookeeper/data1/myid [root@master conf]# echo 2 >/usr/local/zookeeper/data2/myid [root@master conf]# echo 3 >/usr/local/zookeeper/data3/myid
然后启动这3个ZooKeeper实例。
[root@master conf]# zkServer.sh start /usr/local/zookeeper/conf/zoo1.cfg ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/conf/zoo1.cfg Starting zookeeper ... already running as process 1192. [root@master conf]# zkServer.sh start /usr/local/zookeeper/conf/zoo2.cfg ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/conf/zoo2.cfg Starting zookeeper ... STARTED [root@master conf]# zkServer.sh start /usr/local/zookeeper/conf/zoo3.cfg ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/conf/zoo3.cfg Starting zookeeper ... STARTED
查看当前java进程列表:
[root@master conf]# /usr/local/jdk1.8.0_191/bin/jps -l 2805 org.apache.zookeeper.server.quorum.QuorumPeerMain 1192 org.apache.zookeeper.server.quorum.QuorumPeerMain 2840 org.apache.zookeeper.server.quorum.QuorumPeerMain 2921 sun.tools.jps.Jps
查看这3个实例之间的关系:zoo3.cfg被选为leader,其它是follower。
ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/conf/zoo1.cfg Mode: follower [root@master conf]# zkServer.sh status /usr/local/zookeeper/conf/zoo2.cfg ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/conf/zoo2.cfg Mode: follower [root@master conf]# zkServer.sh status /usr/local/zookeeper/conf/zoo3.cfg ZooKeeper JMX enabled by default Using config: /usr/local/zookeeper/conf/zoo3.cfg Mode: leader
可以用ZooKeeper提供的命令行客户端工具来测试,写入其中一个实例的数据,其它实例也能收到。
[root@master conf]# zkCli.sh -server localhost:2181 create /my_znode hello Connecting to localhost:2181 2020-03-05 19:40:29,084 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2020-03-05 19:40:29,087 [myid:] - INFO [main:Environment@100] - Client environment:host.name=master 2020-03-05 19:40:29,087 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_191 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/local/jdk1.8.0_191/jre 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../zookeeper-server/target/classes:/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../zookeeper-server/target/lib/*.jar:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.7.25.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.7.25.jar:/usr/local/zookeeper/bin/../lib/netty-3.10.6.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.17.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../lib/audience-annotations-0.5.0.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.14.jar:/usr/local/zookeeper/bin/../zookeeper-server/src/main/resources/lib/*.jar:/usr/local/zookeeper/bin/../conf: 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-693.el7.x86_64 2020-03-05 19:40:29,090 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2020-03-05 19:40:29,091 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2020-03-05 19:40:29,091 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/local/zookeeper/conf 2020-03-05 19:40:29,092 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@799f7e29 2020-03-05 19:40:29,138 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 2020-03-05 19:40:29,267 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2020-03-05 19:40:29,287 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x100002d7ab00001, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null Created /my_znode [root@master conf]# zkCli.sh -server localhost:2182 ls / Connecting to localhost:2182 2020-03-05 19:40:47,129 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2020-03-05 19:40:47,159 [myid:] - INFO [main:Environment@100] - Client environment:host.name=master 2020-03-05 19:40:47,160 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_191 2020-03-05 19:40:47,166 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2020-03-05 19:40:47,167 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/local/jdk1.8.0_191/jre 2020-03-05 19:40:47,167 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../zookeeper-server/target/classes:/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../zookeeper-server/target/lib/*.jar:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.7.25.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.7.25.jar:/usr/local/zookeeper/bin/../lib/netty-3.10.6.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.17.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../lib/audience-annotations-0.5.0.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.14.jar:/usr/local/zookeeper/bin/../zookeeper-server/src/main/resources/lib/*.jar:/usr/local/zookeeper/bin/../conf: 2020-03-05 19:40:47,167 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2020-03-05 19:40:47,167 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-693.el7.x86_64 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2020-03-05 19:40:47,168 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/local/zookeeper/conf 2020-03-05 19:40:47,170 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2182 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@799f7e29 2020-03-05 19:40:47,233 [myid:] - INFO [main-SendThread(localhost:2182):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2182. Will not attempt to authenticate using SASL (unknown error) 2020-03-05 19:40:47,349 [myid:] - INFO [main-SendThread(localhost:2182):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2182, initiating session 2020-03-05 19:40:47,392 [myid:] - INFO [main-SendThread(localhost:2182):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2182, sessionid = 0x200004a40590000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zookeeper]
follower