大数据-zookeeper的数据结构及节点操作

2024-03-20 09:28:46

1         zookeeper结构和命令

1.1       zookeeper两大功能：

读写数据：往往是状态信息或配置信息

提供监听：发现数据的变化情况

 

 

1.2       zookeeper特性

1、Zookeeper：一个leader，多个follower组成的集群

2、 全局数据一致：每个server保存一份相同的数据副本，client无论连接到哪个server，数据都是一致的

3、分布式读写，更新请求转发，由leader实施

4、更新请求顺序进行，来自同一个client的更新请求按其发送顺序依次执行

5、数据更新原子性，一次数据更新要么成功，要么失败

6、实时性，在一定时间范围内，client能读到最新数据

 

 

1.3       zookeeper数据结构

1、层次化的目录结构，命名符合常规文件系统规范(见下图)

2、每个节点在zookeeper中叫做znode,并且其有一个唯一的路径标识

3、节点Znode可以包含数据和子节点（但是EPHEMERAL类型的节点不能有子节点，下一页详细讲解）

4、客户端应用可以在节点上设置监视器（后续详细讲解）

 

 

1.4       数据结构图

大数据-zookeeper的数据结构及节点操作

 

 

1.5       读写节点数据:

1.5.1     命令行连接zk服务器

zkCli.sh

返回如下信息：

[root@min1 ~]# zkCli.sh

Connecting to localhost:2181

2018-11-13 07:30:14,084 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT

2018-11-13 07:30:14,094 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=min1

2018-11-13 07:30:14,095 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.7.0_45

2018-11-13 07:30:14,099 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation

2018-11-13 07:30:14,099 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/root/apps/jdk1.7.0_45/jre

2018-11-13 07:30:14,100 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/root/apps/zookeeper/bin/../build/classes:/root/apps/zookeeper/bin/../build/lib/*.jar:/root/apps/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/apps/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/root/apps/zookeeper/bin/../lib/netty-3.2.2.Final.jar:/root/apps/zookeeper/bin/../lib/log4j-1.2.15.jar:/root/apps/zookeeper/bin/../lib/jline-0.9.94.jar:/root/apps/zookeeper/bin/../zookeeper-3.4.5.jar:/root/apps/zookeeper/bin/../src/java/lib/*.jar:/root/apps/zookeeper/bin/../conf:

2018-11-13 07:30:14,100 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

2018-11-13 07:30:14,101 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp

2018-11-13 07:30:14,101 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>

2018-11-13 07:30:14,102 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux

2018-11-13 07:30:14,103 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64

2018-11-13 07:30:14,103 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-573.el6.x86_64

2018-11-13 07:30:14,104 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root

2018-11-13 07:30:14,105 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root

2018-11-13 07:30:14,106 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/root

2018-11-13 07:30:14,108 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5cf2baac

Welcome to ZooKeeper!

2018-11-13 07:30:14,134 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@966] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)

2018-11-13 07:30:14,149 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@849] - Socket connection established to localhost/127.0.0.1:2181, initiating session

JLine support is enabled

2018-11-13 07:30:14,179 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1670a2f4010000a, negotiated timeout = 30000

 

WATCHER::

 

WatchedEvent state:SyncConnected type:None path:null

[zk: localhost:2181(CONNECTED) 0]

 

1.5.2     切换连接的服务器

connect 192.168.33.62:2181  connect min2:2181

返回：

[zk: localhost:2181(CONNECTED) 1] connect min2:2181

2018-11-13 07:31:44,978 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down

2018-11-13 07:31:44,979 [myid:] - INFO  [main:ZooKeeper@684] - Session: 0x1670a2f4010000a closed

2018-11-13 07:31:44,985 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=min2:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@7bae9cec

2018-11-13 07:31:44,988 [myid:] - INFO  [main-SendThread(min2:2181):ClientCnxn$SendThread@966] - Opening socket connection to server min2/192.168.33.62:2181. Will not attempt to authenticate using SASL (unknown error)

ZooKeeper -server host:port cmd args

        connect host:port

        get path [watch]

        ls path [watch]

        set path data [version]

        rmr path

        delquota [-n|-b] path

        quit

        printwatches on|off

        create [-s] [-e] path data acl

        stat path [watch]

        close

        ls2 path [watch]

        history

        listquota path

        setAcl path acl

        getAcl path

        sync path

        redo cmdno

        addauth scheme auth

        delete path [version]

        setquota -n|-b val path

[zk: min2:2181(CONNECTING) 2] 2018-11-13 07:31:44,990 [myid:] - INFO  [main-SendThread(min2:2181):ClientCnxn$SendThread@849] - Socket connection established to min2/192.168.33.62:2181, initiating session

2018-11-13 07:31:44,996 [myid:] - INFO  [main-SendThread(min2:2181):ClientCnxn$SendThread@1207] - Session establishment complete on server min2/192.168.33.62:2181, sessionid = 0x2670a2f1b430005, negotiated timeout = 30000

 

WATCHER::

 

WatchedEvent state:SyncConnected type:None path:null

 

[zk: min2:2181(CONNECTED) 2]

 

1.5.3     查看节点

ls

[zk: min2:2181(CONNECTED) 7] ls /

[zookeeper]

 

1.5.4     创建节点

create /app1 "this is app1 servers parent"

[zk: min2:2181(CONNECTED) 8] create /app1 "this is app1 servers parent"

Created /app1

ls

[zk: min2:2181(CONNECTED) 10] ls /

[app1, zookeeper]

 create /app1/server01 "192.168.33.61,100"

[zk: min2:2181(CONNECTED) 11] create /app1/server01 "192.168.33.61,100"

Created /app1/server01

 

[zk: min2:2181(CONNECTED) 12] ls /app1

[server01]

1.5.5     获取节点数据

get /app1/server01

[zk: localhost:2181(CONNECTED) 0] get /app1/server01

"192.168.33.61,100"

cZxid = 0x200000023  #内部事务编号

ctime = Tue Nov 13 07:58:10 CST 2018 #创建时间

mZxid = 0x200000023

mtime = Tue Nov 13 07:58:10 CST 2018 #修改时间

pZxid = 0x200000023 #持久化事务

cversion = 0 #创建版本

dataVersion = 0 #数据版本

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 19 #数据长度

numChildren = 0 #子节点数

1.5.6     创建一个短暂节点

1.5.6.1      创建短暂节点

[zk: localhost:2181(CONNECTED) 2] create -e /app-emphemeral 181818

Created /app-emphemeral

1.5.6.2      查看节点信息

[zk: localhost:2181(CONNECTED) 4] ls /

[app1, app-emphemeral, zookeeper] #可以看到临时节点

1.5.6.3      退出后再链接进去查看节点信息

quit

zkCli.sh

ls /

[zk: localhost:2181(CONNECTED) 0] ls /

[app1, zookeeper]   #临时节点已经没有了；

1.5.7     创建带有序号的节点

[zk: localhost:2181(CONNECTED) 18] create /test 888

Created /test

[zk: localhost:2181(CONNECTED) 19] create -s /test/aa 999

Created /test/aa0000000000

继续创建

[zk: localhost:2181(CONNECTED) 20] create -s /test/aa 999

Created /test/aa0000000001   #会自动增加序号添加节点

[zk: localhost:2181(CONNECTED) 21] create -s /test/aa 999

Created /test/aa0000000002

[zk: localhost:2181(CONNECTED) 22] create -s /test/aa 999

Created /test/aa0000000003

[zk: localhost:2181(CONNECTED) 23] create -s /test/aa 999

Created /test/aa0000000004

[zk: localhost:2181(CONNECTED) 24] create -s /test/aa 999

Created /test/aa0000000005

[zk: localhost:2181(CONNECTED) 25] create -s /test/aa 999

Created /test/aa0000000006

[zk: localhost:2181(CONNECTED) 26] create -s /test/aa 999

Created /test/aa0000000007

[zk: localhost:2181(CONNECTED) 27] create -s /test/bb 999

Created /test/bb0000000008

[zk: localhost:2181(CONNECTED) 28] create -s /test/bb 999

Created /test/bb0000000009

[zk: localhost:2181(CONNECTED) 29] create -s /test/cc 999

Created /test/cc0000000010

[zk: localhost:2181(CONNECTED) 30] create -s /test/cc 999

创建临时的带序号的节点

[zk: localhost:2181(CONNECTED) 31] create -e -s /test/cc 999

Created /test/cc0000000012

1.5.8     多服务器更新数据同步

61上操作：

[zk: localhost:2181(CONNECTED) 33] set /app1 111

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000041

mtime = Tue Nov 13 08:41:24 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 1

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 34] get /app1

111

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000041

mtime = Tue Nov 13 08:41:24 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 1

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 35]

登录62，操作：

zkCli.sh  #登录62本机

[zk: localhost:2181(CONNECTED) 0] get /app1

111

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000041

mtime = Tue Nov 13 08:41:24 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 1

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 1] set /app1 222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 2] get /app1

222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

 

在61上查看

[zk: localhost:2181(CONNECTED) 0] get /app1

222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

 

 

 

1.6       监听事件

 

1.6.1     数据变化监听

61上监听：

[zk: localhost:2181(CONNECTED) 1] get /app1 watch

222

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000046

mtime = Tue Nov 13 08:43:54 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 2

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

[zk: localhost:2181(CONNECTED) 2]

WATCHER::

 

62上修改

[zk: localhost:2181(CONNECTED) 3] set /app1 333

cZxid = 0x200000022

ctime = Tue Nov 13 07:55:17 CST 2018

mZxid = 0x200000048

mtime = Tue Nov 13 08:55:45 CST 2018

pZxid = 0x200000023

cversion = 1

dataVersion = 3

aclVersion = 0

ephemeralOwner = 0x0

dataLength = 3

numChildren = 1

 

在61上发生变化：注意，该事件只接受一次；再修改就不会接受到监听了；

[zk: localhost:2181(CONNECTED) 2]

WATCHER::

WatchedEvent state:SyncConnected type:NodeDataChanged path:/app1

 

1.6.2     子节点变化监听

61上开启节点监听

[zk: localhost:2181(CONNECTED) 4] ls /app1 watch

[server01]

62上操作，增加节点

[zk: localhost:2181(CONNECTED) 4] create /app1/app2 222

Created /app1/app2

查看61

[zk: localhost:2181(CONNECTED) 5]

WATCHER::

 

WatchedEvent state:SyncConnected type:NodeChildrenChangedpath:/app1

 

[zk: localhost:2181(CONNECTED) 5] ls /app1

[app2, server01]

 

1.7       节点类型

1.7.1     Znode有两种类型：

短暂（ephemeral）（断开连接自己删除）

    如果创建的客户端断开，那么会被自动删除；

     用于服务器掉线的自动感知，掉线后自动删除，那么就知道不存在了；

持久（persistent）（断开连接不删除）

如果创建的客户端断开，那么不会被自动删除；

 

1.7.2     Znode有四种形式的目录节点（默认是persistent ）

PERSISTENT（持久节点）

PERSISTENT_SEQUENTIAL（持久序列/test0000000019 ）

EPHEMERAL（临时节点）

EPHEMERAL_SEQUENTIAL （临时序列/test0000000019）

创建znode时设置顺序标识，znode名称后会附加一个值，顺序号是一个单调递增的计数器，由父节点维护

在分布式系统中，顺序号可以被用于为所有的事件进行全局排序，这样客户端可以通过顺序号推断事件的顺序

上一篇：宋欣蓉，10.27.spss


下一篇：redis集群搭建