1. Zookeeper集群概念
ZK集群:3个节点,一个leader,2个follower
2. 伪分布式集群部署
伪分布式:1台机器上面搭建多个ZK节点
2.1 解压安装zk
解压zookeeper安装包并拷贝成三份,也就是三个zk节点
[hadoop@hadoop000 app]$ cp -r zookeeper-3.4.5-cdh5.7.0 zk001 [hadoop@hadoop000 app]$ cp -r zookeeper-3.4.5-cdh5.7.0 zk002 [hadoop@hadoop000 app]$ cp -r zookeeper-3.4.5-cdh5.7.0 zk003
2.2 修改zk配置文件
分别修改三个zk节点的配置文件
[hadoop@hadoop000 app]$ vi zk001/conf/zoo.cfg dataDir=/home/hadoop/tmp/zookeeper001 server.1=localhost:2888:3888 server.2=localhost:2889:3889 server.3=localhost:2890:3890 [hadoop@hadoop000 app]$ vi zk002/conf/zoo.cfg dataDir=/home/hadoop/tmp/zookeeper002 server.1=localhost:2888:3888 server.2=localhost:2889:3889 server.3=localhost:2890:3890 [hadoop@hadoop000 app]$ vi zk003/conf/zoo.cfg dataDir=/home/hadoop/tmp/zookeeper003 server.1=localhost:2888:3888 server.2=localhost:2889:3889 server.3=localhost:2890:3890 # 其中,2888是服务器与zk leader(master)通信的端口,3888为leader挂掉时,重新选举leader的端口。伪分布式部署时3个zk节点都在同一台机器上,为了端口不发生冲突,所以各节点的端口号依次+1
为每个zk节点创建dataDir文件夹
[hadoop@hadoop000 app]$ mkdir ~/tmp/zookeeper001 [hadoop@hadoop000 app]$ mkdir ~/tmp/zookeeper002 [hadoop@hadoop000 app]$ mkdir ~/tmp/zookeeper003
为每个zk节点创建myid
[hadoop@hadoop000 app]$ echo 1 > ~/tmp/zookeeper001/myid [hadoop@hadoop000 app]$ echo 2 > ~/tmp/zookeeper002/myid [hadoop@hadoop000 app]$ echo 3 > ~/tmp/zookeeper003/myid
2.3 启动3个zk节点
启动3个zk节点,每个节点会产生一个QuorumPeerMain进程
[hadoop@hadoop000 app]$ zk001/bin/zkServer.sh start [hadoop@hadoop000 app]$ jps [hadoop@hadoop000 app]$ zk002/bin/zkServer.sh start [hadoop@hadoop000 app]$ jps [hadoop@hadoop000 app]$ zk003/bin/zkServer.sh start [hadoop@hadoop000 app]$ jps
2.4 查看节点状态
查看每个节点状态,可以看到本次启动后,zk002是leader,zk001和zk003是follower
[hadoop@hadoop000 app]$ zk001/bin/zkServer.sh status Using config: /home/hadoop/app/zk001/bin/../conf/zoo.cfg [hadoop@hadoop000 app]$ zk002/bin/zkServer.sh status Using config: /home/hadoop/app/zk002/bin/../conf/zoo.cfg [hadoop@hadoop000 app]$ zk003/bin/zkServer.sh status Using config: /home/hadoop/app/zk003/bin/../conf/zoo.cfg
2.5 测试集群数据
从节点zk001启动一个zk client,连接到zk server 2182端口(zk002)。并且创建一个节点,写入数据。
[hadoop@hadoop000 app]$ zk001/bin/zkCli.sh -server localhost:2182 [zk: localhost:2182(CONNECTED) 0] ls / [zk: localhost:2182(CONNECTED) 1] create /test hellozk002 [zk: localhost:2182(CONNECTED) 2] get /test ctime = Thu Aug 16 19:53:03 CST 2018 mtime = Thu Aug 16 19:53:03 CST 2018 [zk: localhost:2182(CONNECTED) 3]
去节点zk002启动一个zk client,连接zk server 2183端口(zk003),可以查看到刚才创建的test节点和数据。
[hadoop@hadoop000 app]$ zk002/bin/zkCli.sh -server localhost:2183 [zk: localhost:2183(CONNECTED) 0] ls / [zk: localhost:2183(CONNECTED) 1] get /test ctime = Thu Aug 16 19:53:03 CST 2018 mtime = Thu Aug 16 19:53:03 CST 2018 [zk: localhost:2183(CONNECTED) 2]
2.6 测试节点挂掉情况
将leader节点zk002进程停止,查看集群中各节点选举情况
[hadoop@hadoop000 app]$ kill -9 13969 [hadoop@hadoop000 app]$ zk002/bin/zkServer.sh status Using config: /home/hadoop/app/zk002/bin/../conf/zoo.cfg Error contacting service. It is probably not running. [hadoop@hadoop000 app]$ zk003/bin/zkServer.sh status Using config: /home/hadoop/app/zk003/bin/../conf/zoo.cfg
集群挂掉后数据还是可以查到,集群内部容错性是可以保证的。
3. 分布式集群部署
真正的分布式集群部署,过程基本一致,只是不同节点上的ZK路径完全一致,配置文件要使用同样的端口号。
[hadoop@hadoop000 app]$ vi zookeeper-3.4.5-cdh5.7.0/conf/zoo.cfg dataDir=/home/hadoop/tmp/zookeeper server.1=hadoop000:2888:3888 server.2=hadoop001:2888:3888 server.3=hadoop002:2888:3888
每个节点上的myid还是按照上面伪分布式部署的方式,myid号每个节点不同。其他的步骤与伪分布式一模一样。
|