分享

记录一次程序与zookeeper断开session连接的问题

 印度阿三17 2019-10-26

在项目开发过程中,需要与zookeeper建立连接,但是在程序运行过程中,遇到了程序与zookeeper断开连接的问题:

[2019-10-24 10:02:00.531] [Thread-7-SendThread(10.128.138.178:2181)] WARN  org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 6354ms for sessionid 0x1023f975777000a
[2019-10-24 10:02:00.531] [Thread-8-SendThread(10.128.138.178:2181)] WARN  org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 5919ms for sessionid 0x1023f9757770009
[2019-10-24 10:02:00.531] [Thread-8-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 5919ms for sessionid 0x1023f9757770009, closing socket connection and attempting reconnect
[2019-10-24 10:02:00.531] [Thread-7-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 6354ms for sessionid 0x1023f975777000a, closing socket connection and attempting reconnect
[2019-10-24 10:02:00.633] [Thread-8-EventThread] INFO  cn.cloudwalk.dist.base.handler.AbstractHandler - WatchedEvent state:Disconnected type:None path:null
[2019-10-24 10:02:00.633] [Thread-7-EventThread] INFO  cn.cloudwalk.dist.base.handler.MasterHandler - master exist watch, WatchedEvent is: WatchedEvent state:Disconnected type:None path:null
[2019-10-24 10:02:00.633] [Thread-7-EventThread] INFO  cn.cloudwalk.dist.base.handler.AbstractHandler - WatchedEvent state:Disconnected type:None path:null
[2019-10-24 10:02:00.633] [Thread-8-EventThread] INFO  cn.cloudwalk.dist.base.handler.SlaveHandler - slaveExistsWatcher, WatchedEvent: WatchedEvent state:Disconnected type:None path:null
[2019-10-24 10:02:01.942] [Thread-7-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.128.138.178/10.128.138.178:2181. Will not attempt to authenticate using SASL (unknown error)
[2019-10-24 10:02:01.943] [Thread-7-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established to 10.128.138.178/10.128.138.178:2181, initiating session
[2019-10-24 10:02:01.949] [Thread-7-SendThread(10.128.138.178:2181)] WARN  org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x1023f975777000a has expired
[2019-10-24 10:02:01.949] [Thread-7-EventThread] INFO  cn.cloudwalk.dist.base.handler.MasterHandler - master exist watch, WatchedEvent is: WatchedEvent state:Expired type:None path:null
[2019-10-24 10:02:01.949] [Thread-7-EventThread] INFO  cn.cloudwalk.dist.base.handler.AbstractHandler - WatchedEvent state:Expired type:None path:null
[2019-10-24 10:02:01.949] [Thread-7-EventThread] ERROR cn.cloudwalk.dist.base.handler.AbstractHandler - Session expired
[2019-10-24 10:02:01.949] [Thread-7-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x1023f975777000a has expired, closing socket connection
[2019-10-24 10:02:01.950] [Thread-7-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x1023f975777000a
[2019-10-24 10:02:02.434] [Thread-8-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server 10.128.138.178/10.128.138.178:2181. Will not attempt to authenticate using SASL (unknown error)
[2019-10-24 10:02:02.434] [Thread-8-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established to 10.128.138.178/10.128.138.178:2181, initiating session
[2019-10-24 10:02:02.435] [Thread-8-SendThread(10.128.138.178:2181)] WARN  org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x1023f9757770009 has expired
[2019-10-24 10:02:02.435] [Thread-8-EventThread] INFO  cn.cloudwalk.dist.base.handler.AbstractHandler - WatchedEvent state:Expired type:None path:null
[2019-10-24 10:02:02.435] [Thread-8-SendThread(10.128.138.178:2181)] INFO  org.apache.zookeeper.ClientCnxn - Unable to reconnect to ZooKeeper service, session 0x1023f9757770009 has expired, closing socket connection
[2019-10-24 10:02:02.435] [Thread-8-EventThread] ERROR cn.cloudwalk.dist.base.handler.AbstractHandler - Session expired
[2019-10-24 10:02:02.435] [Thread-8-EventThread] INFO  cn.cloudwalk.dist.base.handler.SlaveHandler - slaveExistsWatcher, WatchedEvent: WatchedEvent state:Expired type:None path:null

调查相关资料,发现可能产生问题的原因:

Your client can no longer talk to the ZooKeeper server. The first thing that happened was there was no answer to the heartbeats within the negotiated session timeout:

o.a.s.z.ClientCnxn [INFO] Client session timed out, have not heard from server in 23789ms for sessionid 0x252f862028c0083, closing socket connection and attempting reconnect

Then when it tried to reconnect, it got a connection refused:

o.a.s.z.ClientCnxn [WARN] Session 0x252f862028c0083 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused

This means either your ZooKeeper server:

  • Is not reachable (network connection down)
  • Is dead (so nothing is listening on the socket)
  • Is GCing itself to death and cannot communicate (although that might have issued a connection timeout error, I'm not sure)

To tell more you will need to check the ZooKeeper server logs on your zookeeper cluster.

来源:https://www./content-1-523051.html

    本站是提供个人知识管理的网络存储空间,所有内容均由用户发布,不代表本站观点。请注意甄别内容中的联系方式、诱导购买等信息,谨防诈骗。如发现有害或侵权内容,请点击一键举报。
    转藏 分享 献花(0

    0条评论

    发表

    请遵守用户 评论公约

    类似文章 更多