手上的一个HBase相关的服务在重启后开始报错(重启前运行良好),错误信息如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
2017-04-25 16:57:59 [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.retryOrThrow] [ERROR] ZooKeeper exists failed after 4 attempts 2017-04-25 16:57:59 [org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.keeperException] [ERROR] hconnection-0x5ee19fb40x0, quorum=zk1.com:2181,zk2.com:2181,zk3.com:2181,zk4.com:2181,zk5.com:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.checkZk(RecoverableZooKeeper.java:145) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119) |
错误信息提示连接zookeeper时间超长。经检查是因为有一个zookeeper节点已经停止运行(zk3.com),和运维商量后将zookeeper故障节点删除,并移掉了另一个节点,剩下了3个节点。修改配置文件并重启后服务恢复。
#######
发表评论