大数据： zookeeper连接数导致kafka storm不能正常工作_CDerek

http://blog.sina.com.cn/u/1087663810

首页博文目录关于我

个人资料

微博

加好友发纸条

写留言加关注

博客等级：
博客积分：

博客访问：
关注人气：
获赠金笔：0支
赠出金笔：0支
荣誉徽章：

正文字体大小：大中小

大数据： zookeeper连接数导致kafka storm不能正常工作

(2014-08-21 10:09:30)

标签：

股票

问题描述：

kafka list-topic报错：

[2014-08-21 10:08:22,728] WARN Session 0x0 for server AY1405161104217929f5Z/10.162.219.53:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)

java.io.IOException: Connection reset by peer

at sun.nio.ch.FileDispatcher.read0(Native Method)

at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)

at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)

at sun.nio.ch.IOUtil.read(IOUtil.java:218)

at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254)

at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:859)

at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1157)

storm tail -1000 worker-6702.log 日志报错如下：

2014-08-21 10:08:57 b.s.d.executor [ERROR]

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/topics/shakatopic/partitions

at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:103) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

at backtype.storm.daemon.executor$fn__5573$fn__5588$fn__5617.invoke(executor.clj:563) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]

at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]

at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.5.1.jar:na]

at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_30]

Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/topics/shakatopic/partitions

at storm.kafka.DynamicBrokersReader.getBrokerInfo(DynamicBrokersReader.java:81) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:79) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

... 6 common frames omitted

Caused by: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/topics/shakatopic/partitions

at storm.kafka.DynamicBrokersReader.getNumPartitions(DynamicBrokersReader.java:94) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

at storm.kafka.DynamicBrokersReader.getBrokerInfo(DynamicBrokersReader.java:65) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

... 7 common frames omitted

Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/topics/shakatopic/partitions

at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) ~[zookeeper-3.4.5.jar:3.4.5-1392090]

at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) ~[zookeeper-3.4.5.jar:3.4.5-1392090]

at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1586) ~[zookeeper-3.4.5.jar:3.4.5-1392090]

at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214) ~[curator-framework-2.4.0.jar:na]

at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203) ~[curator-framework-2.4.0.jar:na]

at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) ~[curator-client-2.4.0.jar:na]

at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199) ~[curator-framework-2.4.0.jar:na]

at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191) ~[curator-framework-2.4.0.jar:na]

at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38) ~[curator-framework-2.4.0.jar:na]

at storm.kafka.DynamicBrokersReader.getNumPartitions(DynamicBrokersReader.java:91) ~[storm-kafka-0.9.2-incubating.jar:0.9.2-incubating]

... 8 common frames omitted

zookeeper的zookeeper.out 日志：

2014-08-21 10:15:34,446 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:35,401 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:36,029 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:38,207 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:38,932 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:40,784 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:40,820 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:43,014 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:43,742 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:45,845 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:46,104 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:47,775 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:48,215 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:50,036 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:50,110 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:52,215 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:52,422 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:54,829 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

2014-08-21 10:15:55,398 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@247] - Too many connections from /10.162.219.53 - max is 10

解决方法：

# add by shaka

# set max client connects

maxClientCnxns=300

阅读┊ 收藏 ┊ 喜欢 ▼ ┊打印┊举报/Report

前一篇：大数据：zookeeper读取data时的注意点

后一篇：大数据：huekill长时间slepp的任务问题。

新浪BLOG意见反馈留言板　欢迎批评指正