hmaster和hregionserver 16020 端口冲突问题

huhuwoo 2015-10-14

展开全文

问题背景：
最近升级了下hadoop、 hbase、 spark，发现hbase升级后主节点上的hregionsever老是启动不了。
查看日志发现如下端口冲突错误：

java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer
    at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2487)
    at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64)
    at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2502)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2485)
    ... 5 more
Caused by: java.net.BindException: Problem binding to hadoop1/192.168.1.201:16020 : 地址已在使用
    at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2371)
    at org.apache.hadoop.hbase.ipc.RpcServer$Listener.<init>(RpcServer.java:524)
    at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:1899)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:790)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:575)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:492)
    ... 10 more
Caused by: java.net.BindException: 地址已在使用
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:444)
    at sun.nio.ch.Net.bind(Net.java:436)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2369)1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

经过仔细查看官方文档，问题解决。特记录一下。

环境：
hadoop版本 : 2.5.1
hbase 版本 : 1.0.0
spark 版本: 1.3.0
其中hbase主节点上也同时启动了hregionserver服务。

问题原因：
在hbase升级到1.0.0版本后，默认端口做了改动。其中16020端口是hmaster服务和hregionserver服务各自使用的默认端口，导致端口冲突。

官方文档相关信息如下
The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers, which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-master-backup.sh. For each backup master you want to start, add a parameter representing the port offset for that master. Each HMaster uses three ports (16010, 16020, and 16030 by default). The port offset is added to these ports, so using an offset of 2, the backup HMaster would use ports 16012, 16022, and 16032. The following command starts 3 backup servers using ports 16012/16022/16032, 16013/16023/16033, and 16015/16025/16035.

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh command allows you to run multiple RegionServers. It works in a similar way to the local-master-backup.sh command, in that each parameter you provide represents the port offset for an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030. However, the base ports for additional RegionServers are not the default ports since the default ports are used by the HMaster, which is also a RegionServer since HBase version 1.0.0. The base ports are 16200 and 16300 instead. You can run 99 additional RegionServers that are not a HMaster or backup HMaster, on a server. The following command starts four additional RegionServers, running on sequential ports starting at 16202/16302 (base ports 16200/16300 plus 2).

解决方法：
按理说不使用默认配置，定义自己的端口配置就可以解决该问题。
比如使用如下配置：

<property>
<name>hbase.master.port</name>
<value>16000</value>
</property>

<property>
<name>hbase.master.info.port</name>
<value>16010</value>
</property>

<property>
<name>hbase.regionserver.port</name>
<value>16201</value>
</property>

<property>
<name>hbase.regionserver.info.port</name>
<value>16301</value>
</property>
</configuration>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20