Hadoop2.5 Eclipse插件制作、连接集群、及hadoop

问题导读
1.hadoop-eclipse-plugin源码如何下载？
2.插件编译需要哪些准备？
3.使用什么命令可以编译Eclipse插件？



制作过程：


1.首先从git下载源码
下载地址：
https://github.com/winghc/hadoop2x-eclipse-plugin百度网盘下载：
链接：http://pan.baidu.com/s/1o6DMY4U 密码：y95i

下载之后，我们会看到下面文件夹



对于像安装git的用户，可以查看下面帖子
软件版本控制-在Windows中使用Git视频介绍


win7安装 git软件下载以及遇到的问题解决解决方法


2.win7编译

2.1编译准备
win7的编译首先需要安装jdk、ant、Eclipse

同样window也可以进行编译，环境如下图所示这里的
Eclipse的版本：
Version: 3.9.1.201308190730
Build id: 3.9.0.201308190730



Eclipse的安装

Eclipse安装很重要，而且版本很重要，最好能版本一致，因为如果你在Linux制作的插件，只能用于Linux，windows插件只能用于windows，具体的Eclipse版本也尽可能一致，否则会遇到各种问题


2.2编译

aboutyun@master:~/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin$ ant jar -Dversion=2.5.0 -Declipse.home=/usr/eclipse -Dhadoop.home=/usr/hadoop

参数说明：
1.Dversion是你编译的版本，此版本为2.5.0
2.Declipse.home是Eclipse安装目录
3.Dhadoop.home是Hadoop安装目录



执行如下命令：

1. E:\hadoop2x-eclipse-plugin\src\contrib\eclipse-plugin>ant jar -Dversion=2.5.0 -D
   
2. eclipse.home=E:\eclipseIDE -Dhadoop.home=E:\bianyi\hadoop-2.2.0

复制代码

上面Hadoop home需要说明的是，这里并不是使用的Hadoop2.5.0，经过各种验证，Dhadoop.home如果使用的是Hadoop2.5是非常痛苦的，需要各种修改，而且不一定能改正确。那么为什么Hadoop home是Hadoop2.2版本的也可以连接Hadoop2.5，这个需要了解内部原理了，感兴趣的同学，可以研究下。

下面附上在win7下，Eclipse版本为3.9.1的插件
链接：http://pan.baidu.com/s/1dDkZbWH 密码：

游客，如果您要查看本帖隐藏内容请回复

然后尝试连接集群：
首先确认集群版本：



1.把插件放入plugins文件夹
然后打开win7下Eclipse，将hadoop-eclipse-plugin-2.5.0.jar插件放到Eclipse安装目录的plugins文件夹下，我这里的路径是E:\eclipseIDE\plugins
其实放到这里面已经完成大部分了。

2.重启Eclipse
通过window-》preference 找到下图所示，选择Hadoop安装目录



3.创建New hadoop location
通过下面操作，单击other


弹出show view对话框，然后单击map/reduce location



看到小象，如下图



右键New hadoop location

4.配置New hadoop location


5.查看效果
配置完毕，查看效果






问题总结：
插件的制作过程还是比较复杂的，原先使用的是Hadoop2.5版本，编译成功了，不能创建New Hadoop location查看log下面问题

1. java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
   
2.         at org.apache.hadoop.eclipse.server.HadoopServer.<init>(HadoopServer.java:224)
   
3.         at org.apache.hadoop.eclipse.servers.HadoopLocationWizard.<init>(HadoopLocationWizard.java:88)
   
4.         at org.apache.hadoop.eclipse.actions.NewLocationAction$1.<init>(NewLocationAction.java:41)
   
5.         at org.apache.hadoop.eclipse.actions.NewLocationAction.run(NewLocationAction.java:40)
   
6.         at org.eclipse.jface.action.Action.runWithEvent(Action.java:499)
   
7.         at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:584)
   
8.         at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:501)
   
9.         at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:411)
   
10.         at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
    
11.         at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1057)
    
12.         at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4170)
    
13.         at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3759)
    
14.         at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1113)
    
15.         at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    
16.         at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:997)
    
17.         at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:138)
    
18.         at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:610)
    
19.         at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
    
20.         at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:567)
    
21.         at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
    
22.         at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:124)
    
23.         at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    
24.         at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    
25.         at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    
26.         at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:354)
    
27.         at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:181)
    
28.         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    
29.         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    
30.         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    
31.         at java.lang.reflect.Method.invoke(Method.java:606)
    
32.         at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:636)
    
33.         at org.eclipse.equinox.launcher.Main.basicRun(Main.java:591)
    
34.         at org.eclipse.equinox.launcher.Main.run(Main.java:1450)

复制代码

网上有很多这方面的资料，尝试，但是还是未能解决。

使用Hadoop2.5作为编译目录，遇到下面问题：

1. 
   
2. ivy-resolve-common:
   
3. [ivy:resolve]
   
4. [ivy:resolve] :: problems summary ::
   
5. [ivy:resolve] :::: WARNINGS
   
6. [ivy:resolve]           module not found: org.apache.hadoop#hadoop-mapreduce-cli
   
7. ent-jobclient;2.5.0
   
8. [ivy:resolve]   ==== maven2: tried
   
9. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-mapredu
   
10. ce-client-jobclient/2.5.0/hadoop-mapreduce-client-jobclient-2.5.0.pom
    
11. [ivy:resolve]     -- artifact org.apache.hadoop#hadoop-mapreduce-client-jobclien
    
12. t;2.5.0!hadoop-mapreduce-client-jobclient.jar:
    
13. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-mapredu
    
14. ce-client-jobclient/2.5.0/hadoop-mapreduce-client-jobclient-2.5.0.jar
    
15. [ivy:resolve]           module not found: org.apache.hadoop#hadoop-mapreduce-cli
    
16. ent-core;2.5.0
    
17. [ivy:resolve]   ==== maven2: tried
    
18. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-mapredu
    
19. ce-client-core/2.5.0/hadoop-mapreduce-client-core-2.5.0.pom
    
20. [ivy:resolve]     -- artifact org.apache.hadoop#hadoop-mapreduce-client-core;2.5
    
21. .0!hadoop-mapreduce-client-core.jar:
    
22. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-mapredu
    
23. ce-client-core/2.5.0/hadoop-mapreduce-client-core-2.5.0.jar
    
24. [ivy:resolve]           module not found: org.apache.hadoop#hadoop-mapreduce-cli
    
25. ent-common;2.5.0
    
26. [ivy:resolve]   ==== maven2: tried
    
27. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-mapredu
    
28. ce-client-common/2.5.0/hadoop-mapreduce-client-common-2.5.0.pom
    
29. [ivy:resolve]     -- artifact org.apache.hadoop#hadoop-mapreduce-client-common;2
    
30. .5.0!hadoop-mapreduce-client-common.jar:
    
31. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-mapredu
    
32. ce-client-common/2.5.0/hadoop-mapreduce-client-common-2.5.0.jar
    
33. [ivy:resolve]           module not found: org.apache.hadoop#hadoop-hdfs;2.5.0
    
34. [ivy:resolve]   ==== maven2: tried
    
35. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-hdfs/2.
    
36. 5.0/hadoop-hdfs-2.5.0.pom
    
37. [ivy:resolve]     -- artifact org.apache.hadoop#hadoop-hdfs;2.5.0!hadoop-hdfs.ja
    
38. r:
    
39. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-hdfs/2.
    
40. 5.0/hadoop-hdfs-2.5.0.jar
    
41. [ivy:resolve]           module not found: org.apache.hadoop#hadoop-common;2.5.0
    
42. [ivy:resolve]   ==== maven2: tried
    
43. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-common/
    
44. 2.5.0/hadoop-common-2.5.0.pom
    
45. [ivy:resolve]     -- artifact org.apache.hadoop#hadoop-common;2.5.0!hadoop-commo
    
46. n.jar:
    
47. [ivy:resolve]     http://repo1./maven2/org/apache/hadoop/hadoop-common/
    
48. 2.5.0/hadoop-common-2.5.0.jar
    
49. [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
    
50. [ivy:resolve]           ::          UNRESOLVED DEPENDENCIES         ::
    
51. [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
    
52. [ivy:resolve]           :: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2
    
53. .5.0: not found
    
54. [ivy:resolve]           :: org.apache.hadoop#hadoop-mapreduce-client-core;2.5.0:
    
55. not found
    
56. [ivy:resolve]           :: org.apache.hadoop#hadoop-mapreduce-client-common;2.5.
    
57. 0: not found
    
58. [ivy:resolve]           :: org.apache.hadoop#hadoop-hdfs;2.5.0: not found
    
59. [ivy:resolve]           :: org.apache.hadoop#hadoop-common;2.5.0: not found
    
60. [ivy:resolve]           ::::::::::::::::::::::::::::::::::::::::::::::
    
61. [ivy:resolve]
    
62. [ivy:resolve]
    
63. [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
    
64. 
    
65. BUILD FAILED

复制代码

经过多次修改未果，由于Hadoop Eclipse插件源码使用的是Hadoop2.2所以，就尝试用是Hadoop2.2作为Hadoop home，竟然编译成功了。