阿里云-云小站(无限量代金券发放中)
【腾讯云】云服务器、云数据库、COS、CDN、短信等热卖云产品特惠抢购

Maven编译hadoop-2.2.0-src

185次阅读
没有评论

共计 6864 个字符,预计需要花费 18 分钟才能阅读完成。

Maven 编译 Hadoop-2.2.0-src

1. 环境

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /opt/maven
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /usr/java/jdk1.7.0_45/jre
Default locale: en_US, platform encoding: UTF-8
OS name: “linux”, version: “2.6.18-371.3.1.el5”, arch: “amd64”, family: “unix”

2. 执行

mvn package -DskipTests -Pdist,native,docs -Dtar

3. 插曲 1

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found

打补丁

Patch :https://issues.apache.org/jira/browse/HADOOP-10110

方法:修改 hadoop 源码目录下的 hadoop-common-project/hadoop-auth/pom.xml

添加或修改如下部分内容:

      <dependency>
          <groupId>org.mortbay.jetty</groupId>
      <artifactId>jetty-util</artifactId>
      <scope>test</scope>
  </dependency>
    <dependency>
      <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty</artifactId>
        <scope>test</scope>
    </dependency>

4. 插曲 2

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program “cmake” (in directory “/opt/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/target/native”): error=2, No such file or directory -> [Help 1]

没装 cmake

人家就是想要这个地方 这可不行  果断

yum install cmake

5. 插曲 3

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc –version’ did not return a version -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc –version’ did not return a version

没装 protobuf-2.5.0。

下载 protobuf-2.5.0.tar.gz  解压  3 部曲编译安装。

6. 继续 (这次学能了,带着 debug)

mvn package -Pdist,native -DskipTests -Dtar -e -X

等了好久好久

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ………………………….. SUCCESS [2:18.165s]
[INFO] Apache Hadoop Project POM ……………………. SUCCESS [50.526s]
[INFO] Apache Hadoop Annotations ……………………. SUCCESS [1:14.522s]
[INFO] Apache Hadoop Assemblies …………………….. SUCCESS [0.760s]
[INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [5.983s]
[INFO] Apache Hadoop Maven Plugins ………………….. SUCCESS [13.161s]
[INFO] Apache Hadoop Auth ………………………….. SUCCESS [10.647s]
[INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [5.010s]
[INFO] Apache Hadoop Common ………………………… SUCCESS [4:32.249s]
[INFO] Apache Hadoop NFS …………………………… SUCCESS [18.927s]
[INFO] Apache Hadoop Common Project …………………. SUCCESS [0.134s]
[INFO] Apache Hadoop HDFS ………………………….. SUCCESS [5:29.380s]
[INFO] Apache Hadoop HttpFS ………………………… SUCCESS [1:43.219s]
[INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [44.693s]
[INFO] Apache Hadoop HDFS-NFS ………………………. SUCCESS [13.217s]
[INFO] Apache Hadoop HDFS Project …………………… SUCCESS [0.219s]
[INFO] hadoop-yarn ………………………………… SUCCESS [1.957s]
[INFO] hadoop-yarn-api …………………………….. SUCCESS [1:36.233s]
[INFO] hadoop-yarn-common ………………………….. SUCCESS [1:31.987s]
[INFO] hadoop-yarn-server ………………………….. SUCCESS [0.334s]
[INFO] hadoop-yarn-server-common ……………………. SUCCESS [26.932s]
[INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [49.536s]
[INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [7.882s]
[INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [1:01.936s]
[INFO] hadoop-yarn-server-tests …………………….. SUCCESS [6.642s]
[INFO] hadoop-yarn-client ………………………….. SUCCESS [11.930s]
[INFO] hadoop-yarn-applications …………………….. SUCCESS [0.437s]
[INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [12.064s]
[INFO] hadoop-mapreduce-client ……………………… SUCCESS [1.363s]
[INFO] hadoop-mapreduce-client-core …………………. SUCCESS [58.193s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [11.262s]
[INFO] hadoop-yarn-site ……………………………. SUCCESS [0.328s]
[INFO] hadoop-yarn-project …………………………. SUCCESS [17.453s]
[INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [43.479s]
[INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [6.048s]
[INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [46.449s]
[INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [9.770s]
[INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [17.249s]
[INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [9.929s]
[INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [12.193s]
[INFO] hadoop-mapreduce ……………………………. SUCCESS [6.817s]
[INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [10.957s]
[INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [32.407s]
[INFO] Apache Hadoop Archives ………………………. SUCCESS [4.950s]
[INFO] Apache Hadoop Rumen …………………………. SUCCESS [18.817s]
[INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [10.909s]
[INFO] Apache Hadoop Data Join ……………………… SUCCESS [6.662s]
[INFO] Apache Hadoop Extras ………………………… SUCCESS [11.594s]
[INFO] Apache Hadoop Pipes …………………………. SUCCESS [3.737s]
[INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [3.623s]
[INFO] Apache Hadoop Tools …………………………. SUCCESS [0.127s]
[INFO] Apache Hadoop Distribution …………………… SUCCESS [2:55.243s]
[INFO] Apache Hadoop Client ………………………… SUCCESS [25.929s]
[INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [2.516s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 33:38.762s
[INFO] Finished at: Sat Dec 28 21:59:12 CST 2013
[INFO] Final Memory: 68M/200M
[INFO] ————————————————————————

注:第一次运行,一般要下很多 jar 包

有时候会出现下载的时候下面这样的东西引起的一大片错误

 Connection to http://repo.maven.apache.org refused

只要重新执行

mvn package -Pdist,native -DskipTests -Dtar -e -X

就能接着下,一般是网络问题。

Maven 权威指南_中文完整版清晰 PDF http://www.linuxidc.com/Linux/2014-06/103690.htm

Linux 安装 Maven http://www.linuxidc.com/Linux/2013-05/84489.htm

Maven3.0 配置和简单使用 http://www.linuxidc.com/Linux/2013-04/82939.htm

Ubuntu 下搭建 sun-jdk 和 Maven2 http://www.linuxidc.com/Linux/2012-12/76531.htm

Maven 使用入门 http://www.linuxidc.com/Linux/2012-11/74354.htm

更多 Hadoop 相关信息见 Hadoop 专题页面 http://www.linuxidc.com/topicnews.aspx?tid=13

正文完
星哥玩云-微信公众号
post-qrcode
 0
星锅
版权声明:本站原创文章,由 星锅 于2022-01-20发表,共计6864字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
【腾讯云】推广者专属福利,新客户无门槛领取总价值高达2860元代金券,每种代金券限量500张,先到先得。
阿里云-最新活动爆款每日限量供应
评论(没有评论)
验证码
【腾讯云】云服务器、云数据库、COS、CDN、短信等云产品特惠热卖中