hadoop2.7.4编译

之前一直没有记录处理一些工作中问题的记录,一段时间后就完全忘记了,以后坚持记录吧,毕竟作为笔记也是极好的。

编译按照hadoop网上也有很多,我只是记录我的编译过程。

环境:

  1.mac

  2.java1.8

  3.mvn3.5

  4.protobuf2.5

在基本环境搭建完成之后我就开始了编译,命令:mvn clean package -Pdist,native -DskipTests -Dtar

接着就出现了各种缺少东西的错,基本跟这位的错一致:cmake、ant、findbugs,我在网上查到一个哥么的编译过程,报错跟我基本一样,如下:

cmake:

Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on
project hadoop-pipes: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part …<exec dir=”/home/pory/workplace/hadoop-2.4.1-src/hadoop-
tools/hadoop-pipes/target/native” executable=”cmake” failonerror=”true”>… @ 5:131 in
/home/pory/workplace/hadoop-2.4.1-src/hadoop-tools/hadoop-pipes/target/antrun/build-main.xml

findbugs:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project hadoop-common: An Ant BuildException has occured: stylesheet /home/hadoop/hadoop-2.7.1-src/hadoop-common-project/hadoop-common/${env.FINDBUGS_HOME}/src/xsl/default.xsl doesn’t exist.

[ERROR] around Ant part …<xslt in=”/home/hadoop/hadoop-2.7.1-src/hadoop-common-project/hadoop-common/target/findbugsXml.xml” style=”${env.FINDBUGS_HOME}/src/xsl/default.xsl” out=”/home/hadoop/hadoop-2.7.1-src/hadoop-common-project/hadoop-common/target/site/findbugs.html”/>… @ 43:251 in /home/hadoop/hadoop-2.7.1-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml

ant:

Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program “cmake” (in directory “/root/hadoop-2.7.1-src/hadoop-common-project/hadoop-common/target/native”): error=2, No such file or directory

这些就添加就好。

最难的一个问题对于我来说就是openssl:

在按照上面的所有依赖的软件之后出现如下的错,而且编译也接近最后,实在不甘心也急于解决,就各种开始网上大搜索。

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-pipes: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failonerror="true" dir="/Users/shihaolin/opt/software/hadoop-2.7.4-src/hadoop-tools/hadoop-pipes/target/native" executable="cmake">... @ 5:140 in /Users/shihaolin/opt/software/hadoop-2.7.4-src/hadoop-tools/hadoop-pipes/target/antrun/build-main.xml

最后在一个帖子里发现跟我一样的错误,按照他给出的方案进行了解决。而且我也跟这个哥么一样是在这个问题上花费了很长的时间,才解决这个问题。解决步骤如下:

  1):终端执行

cmake  /Users/shihaolin/opt/software/hadoop-2.7.4-src/hadoop-tools/hadoop-pipes/src/ -DJVM_ARCH_DATA_MODEL=64

执行成功后

  2):打开配置Java环境变量的地方

  3):添加下面两行(具体根据自己电脑上的openssl安装版本还有文件地址配置)

    export OPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2l

    export OPENSSL_INCLUDE_DIR=/usr/local/Cellar/openssl/1.0.2l/include

  4):记住修改环境变量后一定要执行立即生效命令,然后再执行

   cmake  /Users/shihaolin/opt/software/hadoop-2.7.4-src/hadoop-tools/hadoop-pipes/src/ -DJVM_ARCH_DATA_MODEL=6

这个如果执行成功后再执行,mvn clean package -Pdist,native -DskipTests -Dtar

  5):坐等编译成功

以上就是我的一次编译hadoop2.7.4的过程

编译过程中参考的俩个网上的哥么的文章如下:

  1.http://blog.pengduncun.com/?p=1208

  2.https://blog.csdn.net/syso_alt_hao/article/details/77655862

猜你喜欢

转载自www.cnblogs.com/CulverMei/p/9047624.html