编译流程
一、相关安装包
- [hadoop-2.6.5-src.tar.gz] hadoop 源码包
- http://archive.apache.org/dist/hadoop/common/hadoop-2.6.5/hadoop-2.6.5-src.tar.gz
- [jdk-8uxx-linux-x64.tar.gz] jdk1.8
- https://www.oracle.com/java/technologies/downloads/#java8
- [apache-ant-1.9.9-bin.tar.gz] build 工具,打包用
- http://archive.apache.org/dist/ant/binaries/apache-ant-1.9.9-bin.tar.gz
- [apache-maven-3.3.9-bin.tar.gz] maven 环境
- http://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
- [protobuf-2.5.0.tar.gz] 序列化的框架
- https://github.com/protocolbuffers/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
二、环境准备
下面说到的配置环境变量模版如下
#xxx
export XXX_HOME=/xxx/soft/xxx
export PATH=$XXX_HOME/bin:$PATH
解压命令
# tar.gz
tar -zxvf xxx.tar.gz
- 下载源码包
- 下载对应的包,解压即可
- 安装 jdk1.8
- 下载对应的包,解压 tar.gz 包,配置环境变量
- java -version 查看
- 安装 ant 环境
- 下载对应的包,解压 tar.gz 包,配置环境变量
- ant -version 查看
- 安装 maven 环境
- 下载对应的包,解压 tar.gz 包,配置环境变量
- mvn -version 查看
- 修改 maven 源(conf/settings.xml)
<mirrors>
<!-- mirror
| Specifies a repository mirror site to use instead of a given repository. The repository that
| this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
| for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
|-->
<!-- 阿里云仓库 -->
<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>http://maven.aliyun.com/nexus/content/repositories/central/</url>
</mirror>
</mirrors>
- 安装 protobuf 目录
- 下载对应的包,解压 tar.gz 包
- 编译 protobuf
- ./configure
- make
- make check
- make install
- ldconfig
- 配置环境变量
- protoc -version 查看
#protobuf
export LD_LIBRARY_PATH=/xxx/protobuf
export PATH=$LD_LIBRARY_PATH:$PATH
- glibc-headers、g++、make、cmake、openssl、ncurses-devel
- yum install glibc-headers
- yum instal gcc-c++
- yum install make
- yum install cmake
- yum install openssl-devel
- yum install ncurses-devel
三、编译 Hadoop
- 解压并进入文件夹
- 编译命令
- mvn package -Pdist,native -DskipTests -Dtar -Dmaven.javadoc.skip=true
- 编译报错反复执行,有可能是网络问题,失败可以继续执行,接着上次下载继续执行原命令
- 不接着上一次下载要加 clean(mvn clean package…)
- 编译成功后会有 hadoop-2.6.5-src/hadoop-dist/target 这个文件
- mvn package -Pdist,native -DskipTests -Dtar -Dmaven.javadoc.skip=true
四、报错解决
- Hadoop Annotations 程序包 com.sun.javadoc 找不到问题
- 查看本机的 maven 是否已经有 javadoc
- ~/.m2/repository/org/apache/maven/plugins/maven-javadoc-plugin 下已有 2.8.1
- 修改 hadoop 源码包下的 pom.xml 文件中相应的版本号
- 查看本机的 maven 是否已经有 javadoc
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.8.1</version> //修改这行,没有就添加
<inherited>false</inherited>
<executions>
<execution>
<!-- build aggregate javadoc in parent only -->
<id>default-cli</id>
<goals>
<goal>aggregate</goal>
</goals>
<configuration>
<overview>hadoop-common-project/hadoop-common/src/main/java/overview.html</overview>
</configuration>
</execution>
</executions>
</plugin>
- 如果仍然找不到 com.sum.javadoc 包,查看hadoop-common-project/haddop-annotations 目录下的 pom 文件
- 默认为 1.7,修改 jdk 为本机安装的版本