目录
修改五个配置文件
位于Hadoop安装路径下etc/hadoop/下
配置一下文件中确保ip映射文件配置完成(/etc/hosts)
扫描二维码关注公众号,回复:
14632140 查看本文章
1.Hadoop-env.sh文件
将原内容删掉,把java路径添加进文件中
export $JAVA_HOME=java路径
2.core-site.xml文件
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://主机名:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:Hadoop目录/tmp</value>
</property>
<property>
<name>oi.file.buffer.size</name>
<value>131072</value>
</property>
</configuration>
3.hdfs-site.xml
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:hadoop目录/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:hadoop目录/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
</configuration>
4.mapred-site.xml文件
该文件etc/hadoop/目录下无,需要自行拷贝重命名
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>主机名:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>主机名:19888</value>
</property>
</configuration>
5.yarn-site.xml
<configuration>
<property>
<name>yarn.resourcemanager.address</name>
<value>主机名:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>主机名:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>主机名:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>主机名:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>主机名:8088</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
一定要把防火墙提取关掉,否则会报很多莫名其妙的错
具体参数根据需求自行配置(本文范例)
进程运行在主节点还是从节点可根据情况指定主机