Yarn(统一资源调度器,任务监控管理器)
- 整合hadoop集群中资源(cpu,内存等)进行统一调度
- 监控map和reduce的执行状况
ResourceManager(master,负责整合资源调度)
NodeManager(slave,负责计算)
关键配置
yarn-site.xml
1 <property> 2 <name>yarn.nodemanager.aux-services</name> 3 <value>mapreduce_shuffle</value> 4 </property> 5 <property> 6 <name>yarn.resourcemanager.hostname</name> 7 <value>hadoop</value> 8 </property>
mapred-site.xml(默认只有mapred-site.xml.template,将其复制一份为xml即可)
1 <property> 2 <name>mapreduce.framework.name</name> 3 <value>yarn</value> 4 </property>
注:start-yarn.sh 只能在resourcemanager节点上执行