1、安装Hadoop:brew install Hadoop
2、ssh 免密登陆:见上篇博客
3、修改Hadoop配置文件:
3.1 hadoop-env.sh文件
进入安装目录cd /usr/local/Cellar/hadoop/2.8.0/libexec/etc/hadoop,找到并打开 hadoop-env.sh 文件,将
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"
改为
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true -Djava.security.krb5.realm=true -Djava.security.krb5.kdc=true"
export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home"
(java_home请写上你本机上jdk安装的位置,可以用命令/usr/libexec/java_home查看jdk位置)
3.2 core-site.xml文件
进入目录cd /usr/local/Cellar/hadoop/2.8.0/libexec/etc/hadoop,打开core-site.xml将<configuration></configuration>替换为
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/local/Cellar/hadoop/hdfs/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
</property>
</configuration>
3.3 mapred-site.xml文件
还是进入目录/usr/local/Cellar/hadoop/2.8.0/libexec/etc/hadoop,修改mapred-site.xml,同样将<configuration></configuration>替换为
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:8021</value>
</property>
</configuration>
3.4 hdfs-site.xml文件
还是进入目录/usr/local/Cellar/hadoop/2.8.0/libexec/etc/hadoop,打开hdfs-site.xml加上
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
4、修改完之后先退出终端,然后输入命令 ssh localhost 进入ssh环境下
5、执行命令hdfs namenode -format
在终端会显示成功。
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at localhost/127.0.0.1
************************************************************/
6、配置Hadoop环境变量
vim ~/.bash_profile
在文件最前面加上:export PATH=$PATH:/usr/local/Cellar/hadoop/3.1.1/sbin:/usr/local/Cellar/hadoop/3.1.1/bin
source ~/.bash_profile