------------------------------------------------
1.zookeeper正常安装并启动,参看:https://blog.csdn.net/With__Sunshine/article/details/88538888
------------------------------------------------
2.修改spark-env.sh,配置如下
#注释内容如下:
#SPARK_MASTER_HOST=s101
#SPARK_MASTER_PORT=7077
#添加内容如下:
export SPARK_DAEMON_JAVA_OPTS="
-Dspark.deploy.recoveryMode=ZOOKEEPER
-Dspark.deploy.zookeeper.url=s101,102,s103,s104
-Dspark.deploy.zookeeper.dir=/spark"
------------------------------------------------
3.分发文件
cd /soft/spark/conf
scp -r spark-env.sh centos@s102:/soft/spark/conf
scp -r spark-env.sh centos@s103:/soft/spark/conf
scp -r spark-env.sh centos@s104:/soft/spark/conf
------------------------------------------------
4.在s101上启动全部节点
cd /soft/spark/sbin
./start-all.sh
------------------------------------------------
5.在s102上启动master节点
cd /soft/spark/sbin
./start-master.sh
------------------------------------------------
6.spark HA访问集群
/soft/spark/bin/spark-shell \
--master spark://s101:7077,s102:7077 \
--executor-memory 2g \
--total-executor-cores 2