版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/u012292754/article/details/86592517
1 Spark HistoryServer 配置
spark-env.sh
SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory=file:///home/hadoop/appsData/sparkData/historylogs
spark-defaults.conf
spark.eventLog.enabled true
spark.eventLog.dir file:///home/hadoop/appsData/sparkData/historylogs
spark.eventLog.compress true
- 启动
sbin/start-history-server.sh
[hadoop@node1 spark-2.2.2-bin-2.6.0-cdh5.7.0]$ sbin/start-history-server.sh
starting org.apache.spark.deploy.history.HistoryServer, logging to /home/hadoop/apps/spark-2.2.2-bin-2.6.0-cdh5.7.0/logs/spark-hadoop-org.apache.spark.deploy.history.HistoryServer-1-node1.out
[hadoop@node1 spark-2.2.2-bin-2.6.0-cdh5.7.0]$ jps
1825 Worker
1906 HistoryServer
1702 Master
1965 Jps
http://node1:18080/