目录
2: 修改配置文件 conf/flink-conf.yaml
官方文档: https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/jobmanager_high_availability.html
配置stanlone cluster high availability集群模式:
3台服务器: Machine4, Machine5, Machine6
前提条件: 已经部署了hadoop, zookeeper
1: 下载压缩包
浏览器地址: https://www.apache.org/dyn/closer.lua/flink/flink-1.7.2/flink-1.7.2-bin-hadoop26-scala_2.11.tgz
服务器下载命令: wget http://mirror.bit.edu.cn/apache/flink/flink-1.7.2/flink-1.7.2-bin-hadoop26-scala_2.11.tgz
2: 修改配置文件 conf/flink-conf.yaml
jobmanager.heap.size: 1024m
taskmanager.heap.size: 4096m
taskmanager.numberOfTaskSlots: 4
high-availability: zookeeper
high-availability.zookeeper.storageDir: hdfs://Machine4:8020/flink/recovery
high-availability.zookeeper.path.root: /flink
high-availability.cluster-id: /flinkCluster
high-availability.zookeeper.quorum: Machine4:2181,Machine5:2181,Machine6:2181
注释:
1、TaskManager总共能使用的内存大小(taskmanager.heap.mb)
2、每一台机器上能使用的CPU个数(taskmanager.numberOfTaskSlots)
3: 修改配置文件 conf/masters
Machine4:8081
Machine5:8081
4: 修改配置文件 conf/slaves
Machine4
Machine5
Machine6
5: 将flink复制到其他节点(相同位置)
scp -r flink-1.7.2 [email protected]:/root/app/
6:Start an HA-cluster(开启集群):
[root@Machine4 flink-1.7.2]# bin/start-cluster.sh
7:Stop an HA-cluster(关闭集群)
[root@Machine4 flink-1.7.2]#bin/stop-cluster.sh
服务器显示如下:
Machine4服务器:
命令:jps -m
Machine5服务器:
Machine6服务器: