https://github.com/mvillarrealb/docker-spark-cluster
1:安装前准备
-
Docker 安装好
-
Docker compose 安装好
2:构建镜像
下载下来放在/opt 目录下 进入docker-spark-clouder目录下 执行
chmod +x build-images.sh
./build-images.sh
接下来将会下载以下镜像
-
spark-base:2.3.1: A base image based on java:alpine-jdk-8 wich ships scala, python3 and spark 2.3.1
-
spark-master:2.3.1: A image based on the previously created spark image, used to create a spark master containers.
-
spark-worker:2.3.1: A image based on the previously created spark image, used to create spark worker containers.
-
spark-submit:2.3.1: A image based on the previously created spark image, used to create spark submit containers(run, deliver driver and die gracefully).
3:执行docker-compose
执行 docker-compose up
接下来将会创建集群