方案规划 centOS6.10 + win10
- 方案
| 主机名 |
ip |
软件 |
| node1 |
192.168.1.101 |
jdk,zookeeper,hadoop,spark,scala,hbase,hive,kafka,canal |
| node2 |
192.168.1.102 |
jdk,zookeeper,hadoop,spark,scala,hbase,hive,kafka,canal |
| node3 |
192.168.1.103 |
jdk,zookeeper,hadoop,spark,scala,hbase,hive,kafka,canal |
| 物理机 |
192.168.1.111 |
jdk,secureCRT&FX,IDEA |
- 版本
| 集群软件版本列表 | ||
| jdk-8u221 | apache-hive-2.3.7 | |
| hadoop-2.7.3 | hbase-1.4.13 | |
| zookeeper-3.4.5 | sqoop-1.4.7 | |
| spark-2.3.0 | kafka_2.12-2.2.2 | |
| scala-2.11.12 | canal.deployer-1.1.1 | |
| apache-hive-1.2.2 | ||
环境准备：虚拟环境准备
集群搭建
安装JDK
- 解压 tar -zxvf jdk-8u231-linux-x64.tar.gz -C /usr/java/
scp -r java/jdk1.8.0_231/ 192.168.8.102:/usr/java/ scp -r java/jdk1.8.0_231/ 192.168.8.103:/usr/java/ - 修改环境变量 vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_231 export HADOOP_HOME=/cluster/hadoop-2.7.3 export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin - 刷新环境变量 source /etc/profile