在部署3个节点的Zookeeper伪分布式集群这篇文章的基础上,部署3个节点的Kafka伪分布式集群
Kafka集群不需要像Zookeeper集群似的拷贝多份程序,只需要启动时指定不同的配置文件即可,只需要一份程序
1、下载
cd /opt/
wget https://archive.apache.org/dist/kafka/2.1.0/kafka_2.12-2.1.0.tgz
2、解压并重命名
tar -zxvf kafka_2.12-2.1.0.tgz
mv kafka_2.12-2.1.0 kafka
3、修改配置文件
vi kafka/config/server.properties
broker.id=0
listeners=PLAINTEXT://服务器ip:9092
log.dirs=/opt/data/kafka/kafka-logs-1
delete.topic.enable=true
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
mv kafka/config/server1.properties
cp kafka/config/server1.properties kafka/config/server2.properties
cp kafka/config/server1.properties kafka/config/server3.properties
vi kafka/config/server2.properties
broker.id=1
listeners=PLAINTEXT://服务器ip:9093
log.dirs=/opt/data/kafka/kafka-logs-2
delete.topic.enable=true
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
vi kafka/config/server3.properties
broker.id=2
listeners=PLAINTEXT://服务器ip:9094
log.dirs=/opt/data/kafka/kafka-logs-3
delete.topic.enable=true
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
4、分别启动kafka3个节点
cd kafka/
./bin/kafka-server-start.sh config/server1.properties &
./bin/kafka-server-start.sh config/server2.properties &
./bin/kafka-server-start.sh config/server3.properties &
5、如果启动过程中因为内存不足,启动失败,可以修改kafka启动时jvm堆内存大小
vi /opt/kafka/bin/kafka-server-start.sh
将如下代码中1G改为512m
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-Xmx512m -Xms512m"
fi
再次启动即可