准备环境
centOS7
zookeeper
JDK8
安装可参考:
JDK8安装: http://blog.csdn.net/qq_21299835/article/details/79333532
zookeeper集群安装:http://blog.csdn.net/qq_21299835/article/details/79444929
1.创建kafka目录并下载kafka安装包
cd /usr
mkdir kafka
cd kafka
wget http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.0.1/kafka_2.12-1.0.1.tgz
2.解压软件
tar -zxvf kafka_2.12-1.0.1.tgz
3. 在三台虚拟主机上分别重命名解压包
-------------------------------------------------
centOS7_1: mv kafka_2.12-1.0.1 node-01
cd node-01
mkdir logs
centOS7_2: mv kafka_2.12-1.0.1 node-02
cd node-02
mkdir logs
centOS7_3: mv kafka_2.12-1.0.1 node-03
cd node-03
mkdir logs
4. 修改配置文件
(1)切换到config目录下:
cd config
(2)修改server.properties配置文件
主要修改内容如下:
centOS7_1:
broker.id=0 # broker的id,每个kafka节点配置不能一样,可以0,1,2等
port=9092
host.name=192.168.1.217
advertised.host.name=192.168.1.217
listeners=PLAINTEXT://192.168.1.217:9092
advertised.listeners=PLAINTEXT://192.168.1.217:9092
log.dirs=/usr/kafka/node-01/logs
# 消息文件存储的路径,并不是kafka系统日志
# 存放路径,这里不建议存放在/tmp目录下,因为/tmp目录会定时清理
num.partitions=3zookeeper.connect=192.168.1.217:2181,192.168.1.216:2182,192.168.1.218:2183
# 指定连接Zookeeper的连接串,此处填写上一节中安装的三个zk节点的ip
# 和端口即可
centOS7_2:
broker.id=1
port=9092
host.name=192.168.1.216
advertised.host.name=192.168.1.216
listeners=PLAINTEXT://192.168.1.216:9092
advertised.listeners=PLAINTEXT://192.168.1.216:9092
num.partitions=3
log.dirs=/usr/kafka/node-02/logs
zookeeper.connect=192.168.1.217:2181,192.168.1.216:2182,192.168.1.218:2183
centOS7_3:
broker.id=2
port=9092
host.name=192.168.1.218
advertised.host.name=192.168.1.218
listeners=PLAINTEXT://192.168.1.218:9092
advertised.listeners=PLAINTEXT://192.168.1.218:9092
num.partitions=3
log.dirs=/usr/kafka/node-03/logszookeeper.connect=192.168.1.217:2181,192.168.1.216:2182,192.168.1.218:2183
5. 启动zookeeper服务
6. 启动kafka服务
cd bin
centOS7_01:
./kafka-server-start.sh /usr/kafka/node-01/config/server.properties &
centOS7_02:
./kafka-server-start.sh /usr/kafka/node-02/config/server.properties &
centOS7_03:
./kafka-server-start.sh /usr/kafka/node-03/config/server.properties &
注:启动kafka服务时,可能报以下错误:
commit_memory(0x00000000c0000000, 1073741824, 0) failed; error='Cannot alloc
可修改kafka-server-start.sh 启动文件
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G" 为 -Xmx256M -Xms128M"
7. 打开9092端口
firewall-cmd --zone=public --add-port=9092/tcp --permanent
8. 重启防火墙
systemctl restart firewalld.service
9. 测试
(1)在centOS7_01 (192.168.1.217)中kafka的bin中输入
./kafka-console-producer.sh --broker-list 192.168.1.217:9092,192.168.1.216:9092 --topic my-replicated-topic5
在centOS7_02 (192.168.1.216)中kafka的bin中输入
./kafka-console-consumer.sh --zookeeper 192.168.1.217:2181,192.168.1.216:2182 --from-beginning --topic my-replicated-topic5
在centOS7_01中输入能在centOS7_02接收到则配置成功
centOS7_01: centOS7_02: