zookeeper环境参照《Zookeeper集群环境搭建》
1、下载解压kafka
在宿主机器内操作
$ wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/1.1.1/kafka_2.11-1.1.1.tgz
$ tar xvf kafka_2.11-1.1.1.tgz
通过docker volumes将kafka映射到docker容器中。
2、启动docker集群
在宿主主机内操作
编写docker-compose.yml如下
version: '2'
networks:
thinking:
services:
test_1.thinking.com:
image: ubuntu
ports:
- "3333:3333"
container_name: test_1.thinking.com
command: /bin/bash
tty: true
networks:
- thinking
volumes:
- /etc/java-8-openjdk:/etc/java-8-openjdk
- /usr/lib/jvm/java-8-openjdk-amd64:/usr/lib/jvm/java-8-openjdk-amd64
- /home/yong/Desktop/ZK-test/zookeeper-3.4.12:/home/zookeeper-3.4.12
- /home/yong/Desktop/kafka-test/kafka_2.11-1.1.1:/usr/local/kafka
environment:
- JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
- PATH=${PATH}:/usr/lib/jvm/java-8-openjdk-amd64/bin:/usr/lib/jvm/java-8-openjdk-amd64/jre/bin
test_2.thinking.com:
image: ubuntu
ports:
- "4444:4444"
container_name: test_2.thinking.com
command: /bin/bash
tty: true
networks:
- thinking
volumes:
- /etc/java-8-openjdk:/etc/java-8-openjdk
- /usr/lib/jvm/java-8-openjdk-amd64:/usr/lib/jvm/java-8-openjdk-amd64
- /home/yong/Desktop/ZK-test/zookeeper-3.4.12:/home/zookeeper-3.4.12
- /home/yong/Desktop/kafka-test/kafka_2.11-1.1.1:/usr/local/kafka
environment:
- JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
- PATH=${PATH}:/usr/lib/jvm/java-8-openjdk-amd64/bin:/usr/lib/jvm/java-8-openjdk-amd64/jre/bin
test_3.thinking.com:
image: ubuntu
ports:
- "5555:5555"
container_name: test_3.thinking.com
command: /bin/bash
tty: true
networks:
- thinking
volumes:
- /etc/java-8-openjdk:/etc/java-8-openjdk
- /usr/lib/jvm/java-8-openjdk-amd64:/usr/lib/jvm/java-8-openjdk-amd64
- /home/yong/Desktop/ZK-test/zookeeper-3.4.12:/home/zookeeper-3.4.12
- /home/yong/Desktop/kafka-test/kafka_2.11-1.1.1:/usr/local/kafka
environment:
- JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
- PATH=${PATH}:/usr/lib/jvm/java-8-openjdk-amd64/bin:/usr/lib/jvm/java-8-openjdk-amd64/jre/bin
执行
$ docker-compose -f docker-compose.yml up -d
3、借助kafka自带daemon运行单机服务
进入test_1.thinking.com容器
$ docker exec -it test_1.thinking.com /bin/bash
启动kafka
# cd /usr/local/kafka/bin
# ./kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties
创建主题并查询
# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
# ./kafka-topics.sh --zookeeper localhost:2181 --describe --topic test
Topic:test PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test Partition: 0 Leader: 0 Replicas: 0 Isr: 0
向主题发消息,以ctrl+D结束
# ./kafka-console-producer.sh --broker-list localhost:9092 --topic test
>Test Msg 0001
>Test Msg 0002
从主题读取消息
# ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
Test Msg 0001
Test Msg 0002
^CProcessed a total of 2 messages