注意:
我这里需要springboot集成kafka,所以需要安装。集成会有报错,解决见最后。
kafka依赖zookeeper,需要先安装zk,参考:
https://blog.csdn.net/weixin_38750084/article/details/90548149
第一步:
搜索镜像
docker search kafka 查看有哪些镜像可用。
[root@Master config]# docker search kafka
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
wurstmeister/kafka Multi-Broker Apache Kafka Image 919 [OK]
spotify/kafka A simple docker image with both Kafka and Zo… 352 [OK]
sheepkiller/kafka-manager kafka-manager 163 [OK]
ches/kafka Apache Kafka. Tagged versions. JMX. Cluster-… 112 [OK]
bitnami/kafka Apache Kafka is a distributed streaming plat… 52 [OK]
hlebalbau/kafka-manager Kafka Manager Docker Images Build. 37 [OK]
landoop/kafka-topics-ui UI for viewing Kafka Topics config and data … 26 [OK]
solsson/kafka http://kafka.apache.org/documentation.html#q… 14 [OK]
landoop/kafka-lenses-dev Lenses with Kafka. +Connect +Generators +Con… 14 [OK]
debezium/kafka Kafka image required when running the Debezi… 13 [OK]
digitalwonderland/kafka Latest Kafka - clusterable 12 [OK]
danielqsj/kafka-exporter Kafka exporter for Prometheus 10 [OK]
landoop/kafka-connect-ui Web based UI for Kafka Connect. 8 [OK]
sourcelaborg/kafka-webview Easy-to-use web based interface for reading … 6
mesosphere/kafka-client Kafka client 3 [OK]
openwhisk/kafkaprovider OpenWhisk trigger provider services for kafk… 2 [OK]
phenompeople/kafka Apache Kafka is a distributed streaming plat… 2 [OK]
pceric/kafka-offset-lag-for-prometheus Expose kafka consumer offset lag to promethe… 1 [OK]
datamountaineer/kafka-connect-cassandra Kafka Connect Image for Cassandra Source and… 1
solsson/kafka-manager https://github.com/yahoo/kafka-manager 1 [OK]
jamessoubry/kafka-mesos-exporter Kafka-Mesos Exporter for Prometheus 0 [OK]
enmasseproject/kafka-inmemory This image is obsolete, has been replaced by… 0
solsson/kafka-prometheus-jmx-exporter For monitoring of Kubernetes Kafka clusters … 0 [OK]
anchorfree/kafka Kafka broker and Zookeeper image 0
zenreach/kafka-connect Zenreach's Kafka Connect Docker Image 0
[root@Master config]#
第二步:
拉取镜像
docker pull wurstmeister/kafka
[root@Master config]# docker pull wurstmeister/kafka
Using default tag: latest
latest: Pulling from wurstmeister/kafka
bdf0201b3a05: Pull complete
9e12771959ad: Pull complete
ae272eb2814b: Pull complete
a2dae775166f: Pull complete
4d53f6c7ca48: Pull complete
aa58da8c41d7: Pull complete
Digest: sha256:1a70e7311d49a534cd25e72aa9e2d63b141c02b695bf61cd65195483c402367d
Status: Downloaded newer image for wurstmeister/kafka:latest
[root@Master config]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
zookeeper latest 8eccc77fd8d0 3 weeks ago 150MB
wurstmeister/kafka latest c364cbed5b86 6 weeks ago 421MB
mysql 5.7.25 98455b9624a9 2 months ago 372MB
amancevice/superset latest 26506f00f388 2 months ago 1.4GB
hello-world latest fce289e99eb9 5 months ago 1.84kB
redis 3.2 87856cc39862 7 months ago 76MB
[root@Master config]#
第三步:
[root@Master config]# docker run -d --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=192.168.65.113:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://192.168.124.12:9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -t wurstmeister/kafka:2.12-2.1.0
Unable to find image 'wurstmeister/kafka:2.12-2.1.0' locally
2.12-2.1.0: Pulling from wurstmeister/kafka
6c40cc604d8e: Pull complete
e78b80385239: Pull complete
f41fe1b6eee3: Pull complete
721a465103fa: Pull complete
fc5a499bfd17: Pull complete
2dbf197dc259: Pull complete
Digest: sha256:2d61489969d56bc9341097ab621283d6803b4cf0245d6af9a83191a0a2378074
Status: Downloaded newer image for wurstmeister/kafka:2.12-2.1.0
WARNING: IPv4 forwarding is disabled. Networking will not work.
ccd9c11429bdcc97234095731c4db22b476f23b634b13578e82fc7692b3b21c5
[root@Master config]#
root@Master config]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ccd9c11429bd wurstmeister/kafka:2.12-2.1.0 "start-kafka.sh" 46 seconds ago Up 44 seconds 0.0.0.0:9092->9092/tcp kafka
3108b2bbef86 redis:3.2 "docker-entrypoint.s…" 9 days ago Up 9 days 0.0.0.0:6379->6379/tcp hardcore_banach
590a923a5fcd zookeeper "/docker-entrypoint.…" 9 days ago Up 9 days 2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp zookeeper
aac0f7b27727 mysql:5.7.25 "docker-entrypoint.s…" 2 weeks ago Up 9 days 0.0.0.0:3306->3306/tcp, 33060/tcp mysql_springboot
5dd1c0bc2d60 amancevice/superset "gunicorn superset:a…" 2 weeks ago Up 9 days (healthy) 0.0.0.0:8088->8088/tcp superset
[root@Master config]#
因为我开了防火墙,这里开启9092端口:
[root@Master config]# firewall-cmd --list-ports
80/tcp 3306/tcp 8088/tcp 2181/tcp 7712/tcp
[root@Master config]# firewall-cmd --zone=public --add-port=9092/tcp --permanent
success
[root@Master config]# firewall-cmd --list-ports
80/tcp 3306/tcp 8088/tcp 2181/tcp 7712/tcp
[root@Master config]# firewall-cmd --reload
bash: firewall-cmd: 未找到命令...
[root@Master config]# firewall-cmd --reload
success
[root@Master config]# firewall-cmd --list-ports
80/tcp 3306/tcp 8088/tcp 2181/tcp 7712/tcp 9092/tcp
[root@Master config]#
启动springboot项目依然报错:
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'testController': Unsatisfied dependency expressed through field 'kafkaTemplate'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'kafkaProducerConfig': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'kafka.producer.servers' in value "${kafka.producer.servers}"
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:586) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:91) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:372) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:572) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:759) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869) ~[spring-context-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) ~[spring-context-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:762) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:398) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:330) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1258) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246) [spring-boot-2.0.4.RELEASE.jar:2.0.4.RELEASE]
at com.tzb.myspringboot.ServiceApplication.main(ServiceApplication.java:27) [classes/:na]
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'kafkaProducerConfig': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'kafka.producer.servers' in value "${kafka.producer.servers}"
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:378) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1341) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:572) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:372) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1247) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1096) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:535) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:251) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1135) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1062) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:583) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
... 19 common frames omitted
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'kafka.producer.servers' in value "${kafka.producer.servers}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:172) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:124) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:237) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:211) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.context.support.PropertySourcesPlaceholderConfigurer.lambda$processProperties$0(PropertySourcesPlaceholderConfigurer.java:175) ~[spring-context-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:839) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1083) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1062) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:583) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:91) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:372) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
... 39 common frames omitted
查看报错,发现找不到kafka主机配置,这里我发现启动环境是dev,dev确实没 配置,切换为local试试:
报错如下:
2019-06-03 22:58:35.353 INFO 2216 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka version : 1.0.2
2019-06-03 22:58:35.353 INFO 2216 --- [ main] o.a.kafka.common.utils.AppInfoParser : Kafka commitId : 2a121f7b1d402825
2019-06-03 22:58:35.389 INFO 2216 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService
2019-06-03 22:58:38.458 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2019-06-03 22:58:38.458 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:host.name=192.168.56.1
2019-06-03 22:58:38.458 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.version=1.8.0_131
2019-06-03 22:58:38.459 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.vendor=Oracle Corporation
2019-06-03 22:58:38.459 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.home=F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre
2019-06-03 22:58:38.459 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.class.path=F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\charsets.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\deploy.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\access-bridge-32.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\cldrdata.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\dnsns.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\jaccess.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\jfxrt.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\localedata.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\nashorn.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\sunec.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\sunjce_provider.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\sunmscapi.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\sunpkcs11.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\ext\zipfs.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\javaws.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\jce.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\jfr.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\jfxswt.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\jsse.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\management-agent.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\plugin.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\resources.jar;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\lib\rt.jar;D:\Workspaces\dubbospringboot\dubbo-provider\target\classes;D:\Workspaces\dubbospringboot\dubbo-dao\target\classes;D:\Workspaces\dubbospringboot\dubbo-api\target\classes;D:\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar;D:\Workspaces\dubbospringboot\dubbo-common\target\classes;D:\repository\org\apache\commons\commons-lang3\3.7\commons-lang3-3.7.jar;D:\repository\com\alibaba\fastjson\1.2.49\fastjson-1.2.49.jar;D:\repository\com\belerweb\pinyin4j\2.5.0\pinyin4j-2.5.0.jar;D:\repository\org\springframework\boot\spring-boot-starter-web\2.0.4.RELEASE\spring-boot-starter-web-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-starter\2.0.4.RELEASE\spring-boot-starter-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot\2.0.4.RELEASE\spring-boot-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-autoconfigure\2.0.4.RELEASE\spring-boot-autoconfigure-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-starter-logging\2.0.4.RELEASE\spring-boot-starter-logging-2.0.4.RELEASE.jar;D:\repository\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;D:\repository\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\repository\org\apache\logging\log4j\log4j-to-slf4j\2.10.0\log4j-to-slf4j-2.10.0.jar;D:\repository\org\apache\logging\log4j\log4j-api\2.10.0\log4j-api-2.10.0.jar;D:\repository\org\slf4j\jul-to-slf4j\1.7.25\jul-to-slf4j-1.7.25.jar;D:\repository\javax\annotation\javax.annotation-api\1.3.2\javax.annotation-api-1.3.2.jar;D:\repository\org\yaml\snakeyaml\1.19\snakeyaml-1.19.jar;D:\repository\org\springframework\boot\spring-boot-starter-json\2.0.4.RELEASE\spring-boot-starter-json-2.0.4.RELEASE.jar;D:\repository\com\fasterxml\jackson\core\jackson-databind\2.9.6\jackson-databind-2.9.6.jar;D:\repository\com\fasterxml\jackson\core\jackson-annotations\2.9.0\jackson-annotations-2.9.0.jar;D:\repository\com\fasterxml\jackson\core\jackson-core\2.9.6\jackson-core-2.9.6.jar;D:\repository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.9.6\jackson-datatype-jdk8-2.9.6.jar;D:\repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.9.6\jackson-datatype-jsr310-2.9.6.jar;D:\repository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.9.6\jackson-module-parameter-names-2.9.6.jar;D:\repository\org\springframework\boot\spring-boot-starter-tomcat\2.0.4.RELEASE\spring-boot-starter-tomcat-2.0.4.RELEASE.jar;D:\repository\org\apache\tomcat\embed\tomcat-embed-core\8.5.32\tomcat-embed-core-8.5.32.jar;D:\repository\org\apache\tomcat\embed\tomcat-embed-el\8.5.32\tomcat-embed-el-8.5.32.jar;D:\repository\org\apache\tomcat\embed\tomcat-embed-websocket\8.5.32\tomcat-embed-websocket-8.5.32.jar;D:\repository\org\hibernate\validator\hibernate-validator\6.0.11.Final\hibernate-validator-6.0.11.Final.jar;D:\repository\javax\validation\validation-api\2.0.1.Final\validation-api-2.0.1.Final.jar;D:\repository\org\jboss\logging\jboss-logging\3.3.2.Final\jboss-logging-3.3.2.Final.jar;D:\repository\com\fasterxml\classmate\1.3.4\classmate-1.3.4.jar;D:\repository\org\springframework\spring-web\5.0.8.RELEASE\spring-web-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-beans\5.0.8.RELEASE\spring-beans-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-webmvc\5.0.8.RELEASE\spring-webmvc-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-aop\5.0.8.RELEASE\spring-aop-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-expression\5.0.8.RELEASE\spring-expression-5.0.8.RELEASE.jar;D:\repository\junit\junit\4.12\junit-4.12.jar;D:\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;D:\repository\org\springframework\spring-core\5.0.8.RELEASE\spring-core-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-jcl\5.0.8.RELEASE\spring-jcl-5.0.8.RELEASE.jar;D:\repository\mysql\mysql-connector-java\5.1.46\mysql-connector-java-5.1.46.jar;D:\repository\org\springframework\boot\spring-boot-starter-data-jpa\2.0.4.RELEASE\spring-boot-starter-data-jpa-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-starter-aop\2.0.4.RELEASE\spring-boot-starter-aop-2.0.4.RELEASE.jar;D:\repository\org\aspectj\aspectjweaver\1.8.13\aspectjweaver-1.8.13.jar;D:\repository\org\springframework\boot\spring-boot-starter-jdbc\2.0.4.RELEASE\spring-boot-starter-jdbc-2.0.4.RELEASE.jar;D:\repository\com\zaxxer\HikariCP\2.7.9\HikariCP-2.7.9.jar;D:\repository\org\springframework\spring-jdbc\5.0.8.RELEASE\spring-jdbc-5.0.8.RELEASE.jar;D:\repository\org\hibernate\hibernate-core\5.2.17.Final\hibernate-core-5.2.17.Final.jar;D:\repository\org\hibernate\javax\persistence\hibernate-jpa-2.1-api\1.0.2.Final\hibernate-jpa-2.1-api-1.0.2.Final.jar;D:\repository\org\javassist\javassist\3.22.0-GA\javassist-3.22.0-GA.jar;D:\repository\antlr\antlr\2.7.7\antlr-2.7.7.jar;D:\repository\org\jboss\jandex\2.0.3.Final\jandex-2.0.3.Final.jar;D:\repository\dom4j\dom4j\1.6.1\dom4j-1.6.1.jar;D:\repository\org\hibernate\common\hibernate-commons-annotations\5.0.1.Final\hibernate-commons-annotations-5.0.1.Final.jar;D:\repository\javax\transaction\javax.transaction-api\1.2\javax.transaction-api-1.2.jar;D:\repository\org\springframework\data\spring-data-jpa\2.0.9.RELEASE\spring-data-jpa-2.0.9.RELEASE.jar;D:\repository\org\springframework\data\spring-data-commons\2.0.9.RELEASE\spring-data-commons-2.0.9.RELEASE.jar;D:\repository\org\springframework\spring-orm\5.0.8.RELEASE\spring-orm-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-aspects\5.0.8.RELEASE\spring-aspects-5.0.8.RELEASE.jar;D:\repository\com\alibaba\dubbo\2.5.3\dubbo-2.5.3.jar;D:\repository\org\jboss\netty\netty\3.2.5.Final\netty-3.2.5.Final.jar;D:\repository\org\apache\zookeeper\zookeeper\3.4.6\zookeeper-3.4.6.jar;D:\repository\org\slf4j\slf4j-api\1.7.25\slf4j-api-1.7.25.jar;D:\repository\jline\jline\0.9.94\jline-0.9.94.jar;D:\repository\io\netty\netty\3.7.0.Final\netty-3.7.0.Final.jar;D:\repository\com\github\sgroschupf\zkclient\0.1\zkclient-0.1.jar;D:\repository\net\sf\json-lib\json-lib\2.4\json-lib-2.4-jdk15.jar;D:\repository\commons-beanutils\commons-beanutils\1.8.0\commons-beanutils-1.8.0.jar;D:\repository\commons-collections\commons-collections\3.2.1\commons-collections-3.2.1.jar;D:\repository\commons-lang\commons-lang\2.5\commons-lang-2.5.jar;D:\repository\commons-logging\commons-logging\1.1.1\commons-logging-1.1.1.jar;D:\repository\net\sf\ezmorph\ezmorph\1.0.6\ezmorph-1.0.6.jar;D:\repository\org\springframework\kafka\spring-kafka\2.1.7.RELEASE\spring-kafka-2.1.7.RELEASE.jar;D:\repository\org\springframework\spring-context\5.0.8.RELEASE\spring-context-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-messaging\5.0.8.RELEASE\spring-messaging-5.0.8.RELEASE.jar;D:\repository\org\springframework\spring-tx\5.0.8.RELEASE\spring-tx-5.0.8.RELEASE.jar;D:\repository\org\springframework\retry\spring-retry\1.2.2.RELEASE\spring-retry-1.2.2.RELEASE.jar;D:\repository\org\apache\kafka\kafka-clients\1.0.2\kafka-clients-1.0.2.jar;D:\repository\org\lz4\lz4-java\1.4\lz4-java-1.4.jar;D:\repository\org\xerial\snappy\snappy-java\1.1.4\snappy-java-1.1.4.jar;H:\Program Files\JetBrains\IntelliJ IDEA 2017.2.5\lib\idea_rt.jar
2019-06-03 22:58:38.463 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.library.path=F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;d:\ProgramData\Anaconda3;d:\ProgramData\Anaconda3\Library\mingw-w64\bin;d:\ProgramData\Anaconda3\Library\usr\bin;d:\ProgramData\Anaconda3\Library\bin;d:\ProgramData\Anaconda3\Scripts;F:\app\TANGZHANBO\product\11.2.0\dbhome_1\bin;C:\Program Files (x86)\Common Files\NetSarang;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\bin;F:\MyDevelopKit\1.software\jdk\1.8\jdk1.8.0_131\jre\bin;H:\Program Files\TortoiseSVN\bin;E:\spark\scala-2.11.0\bin;E:\spark\安装软件\hadoop-2.7.5\bin;E:\spark\安装软件\sbt-1.1.1\sbt\bin;E:\spark\安装软件\spark-2.1.0-bin-hadoop2\spark-2.1.0-bin-hadoop2.7\bin;%Sy;temRoot%\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;E:\maven\apache-maven-3.2.2-bin\apache-maven-3.2.2\bin\;h:\oracle;h:\Program Files\Git\cmd;C:\WINDOWS\system32;C:\WINDOWS\System32\OpenSSH\;E:\spark\windows-environment\spark-2.1.0-bin-hadoop2\spark-2.1.0-bin-hadoop2.6\bin;C:\Program Files (x86)\MySQL\MySQL Server 5.7\bin;D:\Program Files\nodejs\;%NODEJS_PATH%\;%NODEJS_PATH%\node_global\;D:\Program Files\nodejs\node_cache;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Users\TANGZHANBO\AppData\Local\Microsoft\WindowsApps;C:\Users\TANGZHANBO\AppData\Local\GitHubDesktop\bin;C:\Users\TANGZHANBO\AppData\Local\Microsoft\WindowsApps;;D:\Microsoft VS Code\bin;C:\Users\TANGZHANBO\AppData\Roaming\npm;.
2019-06-03 22:58:38.463 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.io.tmpdir=D:\c盘迁移文件\Temp\
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:java.compiler=<NA>
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:os.name=Windows 10
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:os.arch=x86
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:os.version=10.0
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:user.name=TANGZHANBO
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:user.home=C:\Users\TANGZHANBO
2019-06-03 22:58:38.464 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Client environment:user.dir=D:\Workspaces\dubbospringboot
2019-06-03 22:58:38.468 INFO 2216 --- [ main] org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=192.168.65.113:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@1cb927e
2019-06-03 22:58:38.514 INFO 2216 --- [68.65.113:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server 192.168.65.113/192.168.65.113:2181. Will not attempt to authenticate using SASL (unknown error)
2019-06-03 22:58:56.825 WARN 2216 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 22:58:59.534 WARN 2216 --- [68.65.113:2181)] org.apache.zookeeper.ClientCnxn : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_131]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[na:1.8.0_131]
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) ~[zookeeper-3.4.6.jar:3.4.6-1569965]
2019-06-03 22:59:00.638 INFO 2216 --- [68.65.113:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server 192.168.65.113/192.168.65.113:2181. Will not attempt to authenticate using SASL (unknown error)
把zk地址换成本机windows的:
2019-06-03 23:03:33.198 INFO 16028 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 7722 (http) with context path ''
2019-06-03 23:03:33.208 INFO 16028 --- [ main] com.tzb.myspringboot.ServiceApplication : Started ServiceApplication in 18.626 seconds (JVM running for 24.025)
=====springboot项目启动成功=======
2019-06-03 23:03:34.299 WARN 16028 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:03:36.171 WARN 16028 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:03:38.191 WARN 16028 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
看样子是linux上的zk不好用了。
重启了也不好使,所以 我 重新启动 一个zk,需要换个容器名字,之前是zookeeper,删掉重建(我试了改名没成功,反反复复报错,千万注意 第一点 :zk是否跟宿主机映射上了 第2点:2181,2888,3888三个端口在防火墙开放了吗)
[root@Master config]# docker run -d -p 2181:2181 -v /opt/docker/zookeeper/data/:/data/ --name=zookeeper --privileged zookeeper
[root@Master config]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ccd9c11429bd wurstmeister/kafka:2.12-2.1.0 "start-kafka.sh" 39 minutes ago Up 39 minutes 0.0.0.0:9092->9092/tcp kafka
3108b2bbef86 redis:3.2 "docker-entrypoint.s…" 9 days ago Up 9 days 0.0.0.0:6379->6379/tcp hardcore_banach
590a923a5fcd zookeeper "/docker-entrypoint.…" 9 days ago Up 4 minutes 2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp zookeeper
aac0f7b27727 mysql:5.7.25 "docker-entrypoint.s…" 2 weeks ago Up 9 days 0.0.0.0:3306->3306/tcp, 33060/tcp mysql_springboot
5dd1c0bc2d60 amancevice/superset "gunicorn superset:a…" 2 weeks ago Up 9 days (healthy) 0.0.0.0:8088->8088/tcp superset
[root@Master config]#
[root@Master config]#
[root@Master config]#
[root@Master config]#
[root@Master config]#
[root@Master config]# docker stop 590a923a5fcd
590a923a5fcd
[root@Master config]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ccd9c11429bd wurstmeister/kafka:2.12-2.1.0 "start-kafka.sh" 41 minutes ago Up 41 minutes 0.0.0.0:9092->9092/tcp kafka
3108b2bbef86 redis:3.2 "docker-entrypoint.s…" 9 days ago Up 9 days 0.0.0.0:6379->6379/tcp hardcore_banach
590a923a5fcd zookeeper "/docker-entrypoint.…" 9 days ago Exited (143) 4 seconds ago zookeeper
aac0f7b27727 mysql:5.7.25 "docker-entrypoint.s…" 2 weeks ago Up 9 days 0.0.0.0:3306->3306/tcp, 33060/tcp mysql_springboot
5dd1c0bc2d60 amancevice/superset "gunicorn superset:a…" 2 weeks ago Up 9 days (healthy) 0.0.0.0:8088->8088/tcp superset
[root@Master config]#
[root@Master config]#
[root@Master config]# docker run -d -p 2181:2181 -v /opt/docker/zookeeper/data/:/data/ --name=zookeeper --privileged zookeeper
docker: Error response from daemon: Conflict. The container name "/zookeeper" is already in use by container "590a923a5fcdca72d3a33e77aa4ba0d5a9fdd974a9aff9f192a8c85902212831". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
[root@Master config]# docker run -d -p 2181:2181 -v /opt/docker/zookeeper/data/:/data/ --name=zookeeper --privileged my_zookeeper
Unable to find image 'my_zookeeper:latest' locally
docker: Error response from daemon: pull access denied for my_zookeeper, repository does not exist or may require 'docker login'.
See 'docker run --help'.
[root@Master config]# docker run -d -p 2181:2181 -v /opt/docker/zookeeper/data/:/data/ --name=my_zookeeper --privileged zookeeper
WARNING: IPv4 forwarding is disabled. Networking will not work.
1acedeb50e94711c1c834f75c226cea5d7dc8b5cc0776b68dec3da6be92e8265
[root@Master config]#
[root@Master config]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1acedeb50e94 zookeeper "/docker-entrypoint.…" 7 seconds ago Up 5 seconds 2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp my_zookeeper
ccd9c11429bd wurstmeister/kafka:2.12-2.1.0 "start-kafka.sh" 42 minutes ago Up 42 minutes 0.0.0.0:9092->9092/tcp kafka
3108b2bbef86 redis:3.2 "docker-entrypoint.s…" 9 days ago Up 9 days 0.0.0.0:6379->6379/tcp hardcore_banach
590a923a5fcd zookeeper "/docker-entrypoint.…" 9 days ago Exited (143) About a minute ago zookeeper
aac0f7b27727 mysql:5.7.25 "docker-entrypoint.s…" 2 weeks ago Up 9 days 0.0.0.0:3306->3306/tcp, 33060/tcp mysql_springboot
5dd1c0bc2d60 amancevice/superset "gunicorn superset:a…" 2 weeks ago Up 9 days (healthy) 0.0.0.0:8088->8088/tcp superset
[root@Master config]#
这样 我重新启动springboot项目,就成功连上了zk。
但是kafka还有 问题:
=====springboot项目启动成功=======
2019-06-03 23:20:19.475 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:20:40.534 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:21:01.638 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:21:22.845 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:21:44.256 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:22:06.211 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:22:28.331 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:22:50.251 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:23:12.168 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:23:34.023 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Connection to node -1 could not be established. Broker may not be available.
2019-06-03 23:23:56.042 WARN 18696 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=
进入kafka目录:
docker exec -it ccd9c11429bd /bin/bash
进入kafka默认目录 /opt/kafka
进入config目录,vi server.properies
############################# Zookeeper #############################
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=192.168.65.113:2181
# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000
尝试 创建主题报错了:
ash-4.4# cd /opt/kafka_2.12-2.1.0/
bash-4.4# ls
LICENSE NOTICE bin config libs logs site-docs
bash-4.4# bin/kafka-topics.sh --zookeeper 192.168.65.113:2181 --topic TestTopic --replication-factor 1 --partitions 1 --create
Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
[2019-06-03 15:40:52,659] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.
(kafka.admin.TopicCommand$)
bash-4.4#
看错误信息“1 larger than available brokers: 0”,意思是至少有一个brokers.也就是说并没有有效的brokers可以用。
后来发现创建kafka容器命令,宿主机 ip写成windows了,删了重建就可以了。
再次启动后
进入zk查看是否已注册:
docker exec -it 1acedeb50e94 /bin/bash
[root@Master ~]# docker exec -it 28f308f20f48 /bin/bash
bash-4.4# ls /
bin datalog etc logs opt run sys var
conf dev home media proc sbin tmp zookeeper-3.4.14
data docker-entrypoint.sh lib mnt root srv usr
bash-4.4# cd bin/
bash-4.4# ls
README.txt zkCli.cmd zkEnv.cmd zkServer.cmd zkTxnLogToolkit.cmd
zkCleanup.sh zkCli.sh zkEnv.sh zkServer.sh zkTxnLogToolkit.sh
bash-4.4# ./zkCli.sh
Connecting to localhost:2181
2019-06-03 17:43:49,470 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT
2019-06-03 17:43:49,483 [myid:] - INFO [main:Environment@100] - Client environment:host.name=28f308f20f48
2019-06-03 17:43:49,484 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_212
2019-06-03 17:43:49,494 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=IcedTea
2019-06-03 17:43:49,496 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8-openjdk/jre
2019-06-03 17:43:49,499 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/zookeeper-3.4.14/bin/../build/classes:/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/zookeeper-3.4.14/bin/../build/lib/*.jar:/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/conf:
2019-06-03 17:43:49,501 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-06-03 17:43:49,502 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-06-03 17:43:49,504 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-06-03 17:43:49,506 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-06-03 17:43:49,508 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-06-03 17:43:49,511 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-693.el7.x86_64
2019-06-03 17:43:49,513 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-06-03 17:43:49,514 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-06-03 17:43:49,515 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/zookeeper-3.4.14/bin
2019-06-03 17:43:49,518 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89
Welcome to ZooKeeper!
2019-06-03 17:43:49,623 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-06-03 17:43:50,007 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2019-06-03 17:43:50,056 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1000017074e0003, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[cluster, controller_epoch, controller, brokers, zookeeper, dubbo, admin, isr_change_notification, consumers, log_dir_event_notification, latest_producer_id_block, config]
[zk: localhost:2181(CONNECTED) 1] ls /brokers
[ids, topics, seqid]
[zk: localhost:2181(CONNECTED) 2] ls /brokers/ids
[0]
[zk: localhost:2181(CONNECTED) 3] ls /brokers/topics
[TestTopic, __consumer_offsets]
[zk: localhost:2181(CONNECTED) 4] ls /dubbo
[com.tzb.myspringboot.inter.kafka.IKafkaService, com.tzb.myspringboot.inter.test.ITestSecondDataSourceService, com.tzb.myspringboot.inter.pageview.ICalPageViewService, com.tzb.myspringboot.inter.customer.ICalCustomerServerService, com.tzb.myspringboot.inter.test.ITestPrimaryDataSourceService]
[zk: localhost:2181(CONNECTED) 5]
已注册上dubbo和kafka。
[root@Master ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9692a7a3d03f zookeeper:latest "/docker-entrypoint.…" About an hour ago Up 5 minutes 2181/tcp, 2888/tcp, 3888/tcp my_zookeeper1
11e7b292864c wurstmeister/kafka:2.12-2.1.0 "start-kafka.sh" About an hour ago Exited (137) About an hour ago kafka
aac0f7b27727 mysql:5.7.25 "docker-entrypoint.s…" 2 weeks ago Up 5 seconds 0.0.0.0:3306->3306/tcp, 33060/tcp mysql_springboot
5dd1c0bc2d60 amancevice/superset "gunicorn superset:a…" 2 weeks ago Exited (0) 32 minutes ago superset
[root@Master ~]#
日志 也能看出一开始连接不到kafka(我未启动),启动后就连接上了(springboot项目)
2019-06-04 01:43:11.270 WARN 16888 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Error while fetching metadata with correlation id 13 : {TestTopic=LEADER_NOT_AVAILABLE}
2019-06-04 01:43:11.418 WARN 16888 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Error while fetching metadata with correlation id 15 : {TestTopic=LEADER_NOT_AVAILABLE}
2019-06-04 01:43:11.692 WARN 16888 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=bi] Error while fetching metadata with correlation id 17 : {TestTopic=LEADER_NOT_AVAILABLE}
2019-06-04 01:43:14.660 INFO 16888 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-1, groupId=bi] Discovered group coordinator 192.168.65.113:9092 (id: 2147483647 rack: null)
2019-06-04 01:43:14.665 INFO 16888 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-1, groupId=bi] Revoking previously assigned partitions []
2019-06-04 01:43:14.665 INFO 16888 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer : partitions revoked: []
2019-06-04 01:43:14.666 INFO 16888 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-1, groupId=bi] (Re-)joining group
2019-06-04 01:43:15.482 INFO 16888 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-1, groupId=bi] Successfully joined group with generation 1
2019-06-04 01:43:15.485 INFO 16888 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-1, groupId=bi] Setting newly assigned partitions [TestTopic-0]
2019-06-04 01:43:15.603 INFO 16888 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer : partitions assigned: [TestTopic-0]
使用Kafka Tool查看springboot接口发送的kafka消息成功:
参考: