1.概念介绍:
- ELK: 业界使用最为广泛的开源日志管理平台,由Elasticsearch,logstash和kibana三部分组件组成
- Elasticsearch:开源分布式搜索引擎,用来存储各类日志
- Logstash:用来对日志进行收集、分析,并且将收集结果发给Elasticsearch。
- Kibana:提供web界面,接口用作查询和可视化日志
- filebeat:轻量级日志采集,Elasticsearch 或 Logstash,在 Kibana 中实现可视化
2.运行机制:
- 系统可以对容器的运行日志通过 ELK 进行集中式的收集存储。Logstash收集容器产生的日志,并存放到ElasticSearch中,并通过 kibana 进行日志信息的统一展示、管理
3.docker-compose 部署配置:
#elasticSearch.yml
version: "3"
services:
elasticSearchMaster:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
container_name: es01
restart: always
privileged: true
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
ports:
- "9200:9200"
volumes:
- ../data01:/usr/share/elasticsearch/data
elasticSearchSlave1:
image: elasticsearch:2.3.0
container_name: es02
restart: always
privileged: true
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /etc/localtime:/etc/localtime
- ../data02:/usr/share/elasticsearch/data
elasticSearchSlave2:
image: elasticsearch:2.3.0
container_name: es03
restart: always
privileged: true
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /etc/localtime:/etc/localtime
- ../data03:/usr/share/elasticsearch/data
扫描二维码关注公众号,回复:
9954715 查看本文章
#logstash.yml
version: "3" #版本号
services:
logstash:
image: logstash #使用的镜像
restart: "always" #重启策略,能够使服务保持始终运行,生产环境推荐使用
container_name: logstash #容器名称
volumes:
- /usr/games/logstash.conf:/etc/logstash/conf.d/logstash.conf
- /usr/games/logstash.yml:/etc/logstash/logstash.yml
- /usr/games/1.txt:/etc/logstash/1.txt
links:
- elasticsearch-1:es01 #容器关联es01是别名
#kibana.yml配置文件如下
elasticsearch.url: "http://es01:9200"
server.host: "0.0.0.0"
#logstash.yml配置如下
path.config: /etc/logstash/conf.d
#logstash.conf配置如下
input {
file {
path => "/etc/logstash/1.txt" #读取的配置路径,是容器的路径,需要挂载
type => "docker-logstash"
start_position => "beginning" #从文件开始处读写
}
}output{
elasticsearch{
hosts=>["es01:9200"]
index => "logstash-%{+YYYY.MM.dd}"
}
stdout{codec => rubydebug}
}
4.k8s部署配置:
待续
5.使用示例:
待续