来看个小例子:固定周期采集数据,并设置数据预警机制
采集数据配置,拉取RSS数据并存储到ElasticSearch中.
1.下载并安装最新的logstash1.5版本
地址:https://www.elastic.co/products/logstash
2.安装rss插件
cd logstash-1.5.0 bin/plugin install logstash-input-rss
3.配置数据采集脚本
input { rss { url => "http://stackoverflow.com/feeds/tag/elasticsearch+or+logstash+or+kibana" interval => 3600 } } output { elasticsearch { protocol => "http" host => "localhost" } stdout { } }
4.启动logstash
nohup bin/logstash -f rss.conf > logs/rss.log&
5.验证ElasticSearch数据流是否正常
可以通过API,或者Kopf,Head等插件进行验证。
Watcher预警配置
扫描二维码关注公众号,回复:
535738 查看本文章
1.首先定义Script文件,采用Groovy语法,将文件放置到$ES_HOME/config/Scripts目录下。
$ES_HOME/config/scripts/threshold_hits.groovy 编辑内容为: return ctx.payload.hits.total > threshold
2.设置Watcher:
PUT _watcher/watch/rss_watch { "trigger" : { "schedule" : { "daily" : { "at" : "12:00" } #每天12点执行 } }, "input" : { "search" : { "request" : { "indices" : [ "logstash*" ],#指定查询索引 "body" : { "query" : { "filtered" : { "query" : {"match" : {"message": "error problem"}}, #查询条件 "filter" : {"range" : {"@timestamp" : {"gte" : "now-1d"}}} } } } } } }, "condition" : { "script" : { "file" : "threshold_hits",#脚本文件名称 "params" : { "threshold" : 0 #参数 } } }, "actions" : { "send_email" : { #请注意ElasticSearch.yml中的邮件服务器配置信息,可参考之前的文章介绍 "email" : { "to" : "[email protected]", "subject" : "Somebody needs help with ELK", "body" : "The attached Stack Overflow posts were tagged with Elasticsearch, Logstash, or Kibana and mentioned an error or problem.", "attach_data" : true } } } }
3.调试Watcher配,通过以下API可以,立刻执行配置的Watcher
POST _watcher/watch/rss_watch/_execute { "trigger_event" : { "schedule" : { "triggered_time": "now", "scheduled_time": "now" } }, "ignore_throttle" : true, "simulated_actions" : "_all", "record_execution" : true }
4.验证是否可以收到预警邮件。