logstash 切分日志

 日志格式可以大致分为两部分,基本的头信息和json格式的数据,中间以‘|’进行了分割。

2018-08-30 10:41:42,661 ERROR [http-apr-8080-exec-7] [com.intime.soa.framework.auth.AbstractAuthInterceptor 118] - Request <> GET_/favicon.ico
|
{
  "elapsed": 274,
  "headers": {
    "cookie": "JSESSIONID=B78BE242138863E0DE1F4DA8545FBAE5",
    "connection": "keep-alive",
    "accept-language": "zh-CN,zh;q=0.9",
    "host": "localhost:8080",
    "accept": "image/webp,image/apng,image/*,*/*;q=0.8",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36",
    "accept-encoding": "gzip, deflate, br",
    "referer": "http://localhost:8080/"
  },
  "status": 404,
  "method": "GET_/favicon.ico",
  "ip": "127.0.0.1"
}

 split插件将message字段的日志,按照有‘|’进行了切割,切割成了message1和message2两个字段 然后对message2字段进行了json格式化。

filter{
    mutate{
        remove_field => ["host"]
        remove_field => ["agent"]
        remove_field => ["ecs"]
        remove_field => ["tags"]
        remove_field => ["fields"]
        remove_field => ["@version"]
        remove_field => ["input"]
        remove_field => ["log"]

    split=>["message","|"]
                add_field => {
                        "message1" => "%{[message][0]}"
                }
                add_field => {
                        "message2" => "%{[message][1]}"
                }
                remove_field => ["message"]
   }
 
   json{
           source => "message2"
           target => "message2"
        }
}

提示:老版本说split插件必须卸载最后面,split后面的插件是不起作用的,新版本解决了这个问题。

发布了240 篇原创文章 · 获赞 435 · 访问量 9万+

猜你喜欢

转载自blog.csdn.net/fjxcsdn/article/details/102961416