日志格式可以大致分为两部分,基本的头信息和json格式的数据,中间以‘|’进行了分割。
2018-08-30 10:41:42,661 ERROR [http-apr-8080-exec-7] [com.intime.soa.framework.auth.AbstractAuthInterceptor 118] - Request <> GET_/favicon.ico
|
{
"elapsed": 274,
"headers": {
"cookie": "JSESSIONID=B78BE242138863E0DE1F4DA8545FBAE5",
"connection": "keep-alive",
"accept-language": "zh-CN,zh;q=0.9",
"host": "localhost:8080",
"accept": "image/webp,image/apng,image/*,*/*;q=0.8",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36",
"accept-encoding": "gzip, deflate, br",
"referer": "http://localhost:8080/"
},
"status": 404,
"method": "GET_/favicon.ico",
"ip": "127.0.0.1"
}
split插件将message字段的日志,按照有‘|’进行了切割,切割成了message1和message2两个字段 然后对message2字段进行了json格式化。
filter{
mutate{
remove_field => ["host"]
remove_field => ["agent"]
remove_field => ["ecs"]
remove_field => ["tags"]
remove_field => ["fields"]
remove_field => ["@version"]
remove_field => ["input"]
remove_field => ["log"]
split=>["message","|"]
add_field => {
"message1" => "%{[message][0]}"
}
add_field => {
"message2" => "%{[message][1]}"
}
remove_field => ["message"]
}
json{
source => "message2"
target => "message2"
}
}
提示:老版本说split插件必须卸载最后面,split后面的插件是不起作用的,新版本解决了这个问题。