1. 搜索模板
/_search/template
端点允许使用mustache language(继承ES沙箱脚本语言的一种语言,是护胡言乱语吗……)为每个呈现搜索请求,在执行前和用模板参数填充现有的模板:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"source" : {
"query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
"size" : "{{my_size}}"
},
"params" : {
"my_field" : "message",
"my_value" : "some message",
"my_size" : 5
}
}
'
执行这个请求时,ES卡住没有响应,一直处于loading的状态。
模板样例
用单值填充查询字符串
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"source": {
"query": {
"term": {
// 使用占位符的方式{{xxx}}使用一个变量
"user": "{{query_string}}"
}
}
},
"params": {
// 定义上述使用到的变量
"query_string": "kimchy3"
}
}
'
转换参数成JSON
{{#toJson}}parameter{{/toJson}}
函数可以用来将map或数组的参数parameter
转换成对应的JSON(下面是查询用户名为kimchy4
和kimchy3
两个用户):
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
// 将变量users转成对应的JSON
"source": "{ \"query\": { \"terms\": {{#toJson}}users{{/toJson}} }}",
"params": {
// 将users变量填充到上面的函数中
"users" : {
"user": [ "kimchy4", "kimchy3" ]
}
}
}
'
最终的转换结果(但是实际是看不到的)为:
// 转成JSON串
"query": {
"terms": {
"users": {
"user": [
"kimchy4",
"kimchy3"
]
}
}
}
连接数组的值
{{#join}}array{{/join}}
函数可以用来将数组array
内的元素连接为一个用逗号,
分隔的字符串(twitter
索引中有kimchy1
…kimchy4
、kimchy,5
五个文档),下面是一个栗子:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"source": {
"query": {
"match": {
"user": "{{#join}}user{{/join}}"
}
}
},
"params": {
"user": [ "kimchy", "5" ]
}
}
'
这样最终传入的user
变量为kimchy,5
(由数组[ "kimchy", "5" ]
转成kimchy,5
),即直接去匹配了。上述是使用默认的分隔符(即逗号,
)来连接数组各个元素,当前这个分隔符也可以自定义,使用自定义分隔符函数{{#join delimiter='xx'}}date.formats{{/join delimiter='xx'}}
即可将默认的,
改为||
,比如:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"source": {
"query": {
"range": {
"born": {
"gte" : "{{date.min}}",
"lte" : "{{date.max}}",
// 将原来逗号的分隔符改为双竖杠||
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
"params": {
"date": {
"min": "2016",
"max": "31/12/2017",
"formats": ["dd/MM/yyyy", "yyyy"]
}
}
}
'
born
最终的转换为:
"born" : {
"gte" : "2016",
"lte" : "31/12/2017",
"format" : "dd/MM/yyyy||yyyy"
}
默认值
默认值的写法为:{{var}}{{^var}}default{{/var}}
,下面是小栗子:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"source": {
"query": {
// 查询范围内获赞数的文档
"range": {
// 指定按获赞数进行筛选
"likes": {
// 获赞数的下限
"gte": "{{start}}",
// 获赞数的上限
"lte": "{{end}}{{^end}}20{{/end}}"
}
}
}
},
"params": {
...
}
}
'
如果params
中定义start
和end
两个变量,比如:
"params": {
"start": 12,
"end": 13
}
那最终的查询参数最终为:
"line_no": {
"gte": "12",
"lte": "13"
}
但如果params
中只定义了start
变量,没有定义end
变量,如:
"params": {
"start": 12
}
那默认变量{{end}}{{^end}}20{{/end}}
生效,此时查询参数为:
"line_no": {
"gte": "12",
"lte": "20"
}
条件语句(待处理)
条件语句不能使用JSON形式的模板,而是必须用字符串的方式,下面是一个查询行号的小栗子:
{
"query": {
"bool": {
"must": {
"match": {
// 将text参数填充
"line": "{{text}}"
}
},
// 这个过滤器只有params中指定了line_no才会创建,否则不会创建
"filter": {
{{#line_no}} // 这里使用{{#xx}}..{{/xx}}包裹
"range": {
"line_no": {
{{#start}}
"gte": "{{start}}"
{{#end}},{{/end}}
{{/start}}
{{#end}}
"lte": "{{end}}"
{{/end}}
}
}
{{/line_no}}
}
}
},
"params": {
"text": "words to search for",
// 下面这些参数都是可选的
"line_no": { // 这个参数可省
"start": 10, // 可省
"end": 20 // 可省
}
}
}
上述的条件语句中其实就是使用{{#A}}..B..{{/A}}
将执行语句B包裹在内部,换成java里面的写法就是if(A != null) {B}
,即只有变量A不为空就执行B,否则不执行B。特别注意的是上述并不是有效的JSON格式(含有{{#line_on}}
这些标记),所以需要将整个配置写成字符串的形式,注意双引号的转义,简单写成:
"source": "{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"
这里执行遇到如下问题(待解决):
{
"error": {
"root_cause": [
{
"type": "general_script_exception",
"reason": "Failed to compile inline script [{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}] using lang [mustache]"
}
],
"type": "general_script_exception",
"reason": "Failed to compile inline script [{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}] using lang [mustache]",
"caused_by": {
"type": "mustache_exception",
"reason": "Improperly closed variable in query-template:1"
}
},
"status": 500
}
URL编码
{{#url}}value{{/url}}
函数可以用来将HTML文本编码为字符串,比如:
curl -X GET "localhost:9200/_render/template" -H 'Content-Type: application/json' -d'
{
"source" : {
"query" : {
"term": {
"http_access_log": "{{#url}}{{host}}/{{page}}{{/url}}"
}
}
},
"params": {
"host": "https://www.elastic.co/",
"page": "learn"
}
}
'
最终上述请求的结果为:
{
"template_output": {
"query": {
"term": {
"http_access_log": "https%3A%2F%2Fwww.elastic.co%2F%2Flearn"
}
}
}
}
预注册模板
通过存储脚本API可以注册搜索模板,比如:
// 注册模板testtemplate,可以自定义
curl -X POST "localhost:9200/_scripts/testtemplate" -H 'Content-Type: application/json' -d'
{
"script": {
// 定义使用语言未mustache
"lang": "mustache",
"source": {
"query": {
"match": {
"title": "{{query_string}}"
}
}
}
}
}
'
取出上述的模板:
curl -X GET "localhost:9200/_scripts/testtemplate"
// 结果
{
"_id": "testtemplate",
"found": true,
"script": {
"lang": "mustache",
"source": "{\"query\":{\"match\":{\"title\":\"{{query_string}}\"}}}",
"options": {
"content_type": "application/json; charset=UTF-8"
}
}
}
验证模板(关于模板验证见下面的验证模板):
curl -X POST "localhost:9200/_render/template/twittertemplate" -H 'Content-Type: application/json' -d'
{
"params": {
"query_string": "kimchy2"
}
}
'
// 结果
{
"template_output": {
"query": {
"match": {
"user": "kimchy2"
}
}
}
}
使用上述预定义的模板可以如下使用:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
// 这里的id就是注册模板的名字
"id": "<templateName>",
// 未模板注入参数
"params": {
"query_string": "search for these words"
}
}
'
删除模板:
curl -X DELETE "localhost:9200/_scripts/<templatename>"
下面是一套完整的模板从注册到使用的过程:
// 1. 注册模板
curl -X POST "localhost:9200/_scripts/twittertemplate" -H 'Content-Type: application/json' -d'
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"user": "{{query_string}}"
}
}
}
}
}
'
// 2. 使用模板
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"id": "twittertemplate",
"params": {
"query_string": "kimchy2"
}
}
'
// 结果
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 15,
"successful": 15,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.6931472,
"hits": [
{
"_index": "twitter",
"_type": "_doc",
"_id": "2",
"_score": 0.6931472,
"_source": {
"user": "kimchy2",
"likes": 9,
"post_date": 1542197883,
"message": "trying out Elasticsearch"
}
}
]
}
}
验证模板
模板可以带着使用的参数出现在响应中,比如:
curl -X GET "localhost:9200/_render/template" -H 'Content-Type: application/json' -d'
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
'
// 结果
{
"template_output": {
"query": {
"terms": {
"status": [
"pending",
"published"
]
}
}
}
}
通过结果看其实就是看一下转换的结果(可以带入参数查看)。
得分说明
使用模板时也可以使用explain
参数开启得分说明:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"id": "twittertemplate",
"params": {
"query_string": "kimchy2"
},
"explain": true
}
'
// 结果
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 15,
"successful": 15,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.6931472,
"hits": [
{
"_shard": "[twitter][2]",
"_node": "bKeGC-Q-SXuyyGlcarDrMg",
"_index": "twitter",
"_type": "_doc",
"_id": "2",
"_score": 0.6931472,
"_source": {
"user": "kimchy2",
"likes": 9,
"post_date": 1542197883,
"message": "trying out Elasticsearch"
},
"_explanation": {
"value": 0.6931472,
"description": "weight(user:kimchy2 in 0) [PerFieldSimilarity], result of:",
"details": [
{
"value": 0.6931472,
"description": "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
"details": [
{
"value": 0.6931472,
"description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
"details": [
{
"value": 1,
"description": "docFreq",
"details": []
},
{
"value": 2,
"description": "docCount",
"details": []
}
]
},
{
"value": 1,
"description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1) from:",
"details": [
{
"value": 1,
"description": "termFreq=1.0",
"details": []
},
{
"value": 1.2,
"description": "parameter k1",
"details": []
},
{
"value": 0,
"description": "parameter b (norms omitted for field)",
"details": []
}
]
}
]
}
]
}
}
]
}
}
配置
使用模板时也可以使用profile
参数:
curl -X GET "localhost:9200/_search/template" -H 'Content-Type: application/json' -d'
{
"id": "twittertemplate",
"params": {
"query_string": "kimchy2"
},
"profile": true
}
'
2. 多搜索模板(Mutil Search Template)
多搜索模板API可以使用_msearch/template
端点执行多个搜索模板请求(和多搜索请求API类似),形式如下:
header\n
body\n
header\n
body\n
header\n
:头部header
和多搜索API类似支持相同的index
、search_type
、preference
和routing
选项;
body\n
:体body
包括一个搜索模板体请求并支持内联、存储和文件模板,如下:
$ cat requests
{"index": "test"}
// 内联搜索模板请求
{"source": {"query": {"match": {"user" : "{{username}}" }}}, "params": {"username": "john"}}
{"source": {"query": {"{{query_type}}": {"name": "{{name}}" }}}, "params": {"query_type": "match_phrase_prefix", "name": "Smith"}}
{"index": "_all"}
// 基于存储模板的搜索模板请求
{"id": "template_1", "params": {"query_string": "search for these words" }}
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo