问题描述
ElasticSearch官方默认限制索引查询最多只能查询10000条数据,查询第10001条数据开始就会报错:
reason=Result window is too large, from + size must be less than or equal to: [10000] but was [29740]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.]];
原因分析
elasticsearch中
max_result_window
有上限限制:默认10000。
解决方案
第一种解决方案
1.在请求时解除限制
设置查询最大上限20000
PUT _all/_settings
{
"index.max_result_window": 20000
}
2.代码中解除限制并且设置最大返回值
在查询时候把 track_total_hits 设置为 true。
track_total_hits 设置为false禁用跟踪匹配查询的总点击次数
设置为true就会返回真实的命中条数。
SearchSourceBuilder query = new SearchSourceBuilder();
query.trackTotalHits(true);
query.trackTotalHitsUpTo(20000);
query.query(integratedQueryBuilder(searchConditionJson));
3.在使用工具访问时添加track_total_hits设置
GET 索引名/_search
{
"query": {
"match_all": {}
},
"track_total_hits": true
}
第二种解决方案
在创建索引时添加最大上限设置
{
"settings": {
"index": {
"max_result_window": 20000
}
}
}
使用这种方式代码撰写时也要和第一种一样在代码中限制保持一致 。
Java动态处理动态限制:
AI的回答,
官网资料:链接
引入依赖:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
通过RestHighLevelClient,实现动态修改setting的值:
/**
* 修改索引的settings
* @param indices
* @param from
* @param size
*/
private boolean updateIndex(String indices, int from, int size) {
int records = from * size + size;
if (records <= 10000) {
return true;
}
UpdateSettingsRequest request = new UpdateSettingsRequest(indices);
request.settings(Settings.builder().put("index.max_result_window", records));
try {
AcknowledgedResponse updateSettingsResponse = restHighLevelClient.indices().putSettings(request, RequestOptions.DEFAULT);
return updateSettingsResponse.isAcknowledged();
} catch (IOException e) {
log.error("error ",e);
throw new RuntimeException("索引修改setting异常",e);
}
}