ES 简介
Elasticsearch 7.x 是一个基于 Lucene 的分布式搜索引擎,它提供了一个分布式全文搜索引擎,可以快速、准确地搜索、分析和存储海量数据。Elasticsearch 7.x 的主要特点包括:
-
分布式架构:Elasticsearch 7.x 是一个分布式系统,它可以运行在多个节点上,每个节点都是相互独立的,并且数据会自动分片和复制到多个节点上,提高了系统的可用性和可扩展性。
-
实时搜索:Elasticsearch 7.x 可以快速地索引和搜索海量数据,支持实时搜索,即当数据发生变化时,可以立即搜索到最新的结果。
-
多数据类型支持:Elasticsearch 7.x 支持多种数据类型的索引和搜索,包括文本、数字、日期、地理位置等,可以满足各种不同类型数据的搜索需求。
-
分词和分析:Elasticsearch 7.x 支持文本分词和分析,它可以自动将文本分解成单词,然后进行索引和搜索,提高了搜索的准确性和效率。
-
RESTful API:Elasticsearch 7.x 提供了基于 RESTful API 的操作方式,可以通过 HTTP 请求与 Elasticsearch 进行交互,方便易用。
-
多语言支持:Elasticsearch 7.x 支持多种编程语言的客户端,包括 Java、Python、Ruby、PHP 等,可以方便地通过这些客户端进行索引和搜索操作。
-
实时监控和诊断:Elasticsearch 7.x 提供了实时监控和诊断工具,可以方便地监控集群状态、索引性能、节点状态等,提高了系统的可维护性和可靠性。
总之,Elasticsearch 7.x 是一个功能强大、性能优越、易于使用的分布式搜索引擎,可以满足各种不同规模的数据搜索和分析需求。
基本信息查询
查看所有索引
# 查看所有索引
GET _cat/indices
green open indexName1 785t4eK4SiarjIj6d_qDtA 1 0 0 0 283b 283b
green open indexName2 7WB8hk3cRMKuTUJhKWni9g 1 0 0 0 283b 283b
green open indexName3 PDJWT_VoS9iWSfEzUe5LEQ 1 0 0 0 283b 283b
green open indexName4 0qc3VwyvS9SIQ97GbILvJw 1 0 0 0 283b 283b
public Set<String> getAllIndex() {
GetAliasesRequest request = new GetAliasesRequest();
GetAliasesResponse response = null;
try {
response = restHighLevelClient.indices().getAlias(request, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
Map<String, Set<AliasMetadata>> aliases = response.getAliases();
Set<String> strings = aliases.keySet();
return strings;
}
查看索引 mapping
# 查看mapping
GET indexName/_mapping
{
"indexName" : {
"mappings" : {
"properties" : {
"field1" : {
"type" : "integer"
},
"field2" : {
"type" : "keyword"
},
"field3" : {
"type" : "integer"
},
"field4" : {
"type" : "long"
},
"field5" : {
"type" : "keyword"
},
"field6" : {
"type" : "integer"
}
}
}
}
public Map<String, String> getMapping(String indexName){
GetMappingsRequest request = new GetMappingsRequest().indices(indexName);
GetMappingsResponse response = null;
try {
response = restHighLevelClient.indices().getMapping(request, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
Map<String, Object> mappings = response.mappings().get(indexName).getSourceAsMap();
Map<String, Map<String, Object>> properties = (Map<String, Map<String, Object>>) mappings.get("properties");
Map<String, String> result = new HashMap<>();
properties.keySet().forEach(key -> {
String value = properties.get(key).get("type").toString();
System.out.println(key);
result.put(key, value);
} );
return result;
}
查索引中所有值
GET indexName/_search
{
"query":{
"match_all": {}
}
}
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "indexName",
"_type" : "_doc",
"_id" : "ngGsfYcB7ZKupvf2r47w",
"_score" : 1.0,
"_source" : {
"field1" : "value1",
"field2" : "value2"
}
},
{
"_index" : "indexName",
"_type" : "_doc",
"_id" : "ngGsfYcB7ZKupvf2r47w",
"_score" : 1.0,
"_source" : {
"field1" : "value1",
"field2" : "value2"
}
}
]
}
}
public List<Map<String, Object>> getAll(String indexName) {
SearchRequest searchRequest = new SearchRequest(indexName);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(QueryBuilders.matchAllQuery())
.size(10000);
searchRequest.source(searchSourceBuilder);
SearchResponse search = null;
try {
search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
SearchHit[] hits = search.getHits().getHits();
List<Map<String, Object>> result = new ArrayList<>();
Arrays.stream(hits).iterator().forEachRemaining(oneDoc -> {
Map<String, Object> sourceAsMap = (Map<String, Object>) oneDoc.getSourceAsMap();
result.add(sourceAsMap);
});
return result;
}
过滤
ES7.x中的bool查询是一种复合查询,可以将多个查询组合起来进行复杂的逻辑运算。bool查询中常用的子查询有以下几种:
- must查询:所有的子查询都必须匹配才算匹配成功。
- must_not查询:所有的子查询都必须不匹配才算匹配成功。
- should查询:至少有一个子查询匹配成功就算匹配成功,可以指定多个should查询,可以通过minimum_should_match参数控制至少需要匹配的子查询数量。
- filter查询:和must查询相似,但是不会计算得分,只用于过滤文档。 可以将这些子查询组合起来进行复杂的逻辑运算,例如可以使用must查询和should查询组合来实现"必须匹配A和B,或者匹配C"的查询条件。
每种子查询下面又有各种过滤条件:
- 相等:QueryBuilders.termQuery(fieldName, value)
- 取值范围:QueryBuilders.rangeQuery(fieldName).lt(leftValue).gt(rightValue)
- 除此之外还有 lte、gte、equals 等。
以下查询语句表示查询索引名为 indexName 中 fileTypeEnum 字段值必须是 TEMP、timestamp 字段值必须在 1680769009720-1680769909720之间,count 字段必须不是 19 的值。
GET /indexName/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"fileTypeEnum": {
"value": "TEMP"
}
}
},
{
"range": {
"timestamp": {
"gte": 1680769009720,
"lte": 1680769909720
}
}
}
],
"must_not": [
{
"term": {
"count": {
"value": "19"
}
}
}
]
}
},
"size": 10000
}
public List<Map<String, Object>> search(String indexName, SearchSourceBuilder searchSourceBuilder) {
SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(searchSourceBuilder);
SearchResponse search = null;
try {
search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
SearchHit[] hits = search.getHits().getHits();
List<Map<String, Object>> result = new ArrayList<>();
Arrays.stream(hits).iterator().forEachRemaining(oneDoc -> {
Map<String, Object> sourceAsMap = (Map<String, Object>) oneDoc.getSourceAsMap();
result.add(sourceAsMap);
});
return result;
}
定义了
search
函数,只需要传入index名称和一个SearchSourceBuilder对象即可,其中在SearchSourceBuilder对象中可以加入各种过滤条件。
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder()
.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("name", "张三")))
.query(QueryBuilders.boolQuery().mustNot(QueryBuilders.rangeQuery("timestape")
.lte(1001)
.gte(1101)));
List<Map<String, Object>> result = search(indexName, searchSourceBuilder);
模糊查询
模糊查询:QueryBuilders.wildcardQuery(fieldName, queryString)
-
*
表示匹配零个或多个任意字符; -
?
表示匹配一个任意字符。
GET /indexName/_search
{
"query": {
"wildcard": {
"location": "*123*"
}
}
}
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 270,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "indexName",
"_type" : "_doc",
"_id" : "ngGsfYcB7ZKupvf2r47w",
"_score" : 1.0,
"_source" : {
"field1" : "value1",
"field2" : "value2"
}
},
{
"_index" : "indexName",
"_type" : "_doc",
"_id" : "ngGsfYcB7ZKupvf2r47w",
"_score" : 1.0,
"_source" : {
"field1" : "value1",
"field2" : "value2"
}
}
]
}
}
List<String> contextLoads(String indexName) throws IOException {
// 模糊搜索
// *表示匹配任意多个字符(包括零个字符)
// ?表示匹配任意单个字符
WildcardQueryBuilder queryWildcard = QueryBuilders.wildcardQuery("location", "*123*");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(queryWildcard)
.from(0) // 设置分页
.size(500) // 默认只显示10个
.fetchSource(new String[]{"location"}, null) // 只返回location字段
;
SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(searchSourceBuilder);
SearchResponse search = null;
try {
search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
SearchHit[] hits = search.getHits().getHits();
List<String> result = new ArrayList<>();
Arrays.stream(hits).iterator().forEachRemaining(oneDoc -> {
Map<String, Object> sourceAsMap = (Map<String, Object>) oneDoc.getSourceAsMap();
result.add(sourceAsMap.get("location").toString());
});
return result
}
去重
对 indexName 中的 location 字段去重。
POST /indexName/_search
{
"size": 0,
"aggs": {
"distinct": {
"terms": {
"field": "location"
}
}
}
}
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 10000,
"relation" : "gte"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"distinct" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 688,
"buckets" : [
{
"key" : "30-200-1680592680000",
"doc_count" : 172
},
{
"key" : "30-200-1680592801000",
"doc_count" : 172
},
{
"key" : "30-200-1680593042000",
"doc_count" : 172
},
{
"key" : "30-200-1680593161000",
"doc_count" : 172
}
]
}
}
}
通过聚合去重文章来源:https://www.toymoban.com/news/detail-477118.html
public List<String> getDistance() {
SearchRequest searchRequest = new SearchRequest(indexName);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.aggregation(AggregationBuilders.terms("distinct_location").field("location").size(10));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = null;
try {
searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
Terms distinctValues = searchResponse.getAggregations().get("distinct_location");
List<String> collect = distinctValues.getBuckets().stream().map(Terms.Bucket::getKeyAsString).collect(Collectors.toList());
return collect;
}
通过查询去重文章来源地址https://www.toymoban.com/news/detail-477118.html
List<String> getDistance2() throws IOException {
SearchRequest searchRequest = new SearchRequest(indexName);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.collapse(new CollapseBuilder("location")) // 根据字段去重
.from(0) // 设置分页
.size(10) // 默认只显示10个
.fetchSource(new String[]{"location"}, null) // 只返回以上字段
;
searchRequest.source(searchSourceBuilder);
SearchResponse search = null;
try {
search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
SearchHit[] hits = search.getHits().getHits();
List<String> result = new ArrayList<>();
Arrays.stream(hits).iterator().forEachRemaining(oneDoc -> {
Map<String, Object> sourceAsMap = (Map<String, Object>) oneDoc.getSourceAsMap();
result.add(sourceAsMap.get("location").toString());
});
return result;
}
过滤后去重
GET /indexName/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"fileTypeEnum": {
"query": "TEMP"
}
}
},
{
"wildcard": {
"location": "*13*"
}
}
]
}
},
"aggs": {
"distinct": {
"terms": {
"field": "location"
}
}
}
}
String name = "location"; // 名称实际上是 location 字段
// 根据数据类型选择索引名称
SearchRequest searchRequest = new SearchRequest().indices(indexName);
/**
* 封装查询条件
*/
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
// fieldName1
if (StringUtils.isNotEmpty(DTO.getSubsystem())) {
boolQueryBuilder.must(QueryBuilders.termQuery(fieldName1, rawDataAssetsDTO.getSubsystem()));
}
// fieldName1
if (StringUtils.isNotEmpty(DTO.getSubject())) {
boolQueryBuilder.must(QueryBuilders.termQuery(fieldName1, DTO.getSubject()));
}
// 模糊查询 location
if (StringUtils.isNotEmpty(DTO.getLocation())) {
boolQueryBuilder.must(QueryBuilders.wildcardQuery(name, "*" + DTO.getLocation() + "*"));
}
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(boolQueryBuilder)
.aggregation(AggregationBuilders.terms("result").field(name).size(100))
;
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = null;
try {
searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
logger.error("查询ES数据异常:" + e.getMessage());
}
Terms distinctValues = searchResponse.getAggregations().get("result");
List<String> result = distinctValues.getBuckets().stream().map(Terms.Bucket::getKeyAsString).collect(Collectors.toList());
return result;
到了这里,关于ES7.x的查询语句及java查询语句的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!