一、前言
我们在使用搜索功能的时候,有时,为了使搜索的结果更多更广,比如搜索字符串“领导力”,希望有这些组合的结果(领导力、领导、领、导、力)都要能够全部展示出来。
这里我们引入ElasticSearch结合分词插件,来实现这样的搜索功能。
二、搜索功能的需求
比如:一款app需要对“课程”进行关键字搜索。
首先:课程包含的信息有:标题、子标题、讲师、简介、所属标签、封面图片、讲数等等。
搜索的需求为:输入关键字,要能分词匹配课程的这些信息:标题、子标题、讲师、所属标签。
搜索需求:搜索结果的原型图如下图所示:
红色高亮为搜索结果,覆盖了标题、子标题、所属标签、讲师名称,并能进行分词搜索且关键字高亮。
三、需求开发
1、服务器安装ElasticSearch和IK分词器
参考链接: [Linux安装软件详解系列]05 安装ElasticSearch和IK分词器
2、需求开发
1)pom.xml引入jar包:
<!--elasticsearch-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
2)yml增加配置
// 安装ElasticSearch时对应的信息
elasticsearch:
rest:
uris: http://ip:9200
username: test
password: 123456
3)配置类ElasticsearchConfig
ElasticsearchConfig:
package com.test.api.config;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.client.ClientConfiguration;
import org.springframework.data.elasticsearch.client.RestClients;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;
/**
* es 配置
*
* @author /
*/
@EnableElasticsearchRepositories(basePackages = {"com.test.bi.*.repository"})
@Configuration
public class ElasticsearchConfig {
@Value("${spring.elasticsearch.rest.uris}")
private String hostAndPort;
@Value("${spring.elasticsearch.rest.username}")
private String username;
@Value("${spring.elasticsearch.rest.password}")
private String password;
@Bean
public RestHighLevelClient elasticsearchClient() {
ClientConfiguration clientConfiguration = ClientConfiguration.builder()
.connectedTo(hostAndPort)
.withBasicAuth(username, password)
.build();
return RestClients.create(clientConfiguration).rest();
}
}
4)工具类ElasticsearchUtil
ElasticsearchUtil:
package com.test.common.util;
import cn.hutool.core.collection.CollectionUtil;
import cn.hutool.json.JSONUtil;
import com.github.pagehelper.PageInfo;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchScrollRequest;
import org.elasticsearch.action.support.WriteRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.text.Text;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;
import org.springframework.util.ReflectionUtils;
import javax.annotation.Resource;
import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;
/**
* es 工具类
*
* @author /
*/
@Slf4j
@Component
public class ElasticsearchUtil {
@Resource
private RestHighLevelClient restHighLevelClient;
/**
* 普通查询
*
* @param index 索引名称
* @param searchSourceBuilder 查询条件构建
* @param resultClass 类
* @param currentPage 当前页 分页的页码,不是es 的
* @param size 每页显示数据
*/
public <T> PageInfo<T> page(String index, SearchSourceBuilder searchSourceBuilder, Class<T> resultClass,
int currentPage, int size, List<String> highFields) {
SearchRequest request = new SearchRequest(index);
// 高亮字段设置
if (CollectionUtil.isNotEmpty(highFields)) {
buildHighLight(searchSourceBuilder, highFields);
}
// 分页
int num = (currentPage - 1) * size;
searchSourceBuilder.from(num)
.size(size);
request.source(searchSourceBuilder);
SearchResponse response = null;
try {
response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
}
assert response != null;
return analysisResponse(response, resultClass, currentPage, size, highFields);
}
/**
* 解析es 查询结果
*
* @param response 返回
* @param resultClass 转换成对象类
*/
private <T> PageInfo<T> analysisResponse(SearchResponse response, Class<T> resultClass, int currentPage, int size, List<String> highFields) {
SearchHit[] searchHits = response.getHits().getHits();
List<T> retList = new ArrayList<>(searchHits.length);
for (SearchHit searchHit : searchHits) {
String strJson = searchHit.getSourceAsString();
T t = JSONUtil.toBean(strJson, resultClass);
try {
setId(resultClass, t, String.valueOf(searchHit.getId()));
} catch (Exception e) {
log.info("es 查询数据设置主键id值异常", e);
}
// 高亮字段设置后,组织结果,es 结果建议与java 对象 名称一直,基本要求
if (!CollectionUtils.isEmpty(highFields)) {
Map<String, HighlightField> highlightFieldMap = searchHit.getHighlightFields();
HighlightField highlightField;
for (String field : highFields) {
highlightField = highlightFieldMap.get(field);
if (highlightField != null) {
// 获取指定字段的高亮片段
Text[] fragments = highlightField.getFragments();
// 将这些高亮片段拼接成一个完整的高亮字段
StringBuilder builder = new StringBuilder();
for (Text text : fragments) {
builder.append(text);
}
// 设置到实体类中
setValue(resultClass, t, builder.toString(), field);
}
}
}
retList.add(t);
}
long totalNum = response.getHits().getTotalHits();
PageInfo<T> pageVo = new PageInfo<>();
pageVo.setPageNum(currentPage);
pageVo.setPageSize(size);
pageVo.setTotal(totalNum);
pageVo.setList(retList);
return pageVo;
}
/**
* 对象id 为空时,给id 赋值
*
* @param resultClass 类
* @param t 对象
* @param id 主键id 的值
*/
@SneakyThrows
private <T> void setId(Class<T> resultClass, T t, Object id) {
Field field = ReflectionUtils.findField(resultClass, "id");
if (null != field) {
field.setAccessible(true);
Object object = ReflectionUtils.getField(field, t);
if (object == null) {
Method method = resultClass.getMethod("setId", String.class);
ReflectionUtils.invokeMethod(method, t, id);
}
}
}
/**
* 指定字段赋值
*
* @param resultClass 类
* @param t 对象
* @param fieldValue 字段名
* @param fieldName 字段值
*/
@SneakyThrows
private <T> void setValue(Class<T> resultClass, T t, Object fieldValue, String fieldName) {
Field field = ReflectionUtils.findField(resultClass, fieldName);
if (null != field) {
field.setAccessible(true);
String methodName = "set".concat(captureName(fieldName));
Method method = resultClass.getMethod(methodName, String.class);
ReflectionUtils.invokeMethod(method, t, fieldValue);
}
}
/**
* 进行字母的ascii编码前移,效率要高于截取字符串进行转换的操作
*
* @param str /
*/
private String captureName(String str) {
char[] cs = str.toCharArray();
cs[0] -= 32;
return String.valueOf(cs);
}
/**
* 设置高亮
*
* @param searchSourceBuilder /
* @param fields 高亮字段
*/
private void buildHighLight(SearchSourceBuilder searchSourceBuilder, List<String> fields) {
// 设置高亮
HighlightBuilder highlightBuilder = new HighlightBuilder();
fields.forEach(highlightBuilder::field);
highlightBuilder.preTags("<em>");
highlightBuilder.postTags("</em>");
// 给请求设置高亮
searchSourceBuilder.highlighter(highlightBuilder);
}
/**
* es Scroll 深分页 定义bean
*/
@AllArgsConstructor
@Data
public class ScrollPageBean<T> {
private String scrollId;
private PageInfo<T> scrollPage;
}
}
5)返回的数据BO封装
package com.test.bi.bo.course;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
import java.io.Serializable;
/**
* 课程-搜索
*
* @author /
*/
@Document(indexName = "course_es", type = "_doc", replicas = 0)
@Data
public class CourseEsDTO implements Serializable {
@ApiModelProperty(value = "课程编号")
@Id
private String id;
@ApiModelProperty(value = "标题")
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String name;
@ApiModelProperty(value = "讲师名称")
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String teacherName;
@ApiModelProperty(value = "子标题")
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String subtitle;
@ApiModelProperty(value = "所属标签名称列表")
@Field(type = FieldType.Text, analyzer = "ik_max_word")
private String labelName;
@ApiModelProperty(value = "封面图片")
@Field(type = FieldType.Text)
private String pic;
@ApiModelProperty(value = "讲数")
@Field(type = FieldType.Integer)
private Integer count;
}
6)保存数据至ElasticSearch
保存数据到ElasticSearch中,通常有两种方式:一种是通过ElasticsearchRepository 接口,另一种是通过ElasticsearchTemplate接口,我们使用ElasticsearchRepository接口来实现。
实现接口:CourseEsRepository,可以放在mapper里面。
package com.test.bi.mapper;
import com.test.bi.bo.course.CourseEsDTO;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import org.springframework.stereotype.Repository;
/**
* 课程-搜索
*
* @author /
*/
@Repository
public interface CourseEsRepository extends ElasticsearchRepository<CourseEsDTO, String> {
}
保存数据:文章来源:https://www.toymoban.com/news/detail-436457.html
@Resource
private CourseEsRepository courseEsRepository;
@Override
public void saveCourseEs() {
try {
courseEsRepository.deleteAll();
// 从库中取数据
List<CourseEsDTO> list = courseMapper.getCourseEsList();
if (IterUtil.isNotEmpty(list)) {
courseEsRepository.saveAll(list);
}
} catch (Exception e) {
log.error("es error:{}", e.getMessage());
}
}
7)根据关键字搜索,分页返回数据
@Override
public PageInfo<CourseEsDTO> getCourseEs(String keywords, Integer pageNum, Integer pageSize) {
// 使用best_fields模式, 对多个字段进行查询
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
String[] queryFields = {"teacherName", "subtitle", "labelName"
};
QueryBuilder queryBuilder = QueryBuilders.multiMatchQuery(keywords, queryFields)
.field("name", 2F)
.tieBreaker(0.3F);
searchSourceBuilder.query(queryBuilder);
// 高亮
List<String> highFields = ListUtil.toList(queryFields);
highFields.add("name");
// 分页返回数据
return elasticsearchUtil.page("course_es", searchSourceBuilder, CourseEsDTO.class, pageNum,
pageSize, highFields);
}
8)总结
总结一下,就是通过一个小的需求例子,很好地实现了分词搜索,并且要能够高亮显示关键字。
具体的ElasticSearch和对应iK分词的代码功能,我就不一一展开了,可以去官方查看,这里只是演示一下实现搜索功能的简单过程,希望对大家有所帮助。文章来源地址https://www.toymoban.com/news/detail-436457.html
到了这里,关于[Spring Boot]12 ElasticSearch实现分词搜索功能的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!