SpringBoot 基础10 Elasticsearch

2024-03-02 17:42:34

什么是ElasticSearch ？

      基于Lucene的搜索服务器，常被当做一个文档型 NoSQL 数据库在使用
​
     Elasticsearch是用Java语言开发的，并作为Apache许可条款下的开放源码发布，是一种流行的企业级搜索引擎
​
      默认端口：9200 
​

使用 Elastic Search有什么好处？

Elasticsearch可扩展高达PB级的结构化和非结构化数据
Elasticsearch可以用来替代MongoDB和RavenDB等做文档存储。 
Elasticsearch使用非标准化来提高搜索性能。 
Elasticsearch是受欢迎的企业搜索引擎之一，目前被许多大型组织使用，如Wikipedia，The Guardian，*，GitHub等。 
Elasticsearch是开放源代码，可在Apache许可证版本2.0下提供
​

Elastic Search有哪些特性？

Elasticsearch是基于Java开发的，这使得它在几乎每个平台上都兼容
 
Elasticsearch是实时的
​
Elasticsearch是分布式的
​
与Apache Solr相比，在Elasticsearch中处理多租户非常容易
​
Elasticsearch使用JSON对象作为响应Elasticsearch支持
​

如何使用 ElasticSearch？

首先用户将数据提交到Elastic Search 数据库中
​
再通过分词控制器去将对应的语句分词，将其权重和分词结果一并存入数据
​
当用户搜索数据时候，再根据权重将结果排名，打分
​
最后将返回结果呈现给用户
​

新建ElasticSearch项目

ElasticSearch可以从elasticsearch.org下载对应的文件格式(如ZIP和TAR.GZ)
提前安装Java运行时环境
​
所使用的环境是Windows，可从命令窗口运行位于bin文件夹中的elasticsearch.bat
​
启动ElasticSearch在控制台的前台运行，这意味着我们可在控制台中看到运行信息或一些错误信息，并可以使用CTRL + C停止或关闭它•

Elasticsearch和RDBMS之间的比较

在Elasticsearch中，索引是类型的集合，因为数据库是RDBMS(关系数据库管理系统)中表的集合。每个表都是行的集合，就像每个映射都是JSON对象的Elasticsearch集合一样。
Elasticsearch 关系数据库
索引(index) 数据库
碎片 碎片
映射(type) 表
Document 行
Field 列
JSON对象 元组

SpringBoot + Elasticsearch 实战

pom.xml

<!-- Java High Level REST Client -->
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch -->
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>

application.properties

server.port=8833
spring.application.name=my-application
elasticsearch.address=localhost:9200

配置类

@Configuration
@Slf4j
public class ESConfig {
​
    // 地址长度
    private static final int ADDRESS_LENGTH = 2;
    // 地址方式
    private static final String HTTP_SCHEME = "http";
    // 权限验证
    final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
​
    /**
     * 使用冒号隔开ip和端口
     */
    @Value("${elasticsearch.address}")
    private String[] address;
​
    // @Value("${elasticsearch.username}")
    // private String username;
​
    // @Value("${elasticsearch.password}")
    // private String password;
​
​
    @Bean
    public RestClientBuilder restClientBuilder() {
        HttpHost[] hosts = Arrays.stream(address)
                .map(this::makeHttpHost)
                .filter(Objects::nonNull)
                .toArray(HttpHost[]::new);
        log.debug("hosts:{}", Arrays.toString(hosts));
        // 配置权限验证
        // credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
        RestClientBuilder restClientBuilder = RestClient.builder(hosts).setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
            @Override
            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
            }
        });
        return restClientBuilder;
    }
​
​
    @Bean(name = "highLevelClient")
    public RestHighLevelClient highLevelClient(@Autowired RestClientBuilder restClientBuilder) {
        // restClientBuilder.setMaxRetryTimeoutMillis(60000);
        return new RestHighLevelClient(restClientBuilder);
    }
​
    /**
     * 处理请求地址
     * @param s
     * @return HttpHost
     * StringUtils 方法的操作对象是Java.lang.String 类型的对象，是 JDK 提供的 String 类型操作方法的补充，
     * 并且是 null 安全的(即如果输入参数 String 为 null 则不会抛出 NullPointerException ，而是做了相应处理，
     */
    private HttpHost makeHttpHost(String s) {
        assert StringUtils.isNotEmpty(s);
        //
        String[] address = s.split(":");
        if (address.length == ADDRESS_LENGTH) {
            String ip = address[0];
            int port = Integer.parseInt(address[1]);
            return new HttpHost(ip, port, HTTP_SCHEME);
        } else {
            return null;
        }
    }
}
@Configuration
@EnableSwagger2
public class SwaggerConfig {
​
    private String version;
​
    @Bean
    public Docket createRestApi() {
​
        return new Docket(DocumentationType.SWAGGER_2)
                .genericModelSubstitutes(DeferredResult.class)
                .select()
                .paths(PathSelectors.any())
                .build().apiInfo(apiInfo()); //.globalOperationParameters(pars);
​
    }
​
    private ApiInfo apiInfo() {
​
        return new ApiInfoBuilder().title("Earth server")
                .description("****")
                .termsOfServiceUrl("http://www.baidu.com")
                .version("1.0").build();
    }
}

工具类

@Component
public class ESUtil {
​
    @Resource
    RestHighLevelClient restHighLevelClient;
​
    /**
     * 判断索引是否存在
     *
     * @param indexName
     * @return
     */
    public boolean isIndexExists(String indexName) {
        boolean exists = false;
        try {
            GetIndexRequest getIndexRequest = new GetIndexRequest(indexName);
            getIndexRequest.humanReadable(true);
            exists = restHighLevelClient.indices().exists(getIndexRequest,RequestOptions.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return exists;
    }
​
    /**
     * 删除索引
     *
     * @param indexName
     * @return
     */
    public boolean deleteIndex(String indexName) {
        boolean acknowledged = false;
        try {
            DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest(indexName);
            deleteIndexRequest.indicesOptions(IndicesOptions.LENIENT_EXPAND_OPEN);
            AcknowledgedResponse delete = restHighLevelClient.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);
            acknowledged = delete.isAcknowledged();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return acknowledged;
    }
​
}

实体类

@Data
public class Body {
​
    private User user;
​
    private String indexName;
}
@Data   // 实现get set 方法
@AllArgsConstructor  // 实现构造方法
public class ResponseBean {
​
    //状态码
    private Integer code;
    //返回信息
    private String message;
    //返回的数据
    private Object data;
​
}
@Data
public class User {
​
    private String name;
​
    private int age;
​
    private Double money;
​
    private String address;
​
    private String birthday;
}

服务类

@Service
@Slf4j
public class ESTestService {
​
    @Resource
    private RestHighLevelClient restHighLevelClient;
    @Resource
    ESUtil esUtil;
​
    /**
     *
     * @param indexName
     * @return  创建索引
     */
    public ResponseBean createIndex(String indexName){
        try {
            XContentBuilder builder = XContentFactory.jsonBuilder()
                    .startObject()
                    .field("properties")
                    .startObject()
                    .field("name").startObject().field("index", "true").field("type", "keyword").endObject()
                    .field("age").startObject().field("index", "true").field("type", "integer").endObject()
                    .field("money").startObject().field("index", "true").field("type", "double").endObject()
                    .field("address").startObject().field("index", "true").field("type", "text").field("analyzer", "ik_max_word").endObject()
                    .field("birthday").startObject().field("index", "true").field("type", "date").field("format", "strict_date_optional_time||epoch_millis").endObject()
                    .endObject()
                    .endObject();
            CreateIndexRequest createIndexRequest = new CreateIndexRequest(indexName);
            createIndexRequest.mapping(builder);
            CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(createIndexRequest, RequestOptions.DEFAULT);
            boolean acknowledged = createIndexResponse.isAcknowledged();
            if (acknowledged) {
                return new ResponseBean(200, "创建成功", null);
            } else {
                return new ResponseBean(1002, "创建失败", null);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
​
    /**
     * @param indexName
     * @return  查看索引是否村子
     */
    public ResponseBean indexExists(String indexName) {
        boolean isExists = esUtil.isIndexExists(indexName);
        return new ResponseBean(200, "查询成功", isExists);
    }
​
    /**
     *
     * @param indexName
     * @return  删除索引
     */
    public ResponseBean deleteIndex(String indexName) {
        boolean isDelete = esUtil.deleteIndex(indexName);
        if (isDelete) {
            return new ResponseBean(200, "删除成功", null);
        } else {
            return new ResponseBean(10002, "删除失败", null);
        }
    }
​
    /**
     *
     * @param user
     * @param indexName
     * @return   新增文档数据
     */
    public ResponseBean findIndustryClassList(User user, String indexName) {
        IndexRequest indexRequest = new IndexRequest(indexName);
        String userJson = JSONObject.toJSONString(user);
        indexRequest.source(userJson, XContentType.JSON);
        try {
            IndexResponse indexResponse = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
            if (indexResponse != null) {
                String id = indexResponse.getId();
                String index = indexResponse.getIndex();
                long version = indexResponse.getVersion();
                log.info("index:{},id:{}", index, id);
                if (indexResponse.getResult() == DocWriteResponse.Result.CREATED) {
                    System.out.println("新增文档成功!" + index + "-" + id + "-" + version);
                    return new ResponseBean(200, "插入成功", id);
                } else if (indexResponse.getResult() == DocWriteResponse.Result.UPDATED) {
                    System.out.println("修改文档成功!");
                    return new ResponseBean(10001, "插入失败", null);
                }
                // 分片处理信息
                ReplicationResponse.ShardInfo shardInfo = indexResponse.getShardInfo();
                if (shardInfo.getTotal() != shardInfo.getSuccessful()) {
                    System.out.println("分片处理信息.....");
                }
                // 如果有分片副本失败，可以获得失败原因信息
                if (shardInfo.getFailed() > 0) {
                    for (ReplicationResponse.ShardInfo.Failure failure : shardInfo.getFailures()) {
                        String reason = failure.reason();
                        System.out.println("副本失败原因：" + reason);
                    }
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
​
    /**
     *
     * @return    测试文档数据
     */
    public ResponseBean testESFind() {
        SearchRequest searchRequest = new SearchRequest("test_es");
        // SearchRequest searchRequest = new SearchRequest("test07");
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        //如果用name直接查询，其实是匹配name分词过后的索引查到的记录(倒排索引)；如果用name.keyword查询则是不分词的查询，正常查询到的记录
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("birthday").from("1991-01-01").to("2010-10-10").format("yyyy-MM-dd");//范围查询
//        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name.keyword", name);//精准查询
        PrefixQueryBuilder prefixQueryBuilder = QueryBuilders.prefixQuery("name.keyword", "张");//前缀查询
//        WildcardQueryBuilder wildcardQueryBuilder = QueryBuilders.wildcardQuery("name.keyword", "*三");//通配符查询
//        FuzzyQueryBuilder fuzzyQueryBuilder = QueryBuilders.fuzzyQuery("name", "三");//模糊查询
        FieldSortBuilder fieldSortBuilder = SortBuilders.fieldSort("age");//按照年龄排序
        fieldSortBuilder.sortMode(SortMode.MIN);//从小到大排序
​
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
        boolQueryBuilder.must(rangeQueryBuilder).should(prefixQueryBuilder);//and or  查询
​
        sourceBuilder.query(boolQueryBuilder).sort(fieldSortBuilder);//多条件查询
        sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        searchRequest.source(sourceBuilder);
        try {
            SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
            SearchHits hits = response.getHits();
            JSONArray jsonArray = new JSONArray();
            for (SearchHit hit : hits) {
                String sourceAsString = hit.getSourceAsString();
                JSONObject jsonObject = JSON.parseObject(sourceAsString);
                jsonArray.add(jsonObject);
            }
            return new ResponseBean(200, "查询成功", jsonArray);
        } catch (IOException e) {
            e.printStackTrace();
            return new ResponseBean(10001, "查询失败", null);
        }
    }
​
    /**
     *
     * @return  聚合查询
     */
    public ResponseBean testESFindAgg() {
        SearchRequest searchRequest = new SearchRequest("test_es");
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
​
        TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("by_age").field("age");
        sourceBuilder.aggregation(termsAggregationBuilder);
​
        sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        searchRequest.source(sourceBuilder);
​
        try {
            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
            Aggregations aggregations = searchResponse.getAggregations();
            Map<String, Aggregation> stringAggregationMap = aggregations.asMap();
            ParsedLongTerms parsedLongTerms = (ParsedLongTerms) stringAggregationMap.get("by_age");
            List<? extends Terms.Bucket> buckets = parsedLongTerms.getBuckets();
            Map<Integer, Long> map = new HashMap<>();
            for (Terms.Bucket bucket : buckets) {
                long docCount = bucket.getDocCount();//个数
                Number keyAsNumber = bucket.getKeyAsNumber();//年龄
                System.err.println(keyAsNumber + "岁的有" + docCount + "个");
                map.put(keyAsNumber.intValue(), docCount);
            }
            return new ResponseBean(200, "查询成功", map);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
    /**
     *
     * @param id
     * @param money
     * @return  更新文档数据
     */
    public ResponseBean testESUpdate(String id, Double money) {
        UpdateRequest updateRequest = new UpdateRequest("test_es", id);
        Map<String, Object> map = new HashMap<>();
        map.put("money", money);
        updateRequest.doc(map);
        try {
            UpdateResponse updateResponse = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
            if (updateResponse.getResult() == DocWriteResponse.Result.UPDATED) {
                return new ResponseBean(200, "更新成功", null);
            } else {
                return new ResponseBean(10002, "删除失败", null);
            }
        } catch (IOException e) {
            e.printStackTrace();
            return new ResponseBean(1003, "删除异常", null);
        }
    }
​
​
    /**
     *
     * @param id
     * @param indexName
     * @return   删除文档数据
     */
    public ResponseBean testESDelete(String id, String indexName) {
        DeleteRequest deleteRequest = new DeleteRequest(indexName);
        deleteRequest.id(id);
        try {
            DeleteResponse deleteResponse = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
            if (deleteResponse.getResult() == DocWriteResponse.Result.NOT_FOUND) {
                return new ResponseBean(1001, "删除失败", null);
            } else {
                return new ResponseBean(200, "删除成功", null);
            }
        } catch (IOException e) {
            e.printStackTrace();
            return new ResponseBean(1003, "删除异常", null);
        }
    }
}
​

控制类

@Api(value = "ES测试接口", tags = {"ES测试接口"})
@RestController
@RequestMapping("/es")
@CrossOrigin(origins = "*", methods = {RequestMethod.GET, RequestMethod.POST, RequestMethod.DELETE, RequestMethod.PUT})  // 跨域问题
@Slf4j
public class ESTestController {
​
    @Autowired
    private ESTestService esTestService;
​
    // test code
    @RequestMapping("hello")
    public String hello() {
        return "success";
    }
​
​
    @ApiOperation(value = "es测试创建索引接口", notes = "es测试创建索引接口")
    @RequestMapping(value = "/create/index", method = RequestMethod.POST)
    public ResponseBean createIndex(@RequestBody String indexName) {
        return esTestService.createIndex(indexName);
    }
​
​
    @ApiOperation(value = "es测试是否存在索引接口", notes = "es测试是否存在索引接口")
    @RequestMapping(value = "/index/exists", method = RequestMethod.POST)
    public ResponseBean indexExists(@RequestParam String indexName) {
        return esTestService.indexExists(indexName);
    }
​
​
    @ApiOperation(value = "es测试删除索引接口", notes = "es测试删除索引接口")
    @RequestMapping(value = "/delete/index", method = RequestMethod.POST)
    public ResponseBean deleteIndex(@RequestParam String indexName) {
//        boolean isDelete = esUtil.deleteIndex(indexName);
//        if (isDelete) {
//            return new ResponseBean(200, "删除成功", null);
//        } else {
//            return new ResponseBean(10002, "删除失败", null);
//        }
​
        return esTestService.deleteIndex(indexName);
    }
​
​
    @ApiOperation(value = "es测试插入接口", notes = "es测试插入接口")
    @RequestMapping(value = "/insert/data", method = RequestMethod.POST)
    public ResponseBean findIndustryClassList(@RequestBody Body body) {
        return esTestService.findIndustryClassList(body.getUser(),  body.getIndexName());
    }
​
​
    @ApiOperation(value = "es测试普通查询接口", notes = "es测试普通查询接口")
    @RequestMapping(value = "/query/data", method = RequestMethod.GET)
    public ResponseBean testESFind() {
        return esTestService.testESFind();
    }
​
​
    @ApiOperation(value = "es测试聚合查询接口", notes = "es测试聚合查询接口")
    @RequestMapping(value = "/query/agg", method = RequestMethod.GET)
    public ResponseBean testESFindAgg() {
        return esTestService.testESFindAgg();
    }
​
​
    @ApiOperation(value = "es测试更新接口", notes = "es测试更新接口")
    @RequestMapping(value = "/update/data", method = RequestMethod.GET)
    public ResponseBean testESUpdate(@RequestParam String id, @RequestParam Double money) {
        return esTestService.testESUpdate(id, money);
    }
​
​
    @ApiOperation(value = "es测试删除接口", notes = "es测试删除接口")
    @RequestMapping(value = "/delete/data", method = RequestMethod.GET)
    public ResponseBean testESDelete(@RequestParam String id, @RequestParam String indexName) {
        return esTestService.testESDelete(id, indexName);
    }
}
​

 

测试步骤

测试步骤：
(1)  创建索引  POST 请求方式
    http://localhost:8833/es/create/index
    
    name indexName   
    value  test07
    
    ===
    
    name indexName   
    value  test_es
    
(2)  查询索引是否存在   POST 请求方式
   http://localhost:8833/es/index/exists
   
   
(3)  删除索引    POST 请求方式
    http://localhost:8833/es/delete/index
    
    
(4) 添加数据    POST 请求方式
    http://localhost:8833/es/insert/data
{
    "user":{
        "name": "Jone",
​
        "age": 17, 
​
        "money ":9000,
​
        "address": "tianjin",
​
        " birthday": "2020-09-07"
    },
    "indexName" :"test07"
}
​
​
(5)  查询数据   GET 请求方式 
  http://localhost:8833/es/query/data
  
  
(6)  聚合查询   GET 请求方式 
 http://localhost:8833/es/query/agg
 
 
(7) 更新   GET 请求方式 
 http://localhost:8833/es/update/data
 http://localhost:8833/es/update/data?id=dwQSvHgBuv3r_g6B9iYS&money=9000
 
(8) 删除   GET 请求方式 
http://localhost:8833/delete/data

 

上一篇：线性回归api深度介绍


下一篇：iPhone开发Swift基础03 视频、网络请求