1 安装elasticsearch和Kibana
1.1.下载镜像
docker search elasticsearch docker pull elasticsearch:7.14.2
1.2.创建挂载的目录
mkdir -p /mydata/elasticsearch/config mkdir -p /mydata/elasticsearch/data echo "http.host: 0.0.0.0" >> /mydata/elasticsearch/config/elasticsearch.yml
1.3.创建容器并启动
docker run --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms64m -Xmx128m" -v /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /mydata/elasticsearch/data:/usr/share/elasticsearch/data -v /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins -d elasticsearch:7.6.2 其中elasticsearch.yml是挂载的配置文件,data是挂载的数据,plugins是es的插件,如ik,而数据挂载需要权限,需要设置data文件的权限为可读可写,需要下边的指令。 chmod -R 777 要修改的路径 -e "discovery.type=single-node" 设置为单节点 特别注意: -e ES_JAVA_OPTS="-Xms256m -Xmx256m" \ 测试环境下,设置ES的初始内存和最大内存,否则导致过大启动不了ES
1.4..Kibana启动
docker pull kibana:7.6.2 docker run --name kibana -e ELASTICSEARCH_HOSTS=http://自己的IP地址:9200 -p 5601:5601 -d kibana:7.6.2 //docker run --name kibana -e ELASTICSEARCH_URL=http://自己的IP地址:9200 -p 5601:5601 -d kibana:7.6.2 进入容器修改相应内容 server.port: 5601 server.host: 0.0.0.0 elasticsearch.hosts: [ "http://自己的IP地址:9200" ] i18n.locale: "Zh-CN" 然后访问页面 http://自己的IP地址:5601/app/kibana
2. kibana操作ElasticSearch
2.1._cat
GET /_cat/node 查看所有节点 GET /_cat/health 查看es健康状况 GET /_cat/master 查看主节点 GET /_cat/indices 查看所有索引
2.2 保存文档
保存一个数据,保存在那个索引的那个类型下,指定用唯一的标识,customer为索引,external为类型,1为标识。其中PUT和POST都可以,POST新增。如果不指定ID,会自动生成ID,指定ID就会修改这个数据,并新增版本号。PUT可以新增可以修改,PUT必须指定ID,一般都用来修改操作,不指定ID会报错。
PUT customer/external/1 { "name":"张三" } 返回结果 { "_index" : "customer", "_type" : "external", "_id" : "1", "_version" : 3, "result" : "updated", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "_seq_no" : 1001, "_primary_term" : 2 }
2.3 查询文档
GET customer/external/1 结果: { "_index" : "customer", //在那个索引 "_type" : "external", //在那个类型 "_id" : "1", //记录ID "_version" : 1, //版本号 "_seq_no" : 0, //并发控制字段,每次更新就+1,可用于乐观锁 "_primary_term" : 1, //主分片重新分配,如重启,就会变化 "found" : true, //true就是找到数据了 "_source" : { //数据 "name" : "张三" } }
2.4 更新文档
POST操作带_update会对比原来的数据,如果是一样的那就不会更新了 POST customer/external/1/_update { "doc":{ "name":"你好" } } POST操作不带_update会直接更新操作 POST customer/external/1 { "name":"你好" }
2.5 删除文档
DELETE customer/external/1
2.6 bulk批量API
需要加_bulk,然后请求体中的index是id,下边的是要保存的内容 POST customer/external/_bulk {"index":{"_id":1}} {"name":"榨干"} {"index":{"_id":2}} {"name":"你瞅啥"}
2.7 查询操作 .
先导入批量的数据,在进行查询操作。
1>.一种是通过REST request URI 发送搜索的参数,其中_search是固定写法,q=*是查询所有,sort=balance排序是按照balance排序的,asc是升序排序 GET customer/_search?q=*&sort=balance:asc 结果集,took是花费时间,timed_out没有超时,hits是命中的记录 2>.另一种是通过REST request body 来发送,query代表查询条件,match_all是查询所有,sort代表排序条件
GET customer/_search { "query": { "match_all": {} }, "sort": [ { "balance": "asc" } ] }
3>.分页操作,from是从第几条数据开始,size是一页多少个,默认是十条数据
4>.按需返回参数为,_source
GET customer/_search { "query": { "match_all": {} }, "sort": [ { "balance": "asc" } ], "from": 11, "size": 2, "_source": ["account_number","balance"] }
5>.全文检索,使用match操作,查询的结果是按照评分从高到低排序的
GET customer/_search { "query": { "match": { "age": 20 } } }
6>.match_phrase的精确匹配,
GET customer/_search { "query": { "match_phrase": { "age": 20 } } }
7>.多字段匹配,multi_match
GET customer/_search { "query": { "multi_match": { "query": "mill", "fields": ["address","email"] } } }
8>.复合查询bool,其中must是必须满足,must_not是必须不满足,should是应该满足,不过不满足的也能查出来,就是得分低,range是区间查询
GET customer/_search { "query": { "bool": { "must": [ {"match": { "gender": "F" }}, {"match": { "address": "Mill" }} ], "must_not": [ {"match": { "age": "38" }} ], "should": [ {"match": { "lastname": "Long" }} ] } } }
9>.filter过滤,区间查询操作,而且filter不会计算相关性得分
GET customer/_search { "query": { "bool": { "filter": [ {"range": { "age": { "gte": 10, "lte": 30 } }} ] } } }
10>.team查询,一些精确字段的推荐使用team,而一些全文检索的推荐使用match
GET customer/_search { "query": { "term": { "age": "28" } } }
11.keyword的作用:当有keyword的时候,就会精确查找,而没有keyword的时候,这个值会当成一个关键字
GET customer/_search { "query": {"match": { "address.keyword": "789 Madison" }} } GET customer/_search { "query": {"match_phrase": { "address": "789 Madison" }} }
2.13 es分析功能(聚合函数)
搜索address中包含mill的所有人的年龄分布以及平均年龄,但不显示这些人的详情 其中,aggs代表使用聚合函数,terms为结果种类求和,avg为平均值,size为0则不显示详细信息 GET customer/_search { "query": { "match": { "address": "mill" } }, "aggs": { "ageagg": { "terms": { "field": "age", "size": 10 } }, "ageavg":{ "avg": { "field": "age" } } }, "size": 0 } 聚合中还可以有子聚合 GET customer/_search { "query": { "match_all": {} }, "aggs": { "ageagg": { "terms": { "field": "age", "size": 10 }, "aggs": { "ageAvg": { "avg": { "field": "balance" } } } } }, "size": 0 }
3 rest-high-level-client整合ElasticSearch
3.1.导入依赖
<!-- 修改springboot默认整合的es的版本 --> <properties> <java.version>1.8</java.version> <elasticsearch.version>7.6.2</elasticsearch.version> </properties> <!-- elasticsearch-rest-high-level-client --> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.6.2</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.68</version> </dependency>
3.2.编写配置类
@Configuration public class ElasticSearchClientConfig { @Bean public RestHighLevelClient restHighLevelClient(){ RestHighLevelClient client = new RestHighLevelClient( RestClient.builder( new HttpHost("自己的IP地址", 9200, "http") ) ); return client; } }
3.3.进行es的索引操作
@Autowired @Qualifier("restHighLevelClient") private RestHighLevelClient client; //index名字,静态一般都是放在另一个类中的 public static final String ES_INDEX="han_index"; //创建索引 @Test public void createIndex() throws IOException { //1. 创建索引 CreateIndexRequest index = new CreateIndexRequest(ES_INDEX); //2. 客户端执行请求,请求后获得相应 CreateIndexResponse response = client.indices().create(index, RequestOptions.DEFAULT); //3.打印结果 System.out.println(response.toString()); } //测试索引是否存在 @Test public void exitIndex() throws IOException{ //1. GetIndexRequest request = new GetIndexRequest(ES_INDEX); boolean exists = client.indices().exists(request, RequestOptions.DEFAULT); System.out.println("是否存在"+exists); } //删除索引 @Test public void deleteIndex() throws IOException{ DeleteIndexRequest request = new DeleteIndexRequest(ES_INDEX); AcknowledgedResponse response = client.indices().delete(request, RequestOptions.DEFAULT); System.out.println("是否删除"+response); }
3.4.es的文档操作
@Autowired @Qualifier("restHighLevelClient") private RestHighLevelClient client; public static final String ES_INDEX="han_index"; //创建文档 @Test public void createDocument() throws IOException { //创建对象 UserInfo userInfo = new UserInfo("张三",12); //创建请求 IndexRequest request = new IndexRequest(ES_INDEX); //规则 request.id("1").timeout(TimeValue.timeValueSeconds(1)); //将数据放到请求中 request.source(JSON.toJSONString(userInfo), XContentType.JSON); //客户端发送请求,获取相应的结果 IndexResponse response = client.index(request, RequestOptions.DEFAULT); //打印一下 System.out.println(response.toString()); System.out.println(response.status()); } //判断是否存在 @Test public void exitDocument() throws IOException { GetRequest request = new GetRequest(ES_INDEX, "1"); //不获取返回的_source 的上下文 request.fetchSourceContext(new FetchSourceContext(false)); request.storedFields("_none"); boolean exists = client.exists(request, RequestOptions.DEFAULT); System.out.println(exists); } //获取文档信息 @Test public void getDocument() throws IOException { GetRequest request = new GetRequest(ES_INDEX, "1"); GetResponse response = client.get(request, RequestOptions.DEFAULT); System.out.println("获取到的结果"+response.getSourceAsString()); } //更新文档 @Test public void updateDocument() throws IOException { //创建对象 UserInfo userInfo = new UserInfo("李四",12); UpdateRequest request = new UpdateRequest(ES_INDEX, "1"); request.timeout("1s"); request.doc(JSON.toJSONString(userInfo),XContentType.JSON); UpdateResponse response = client.update(request, RequestOptions.DEFAULT); System.out.println(response.status()); } //删除文档 @Test public void deleteDocument() throws IOException{ DeleteRequest request = new DeleteRequest(ES_INDEX, "1"); request.timeout("1s"); DeleteResponse response = client.delete(request, RequestOptions.DEFAULT); System.out.println(response.status()); } //批量添加 @Test public void bulkDocument() throws IOException{ BulkRequest request = new BulkRequest(); request.timeout("10s"); ArrayList<UserInfo> userInfos = new ArrayList<>(); userInfos.add(new UserInfo("李四",1)); userInfos.add(new UserInfo("李四",2)); userInfos.add(new UserInfo("李四",3)); userInfos.add(new UserInfo("李四",4)); userInfos.add(new UserInfo("李四",5)); userInfos.add(new UserInfo("李四",6)); userInfos.add(new UserInfo("李四",7)); //进行批处理请求 for (int i = 0; i <userInfos.size() ; i++) { request.add( new IndexRequest(ES_INDEX) .id(""+(i+1)) .source(JSON.toJSONString(userInfos.get(i)),XContentType.JSON)); } BulkResponse response = client.bulk(request, RequestOptions.DEFAULT); System.out.println(response.hasFailures()); } //查询 @Test public void SearchDocument() throws IOException{ SearchRequest request = new SearchRequest(ES_INDEX); //构建搜索条件 SearchSourceBuilder builder = new SearchSourceBuilder(); //查询条件使用QueryBuilders工具来实现 //QueryBuilders.termQuery 精准查询 //QueryBuilders.matchAllQuery() 匹配全部 MatchQueryBuilder matchQuery = QueryBuilders.matchQuery("name", "李四"); builder.query(matchQuery); builder.timeout(new TimeValue(60, TimeUnit.SECONDS)); request.source(builder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); System.out.println("查询出的结果"+JSON.toJSONString(response.getHits())); }