elasticsearch api client使用 - Go语言中文社区

elasticsearch api client使用


一、基本概念

1、Node 与 Cluster

Elastic 本质上是一个分布式数据库,允许多台服务器协同工作,每台服务器可以运行多个 Elastic 实例。

单个 Elastic 实例称为一个节点(node)。一组节点构成一个集群(cluster)。

2、Index

Elastic 会索引所有字段,经过处理后写入一个反向索引(Inverted Index)。查找数据的时候,直接查找该索引。

所以,Elastic 数据管理的顶层单位就叫做 Index(索引)。它是单个数据库的同义词。每个 Index (即数据库)的名字必须是小写。

下面的命令可以查看当前节点的所有 Index。

$ curl -X GET 'http://localhost:9200/_cat/indices?v'

这里写图片描述

3、Document

Index 里面单条的记录称为 Document(文档)。许多条 Document 构成了一个 Index。

Document 使用 JSON 格式表示,下面是一个例子。

{
    "user": "张三",
    "title": "工程师",
    "desc": "数据库管理"
}

同一个 Index 里面的 Document,不要求有相同的结构(scheme),但是最好保持相同,这样有利于提高搜索效率。

4、Type

Document 可以分组,比如weather这个 Index 里面,可以按城市分组(北京和上海),也可以按气候分组(晴天和雨天)。这种分组就叫做 Type,它是虚拟的逻辑分组,用来过滤 Document。

不同的 Type 应该有相似的结构(schema),举例来说,id字段不能在这个组是字符串,在另一个组是数值。这是与关系型数据库的表的一个区别。性质完全不同的数据(比如products和logs)应该存成两个 Index,而不是一个 Index 里面的两个 Type(虽然可以做到)。

下面的命令可以列出每个 Index 所包含的 Type。

 $ curl 'localhost:9200/_mapping?pretty=true'

根据规划,Elastic 6.x 版只允许每个 Index 包含一个 Type,7.x 版将会彻底移除 Type。

以上部分摘自:http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html

二、通过Http请求排序

1、数据格式

{
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAATaBwFklfYTRhdy0wVHJxQUNpcm5sWVBHeHcAAAAAAEvhqhYwNTgtVi1xT1FUNlkxMl9CVldWM1lnAAAAAACXzBgWVlhBRnRfd2xRd09HdlduY2tRNXpmQQ==",
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "failed": 0
    },
    "hits": {
        "total": 9564,
        "max_score": 1,
        "hits": [
            {
                "_index": "alert-201712s",
                "_type": "HISTORY",
                "_id": "000E94E15DA381A680F9C0E0C14F1E7F-1513323398",
                "_score": 1,
                "_source": {
                    "duration": 120,
                    "times": 2,
                    "status": "resolve",
                    "level": "warning",
                    "project": "AAAA"                 
                }
            },
            {
                "_index": "alert-201712s",
                "_type": "HISTORY",
                "_id": "00A70A194DCF6DE937BC97610715DDCE-1513320277",
                "_score": 1,
                "_source": {
                    "duration": 120,
                    "times": 54,
                    "level": "critical",
                    "project": "BBBB"
                }
            },
            ..........
        ]
    }
}

想要先按照project聚合,再按照level聚合,再把聚合后的各个项目、各个level的duration求和(类似与sql中的select sum(duration) ….group by project,level)

2、聚合排序

通过postman请求:
请求方式:Post
url:

ip:9200/index名称/Type名称/_search

此处应该是:

localhost:9200/alert-201712s/HISTORY/_search

body参数:

{
    "size": 0,
    "query": {
        "bool": {
            "filter": {
                "terms": {
                    "project": ["AAAA",
                    "BBBB"]
                }
            }
        }
    },
    "aggs": {
        "projects": {
            "terms": {
                "field": "project",
                "size": 10000
            },
            "aggs": {
                "levels": {
                    "terms": {
                        "field": "level"
                    },
                    "aggs": {
                        "durations": {
                            "sum": {
                                "field": "duration"
                            }
                        }
                    }
                }
            }
        }
    }
}

body参数注意aggs的嵌套结构(层级)
查询结果:

{
   "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 3,
        "successful": 3,
        "failed": 0
    },
    "hits": {
        "total": 8768,
        "max_score": 0,
        "hits": []
    },
    "aggregations": {
        "types_count": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "AAA",
                    "doc_count": 2077,
                    "types_count": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "serious",
                                "doc_count": 789,
                                "durations": {
                                    "value": 18720
                                }
                            },
                            {
                                "key": "null",
                                "doc_count": 456,
                                "durations": {
                                    "value": 23
                                }
                            },
                            {
                                "key": "warning",
                                "doc_count": 401,
                                "durations": {
                                    "value": 234
                                }
                            },
                            {
                                "key": "critical",
                                "doc_count": 4,
                                "durations": {
                                    "value": 78
                                }
                            }
                        ]
                    }
                },
                {
                    "key": "BBB",
                    "doc_count": 1225,
                    "types_count": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "serious",
                                "doc_count": 966,
                                "durations": {
                                    "value": 56
                                }
                            },
                            {
                                "key": "null",
                                "doc_count": 258,
                                "durations": {
                                    "value": 34
                                }
                            },
                            {
                                "key": "critical",
                                "doc_count": 1,
                                "durations": {
                                    "value": 2343
                                }
                            }
                        ]
                    }
                }
    }
}

三、java http 请求

1、pom依赖

<dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>5.6.4</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>transport</artifactId>
            <version>5.1.1</version>
        </dependency>

2、代码

import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.entity.StringEntity;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.transport.TransportClient;

import java.util.HashMap;
import java.util.Map;

public class Test {

    private static String es_url = "localhost:9200";
    private TransportClient client;
    private IndexRequest source;

    //将postman中参数直接复制到idea中自动转义的
     private static String str = "{n" +
            "t"size": 0,n" +
            "t"query": {n" +
            "tt"bool": {n" +
            "ttt"filter": {n" +
            "tttt"terms": {n" +
            "ttttt"project": ["AA",n" +
            "ttttt"BB"]n" +
            "tttt}n" +
            "ttt}n" +
            "tt}n" +
            "t},n" +
            "t"aggs": {n" +
            "tt"projects": {n" +
            "ttt"terms": {n" +
            "tttt"field": "project",n" +
            "tttt"size": 10000n" +
            "ttt},n" +
            "ttt"aggs": {n" +
            "tttt"levels": {n" +
            "ttttt"terms": {n" +
            "tttttt"field": "level",n" +
            "tttttt"size": 10000n" +
            "ttttt},n" +
            "ttttt"aggs": {n" +
            "tttttt"durations": {n" +
            "ttttttt"sum": {n" +
            "tttttttt"field": "duration"n" +
            "ttttttt}n" +
            "tttttt}n" +
            "ttttt}n" +
            "tttt}n" +
            "ttt}n" +
            "tt}n" +
            "t}n" +
            "}";


    public static void main(String[] args) throws Exception {
        HttpHost[] hosts = new HttpHost[1];
        hosts[0] = HttpHost.create(es_url);
        //创建ES请求客户端
        RestClient restClient = RestClient.builder(hosts).build();
        String index = "alert-201712s";
        String type = "HISTORY";
        String endpoint = "/" + index + "/" + type + "/_search";
        Map params = new HashMap();
        StringEntity queryBody = new StringEntity(str, "UTF-8");
        Header header = new BasicHeader("content-type", "application/json");

        Response response = restClient.performRequest("GET", endpoint, params, queryBody, header);
        //System.out.println(response);

        String resultJson = EntityUtils.toString(response.getEntity());
        Gson gson = new Gson();
        //获取到返回的数据
        JsonObject resultObj = gson.fromJson(resultJson, JsonObject.class);

    }

}

个人微信公众号:
这里写图片描述

作者:jiankunking 出处:http://blog.csdn.net/jiankunking

版权声明:本文来源CSDN,感谢博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
原文链接:https://blog.csdn.net/xunzaosiyecao/article/details/78915618
站方申明:本站部分内容来自社区用户分享,若涉及侵权,请联系站方删除。
  • 发表于 2019-08-26 20:04:17
  • 阅读 ( 1915 )
  • 分类:

0 条评论

请先 登录 后评论

官方社群

GO教程

推荐文章

猜你喜欢