社区微信群开通啦,扫一扫抢先加入社区官方微信群
社区微信群
Elastic 本质上是一个分布式数据库,允许多台服务器协同工作,每台服务器可以运行多个 Elastic 实例。
单个 Elastic 实例称为一个节点(node)。一组节点构成一个集群(cluster)。
Elastic 会索引所有字段,经过处理后写入一个反向索引(Inverted Index)。查找数据的时候,直接查找该索引。
所以,Elastic 数据管理的顶层单位就叫做 Index(索引)。它是单个数据库的同义词。每个 Index (即数据库)的名字必须是小写。
下面的命令可以查看当前节点的所有 Index。
$ curl -X GET 'http://localhost:9200/_cat/indices?v'
Index 里面单条的记录称为 Document(文档)。许多条 Document 构成了一个 Index。
Document 使用 JSON 格式表示,下面是一个例子。
{
"user": "张三",
"title": "工程师",
"desc": "数据库管理"
}
同一个 Index 里面的 Document,不要求有相同的结构(scheme),但是最好保持相同,这样有利于提高搜索效率。
Document 可以分组,比如weather这个 Index 里面,可以按城市分组(北京和上海),也可以按气候分组(晴天和雨天)。这种分组就叫做 Type,它是虚拟的逻辑分组,用来过滤 Document。
不同的 Type 应该有相似的结构(schema),举例来说,id字段不能在这个组是字符串,在另一个组是数值。这是与关系型数据库的表的一个区别。性质完全不同的数据(比如products和logs)应该存成两个 Index,而不是一个 Index 里面的两个 Type(虽然可以做到)。
下面的命令可以列出每个 Index 所包含的 Type。
$ curl 'localhost:9200/_mapping?pretty=true'
根据规划,Elastic 6.x 版只允许每个 Index 包含一个 Type,7.x 版将会彻底移除 Type。
以上部分摘自:http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html
{
"_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAATaBwFklfYTRhdy0wVHJxQUNpcm5sWVBHeHcAAAAAAEvhqhYwNTgtVi1xT1FUNlkxMl9CVldWM1lnAAAAAACXzBgWVlhBRnRfd2xRd09HdlduY2tRNXpmQQ==",
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 9564,
"max_score": 1,
"hits": [
{
"_index": "alert-201712s",
"_type": "HISTORY",
"_id": "000E94E15DA381A680F9C0E0C14F1E7F-1513323398",
"_score": 1,
"_source": {
"duration": 120,
"times": 2,
"status": "resolve",
"level": "warning",
"project": "AAAA"
}
},
{
"_index": "alert-201712s",
"_type": "HISTORY",
"_id": "00A70A194DCF6DE937BC97610715DDCE-1513320277",
"_score": 1,
"_source": {
"duration": 120,
"times": 54,
"level": "critical",
"project": "BBBB"
}
},
..........
]
}
}
想要先按照project聚合,再按照level聚合,再把聚合后的各个项目、各个level的duration求和(类似与sql中的select sum(duration) ….group by project,level)
通过postman请求:
请求方式:Post
url:
ip:9200/index名称/Type名称/_search
此处应该是:
localhost:9200/alert-201712s/HISTORY/_search
body参数:
{
"size": 0,
"query": {
"bool": {
"filter": {
"terms": {
"project": ["AAAA",
"BBBB"]
}
}
}
},
"aggs": {
"projects": {
"terms": {
"field": "project",
"size": 10000
},
"aggs": {
"levels": {
"terms": {
"field": "level"
},
"aggs": {
"durations": {
"sum": {
"field": "duration"
}
}
}
}
}
}
}
}
body参数注意aggs的嵌套结构(层级)
查询结果:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 8768,
"max_score": 0,
"hits": []
},
"aggregations": {
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "AAA",
"doc_count": 2077,
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "serious",
"doc_count": 789,
"durations": {
"value": 18720
}
},
{
"key": "null",
"doc_count": 456,
"durations": {
"value": 23
}
},
{
"key": "warning",
"doc_count": 401,
"durations": {
"value": 234
}
},
{
"key": "critical",
"doc_count": 4,
"durations": {
"value": 78
}
}
]
}
},
{
"key": "BBB",
"doc_count": 1225,
"types_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "serious",
"doc_count": 966,
"durations": {
"value": 56
}
},
{
"key": "null",
"doc_count": 258,
"durations": {
"value": 34
}
},
{
"key": "critical",
"doc_count": 1,
"durations": {
"value": 2343
}
}
]
}
}
}
}
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>5.6.4</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>5.1.1</version>
</dependency>
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.apache.http.Header;
import org.apache.http.HttpHost;
import org.apache.http.entity.StringEntity;
import org.apache.http.message.BasicHeader;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.transport.TransportClient;
import java.util.HashMap;
import java.util.Map;
public class Test {
private static String es_url = "localhost:9200";
private TransportClient client;
private IndexRequest source;
//将postman中参数直接复制到idea中自动转义的
private static String str = "{n" +
"t"size": 0,n" +
"t"query": {n" +
"tt"bool": {n" +
"ttt"filter": {n" +
"tttt"terms": {n" +
"ttttt"project": ["AA",n" +
"ttttt"BB"]n" +
"tttt}n" +
"ttt}n" +
"tt}n" +
"t},n" +
"t"aggs": {n" +
"tt"projects": {n" +
"ttt"terms": {n" +
"tttt"field": "project",n" +
"tttt"size": 10000n" +
"ttt},n" +
"ttt"aggs": {n" +
"tttt"levels": {n" +
"ttttt"terms": {n" +
"tttttt"field": "level",n" +
"tttttt"size": 10000n" +
"ttttt},n" +
"ttttt"aggs": {n" +
"tttttt"durations": {n" +
"ttttttt"sum": {n" +
"tttttttt"field": "duration"n" +
"ttttttt}n" +
"tttttt}n" +
"ttttt}n" +
"tttt}n" +
"ttt}n" +
"tt}n" +
"t}n" +
"}";
public static void main(String[] args) throws Exception {
HttpHost[] hosts = new HttpHost[1];
hosts[0] = HttpHost.create(es_url);
//创建ES请求客户端
RestClient restClient = RestClient.builder(hosts).build();
String index = "alert-201712s";
String type = "HISTORY";
String endpoint = "/" + index + "/" + type + "/_search";
Map params = new HashMap();
StringEntity queryBody = new StringEntity(str, "UTF-8");
Header header = new BasicHeader("content-type", "application/json");
Response response = restClient.performRequest("GET", endpoint, params, queryBody, header);
//System.out.println(response);
String resultJson = EntityUtils.toString(response.getEntity());
Gson gson = new Gson();
//获取到返回的数据
JsonObject resultObj = gson.fromJson(resultJson, JsonObject.class);
}
}
个人微信公众号:
作者:jiankunking 出处:http://blog.csdn.net/jiankunking
如果觉得我的文章对您有用,请随意打赏。你的支持将鼓励我继续创作!