首页 文章 精选 留言 我的

精选列表

搜索[小程序·云开发],共10036篇文章
优秀的个人博客,低调大师

Elasticsearch Index API & Aggregations API & Query DSL

这篇小菜给大家演示和讲解一些Elasticsearch的API,如在工作中用到时,方便查阅。 一、Index API 创建索引库 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 curl-XPUT 'http://127.0.0.1:9200/test_index/' -d'{ "settings" :{ "index" :{ "number_of_shards" :3, "number_of_replicas" :1 } }, "mappings" :{ "type_test_01" :{ "properties" :{ "field1" :{ "type" : "string" }, "field2" :{ "type" : "string" } } }, "type_test_02" :{ "properties" :{ "field1" :{ "type" : "string" }, "field2" :{ "type" : "string" } } } } }' 验证索引库是否存在 1 curl–XHEAD-i 'http://127.0.0.1:9200/test_index?pretty' 注: 这里加上的?pretty参数,是为了让输出的格式更好看。 查看索引库的mapping信息 1 curl–XGET-i 'http://127.0.0.1:9200/test_index/_mapping?pretty' 验证当前库type为article是否存在 1 curl-XHEAD-i 'http://127.0.0.1:9200/test_index/article' 查看test_index索引库type为type_test_01的mapping信息 1 curl–XGET-i 'http://127.0.0.1:9200/test_index/_mapping/type_test_01/?pretty' 测试索引分词器 1 2 3 4 5 curl-XGET 'http://127.0.0.1:9200/_analyze?pretty' -d' { "analyzer" : "standard" , "text" : "thisisatest" }' 输出索引库的状态信息 1 curl 'http://127.0.0.1:9200/test_index/_stats?pretty' 输出索引库的分片相关信息 1 curl-XGET 'http://127.0.0.1:9200/test_index/_segments?pretty' 删除索引库 1 curl-XDELETEhttp: //127 .0.0.1:9200 /logstash-nginxacclog-2016 .09.20/ 二、Count API 简易语法 curl -XGET 'http://elasticsearch_server:port/索引库名称/_type(当前索引类型,没有可以不写)/_count 用例: 1、统计 logstash-nginxacclog-2016.10.09 索引库有多少条记录 1 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_count' 2、统计 logstash-nginxacclog-2016.10.09 索引库status为200的有多少条记录 1 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_count?q=status:200' DSL 写法 1 2 3 4 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_count' -d' { "query" : { "term" :{ "status" : "200" }} }' 三、Aggregations API (数据分析和统计) 注: 聚合相关的API只能对数值、日期 类型的字段做计算。 1、求平均数 业务场景: 统计访问日志中的平均响应时长 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "avg_num" :{ "avg" :{ "field" : "responsetime" }} }, "size" :0 #这里的size:0表示不输出匹配到数据,只输出聚合结果。 }' { "took" :598, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :32523067, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "avg_num" :{ "value" :0.0472613558675975 } } } #得到平均响应时长为0.0472613558675975秒 2、求最大值 业务场景:获取访问日志中最长的响应时间 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "max_num" :{ "max" :{ "field" : "responsetime" }} }, "size" :0 }' { "took" :29813, "timed_out" : false , "_shards" :{ "total" :431, "successful" :431, "failed" :0 }, "hits" :{ "total" :476952009, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "max_num" :{ "value" :65.576 } } } #得到最大响应时长为65.576秒 3、求最小值 业务场景: 获取访问日志中最快的响应时间 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "min_num" :{ "min" :{ "field" : "responsetime" }} }, "size" :0 }' { "took" :2145, "timed_out" : false , "_shards" :{ "total" :431, "successful" :431, "failed" :0 }, "hits" :{ "total" :477156773, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "min_num" :{ "value" :0.0 } } } #看来最快的响应时间竟然是0,笔者通过查询日志发现,原来这些响应时间为0的请求是被nginx拒绝掉的。 4、数值求和 业务场景: 统计一天的访问日志中为响应请求总共输出了多少流量。 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "sim_num" :{ "sum" :{ "field" : "size" }} }, "size" :0 }' { "took" :1226, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :32523067, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "sim_num" :{ "value" :6.9285945505E10 } } } #这个数有点大,后面的E10表示6.9285945505X10^10,笔者算了下,大概70GB流量。 5、获取常用的数据统计指标 其中包括( 最大值、最小值、平均值、求和、个数 ) 业务场景: 求访问日志中的 responsetime ( 最大值、最小值、平均值、求和、个数 ) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "like_stats" :{ "stats" :{ "field" : "responsetime" }} }, "size" :0 }' { "took" :2868, "timed_out" : false , "_shards" :{ "total" :431, "successful" :431, "failed" :0 }, "hits" :{ "total" :477797577, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "like_stats" :{ "count" :469345191, "min" :0.0, "max" :65.576, "avg" :0.06088492952649428, "sum" :2.8576048877634E7 } } } 这个是上面统计方式的增强版,新增了几个统计数据 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "like_stats" :{ "extended_stats" :{ "field" : "responsetime" }} }, "size" :0 }' { "took" :2830, "timed_out" : false , "_shards" :{ "total" :431, "successful" :431, "failed" :0 }, "hits" :{ "total" :478145456, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "like_stats" :{ "count" :469687072, "min" :0.0, "max" :65.576, "avg" :0.06087745173159307, "sum" :2.859335205463328E7, "sum_of_squares" :1.3162790273264633E7, "variance" :0.02431853151732958, "std_deviation" :0.1559440012226491, "std_deviation_bounds" :{ "upper" :0.3727654541768913, "lower" :-0.2510105507137051 } } } } #其中新增的三个返回结果分别是: #sum_of_squares平方和 #variance方差 #std_deviation标准差 6、统计数据在某个区间所占的百分比 业务场景: 求出访问日志中响应时间的各个区间,所占的百分比 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "outlier" :{ "percentiles" :{ "field" : "responsetime" }} }, "size" :0 }' { "took" :60737, "timed_out" : false , "_shards" :{ "total" :431, "successful" :431, "failed" :0 }, "hits" :{ "total" :478287997, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "outlier" :{ "values" :{ "1.0" :0.0, "5.0" :0.0, "25.0" :0.02, "50.0" :0.038999979789136247, "75.0" :0.06247223731250421, "95.0" :0.16479760590682113, "99.0" :0.520510492464275 } } } } #values对应的列为所占的百分比,右边则是对应的数据值。表示: #响应时间小于或等于0的请求占1% #响应时间小于或等于0的请求占5% #响应时间小于或等于0.02的请求占25% #响应时间小于或等于0.038999979789136247的请求占50% #响应时间小于或等于0.06247223731250421的请求占75% #响应时间小于或等于0.16479760590682113的请求占95% #响应时间小于或等于0.520510492464275的请求占99% #还可以通过percents参数,自定义一些百分比区间,如10%,30%,60%,90%等。 #注:经笔者测试,这个方法只能对数值类型的字段进行统计,无法操作字符串类型的字段。 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "outlier" :{ "percentiles" :{ "field" : "status" , "percents" :[5,10,20,50,99.9] } } }, "size" :0 }' 7、求指定字段数值在各个区间所占的百分比 业务场景:求响应时间 0, 0.01, 0.1, 0.2 在整个日志文件中,分别所占的百分比。 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "outlier" :{ "percentile_ranks" :{ "field" : "responsetime" , "values" :[0,0.01,0.1,0.2] } } }, "size" :0 }' { "took" :6950, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :32523067, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "outlier" :{ "values" :{ "0.0" :8.79897648675993, "0.01" :17.90331319256336, "0.1" :91.18297638776373, "0.2" :98.22564774611764 } } } } #响应时间小于或等于0的请求占8.7% #响应时间小于或等于0.01的请求占17.9% #响应时间小于或等于0.1的请求占91.1% #响应时间小于或等于0.2的请求占98.2% 8、求该数值范围内有多少文档匹配 业务场景: 求访问日志中的响应时间为,0 ~ 0.02、0.02 ~ 0.1 、大于 0.1 这三个数值区间内,各有多少文档匹配。 "ranges":[{"to": 0.02}, {"from":0.02,"to":0.1},{"from":0.1}] {"to": 0.02} 求响应时间 0 ~ 0.02 区间内的匹配文档数 {"from":0.02,"to":0.1} 求响应时间 0.02 ~ 0.1 区间内匹配的文档数 {"from":0.1} 求响应时间大于 0.1 匹配的文档数 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "range_info" :{ "range" :{ "field" : "responsetime" , "ranges" :[{ "to" :0.02},{ "from" :0.02, "to" :0.1},{ "from" :0.1}] } } }, "size" :0 }' { "took" :474, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :32523067, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "range_info" :{ "buckets" :[{ "key" : "*-0.02" , "to" :0.02, "to_as_string" : "0.02" , "doc_count" :9093600 },{ "key" : "0.02-0.1" , "from" :0.02, "from_as_string" : "0.02" , "to" :0.1, "to_as_string" : "0.1" , "doc_count" :20547128 },{ "key" : "0.1-*" , "from" :0.1, "from_as_string" : "0.1" , "doc_count" :2879418 }] } } } "aggregations" :{ "range_info" :{ "buckets" :[{ "key" : "*-0.02" , "to" :0.02, "to_as_string" : "0.02" , "doc_count" :9093600 } #响应时间在0~0.02的文档数是9093600 ,{ "key" : "0.02-0.1" , "from" :0.02, "from_as_string" : "0.02" , "to" :0.1, "to_as_string" : "0.1" , "doc_count" :20547128 } #响应时间在0.02~0.1的文档数是20547128 ,{ "key" : "0.1-*" , "from" :0.1, "from_as_string" : "0.1" , "doc_count" :2879418 } #响应时间在大于0.1的文档数是2879418 ] } } 9、求时间范围内有多少文档匹配 业务场景:求访问日志中,在 2016-10-09T01:00:00 之前的文档有多少。 和在 2016-10-09T02:00:00 之后的文档有多少。 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "match_all" :{}}, "aggs" :{ "range_info" :{ "date_range" :{ "field" : "@timestamp" , "ranges" :[{ "to" : "2016-10-09T01:00:00" },{ "from" : "2016-10-09T02:00:00" }] } } }, "size" :0 }' { "took" :432, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :32523067, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "range_info" :{ "buckets" :[{ "key" : "*-2016-10-09T01:00:00.000Z" , "to" :1.4759748E12, "to_as_string" : "2016-10-09T01:00:00.000Z" , "doc_count" :613460 }, #在2016-10-09T01:00:00之前的文档数有613460 { "key" : "2016-10-09T02:00:00.000Z-*" , "from" :1.4759784E12, "from_as_string" : "2016-10-09T02:00:00.000Z" , "doc_count" :31264881 } #在2016-10-09T02:00:00之后的文档数有31264881 ] } } } 10、聚合结果不依赖于查询结果集 "global":{} 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "term" :{ "status" : "200" }}, "aggs" :{ "all_articles" :{ "global" :{}, "aggs" :{ "sum_like" :{ "sum" :{ "field" : "responsetime" }} } } }, "size" :0 }' { "took" :1519, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :26686196, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "all_articles" :{ "doc_count" :32523067, "sum_like" :{ "value" :1536946.1929722272 } } } } #可以看到查询结果集hitstotal部分才匹配到26686196条记录。而聚合的文档数则是32523067多于查询结果匹配到的文档。 #聚合结果为1536946.1929722272 #我们再看看没有引用"global":{}参数的方式 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "query" : { "term" :{ "status" : "200" }}, "aggs" :{ "sum_like" :{ "sum" :{ "field" : "responsetime" }} }, "size" :0 }' { "took" :1326, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :26686196, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "sum_like" :{ "value" :1526710.3929916811 } } } #聚合结果小于上诉的结果。表示这次的聚合的值,是依赖于检索匹配到的文档。 11、分组聚合 用于统计指定字段在自定义的固定增长区间下,每个增长后的值,所匹配的文档数量。 1 2 3 4 5 6 7 8 9 10 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "aggs" :{ "like_histogram" :{ "histogram" :{ "field" : "status" , "interval" :200, "min_doc_count" :1} } }, "size" :0 }' #对status字段操作,增长区间为200,为了避免有的区间匹配为0所导致空数据,所以这里指定最小文档数为1"histogram":{"field":"status","interval":200,"min_doc_count":1} 12、分组聚合-基于时间做分组 "date_histogram":{"field": "@timestamp", "interval": "1d","format": "yyyy-MM-dd",} "field": "@timestamp" 指定记录时间的字段 "interval": "1d" 分组区间为每天. 1M 每月、1H 每小时、1m 每分钟 "format": "yyyy-MM-dd" 指定时间的输出格式 统计每天产生的日志数量 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-*/_search?pretty' -d'{ "aggs" :{ "date_histogram_info" :{ "date_histogram" :{ "field" : "@timestamp" , "interval" : "1d" , "format" : "yyyy-MM-dd" , "min_doc_count" :1} } } }' "aggregations" :{ "date_histogram_info" :{ "buckets" :[{ "key_as_string" : "2016-09-27" , "key" :1474934400000, "doc_count" :6895375 },{ "key_as_string" : "2016-09-28" , "key" :1475020800000, "doc_count" :1255775 },{ "key_as_string" : "2016-09-29" , "key" :1475107200000, "doc_count" :38512862 },{ "key_as_string" : "2016-09-30" , "key" :1475193600000, "doc_count" :35314225 },{ "key_as_string" : "2016-10-01" , "key" :1475280000000, "doc_count" :45358162 },{ "key_as_string" : "2016-10-02" , "key" :1475366400000, "doc_count" :42058056 },{ "key_as_string" : "2016-10-03" , "key" :1475452800000, "doc_count" :39945587 },{ "key_as_string" : "2016-10-04" , "key" :1475539200000, "doc_count" :39509128 },{ "key_as_string" : "2016-10-05" , "key" :1475625600000, "doc_count" :40506342 },{ "key_as_string" : "2016-10-06" , "key" :1475712000000, "doc_count" :43303499 },{ "key_as_string" : "2016-10-07" , "key" :1475798400000, "doc_count" :44234780 },{ "key_as_string" : "2016-10-08" , "key" :1475884800000, "doc_count" :32880600 },{ "key_as_string" : "2016-10-09" , "key" :1475971200000, "doc_count" :32523067 },{ "key_as_string" : "2016-10-10" , "key" :1476057600000, "doc_count" :31454044 },{ "key_as_string" : "2016-10-11" , "key" :1476144000000, "doc_count" :2018401 }] } } } #基于小时做分组 #统计当天每小时产生的日志数量 curl-XGET 'http://127.0.0.1:9200/logstash-nginxacclog-2016.10.09/_search?pretty' -d'{ "aggs" :{ "date_histogram_info" :{ "date_histogram" :{ "field" : "@timestamp" , "interval" : "1H" , "format" : "yyyy-MM-dd-H" , "min_doc_count" :1} } }, "size" :0 }' { "took" :530, "timed_out" : false , "_shards" :{ "total" :5, "successful" :5, "failed" :0 }, "hits" :{ "total" :32523067, "max_score" :0.0, "hits" :[] }, "aggregations" :{ "date_histogram_info" :{ "buckets" :[{ "key_as_string" : "2016-10-09-0" , "key" :1475971200000, "doc_count" :613460 },{ "key_as_string" : "2016-10-09-1" , "key" :1475974800000, "doc_count" :644726 },{ "key_as_string" : "2016-10-09-2" , "key" :1475978400000, "doc_count" :687196 },{ "key_as_string" : "2016-10-09-3" , "key" :1475982000000, "doc_count" :730831 },{ "key_as_string" : "2016-10-09-4" , "key" :1475985600000, "doc_count" :1460320 },{ "key_as_string" : "2016-10-09-5" , "key" :1475989200000, "doc_count" :1469098 },{ "key_as_string" : "2016-10-09-6" , "key" :1475992800000, "doc_count" :1004399 },{ "key_as_string" : "2016-10-09-7" , "key" :1475996400000, "doc_count" :962843 },{ "key_as_string" : "2016-10-09-8" , "key" :1476000000000, "doc_count" :1232560 },{ "key_as_string" : "2016-10-09-9" , "key" :1476003600000, "doc_count" :1809741 },{ "key_as_string" : "2016-10-09-10" , "key" :1476007200000, "doc_count" :2802804 },{ "key_as_string" : "2016-10-09-11" , "key" :1476010800000, "doc_count" :3941192 },{ "key_as_string" : "2016-10-09-12" , "key" :1476014400000, "doc_count" :4631032 },{ "key_as_string" : "2016-10-09-13" , "key" :1476018000000, "doc_count" :3651968 },{ "key_as_string" : "2016-10-09-14" , "key" :1476021600000, "doc_count" :2079933 },{ "key_as_string" : "2016-10-09-15" , "key" :1476025200000, "doc_count" :973578 },{ "key_as_string" : "2016-10-09-16" , "key" :1476028800000, "doc_count" :517435 },{ "key_as_string" : "2016-10-09-17" , "key" :1476032400000, "doc_count" :388382 },{ "key_as_string" : "2016-10-09-18" , "key" :1476036000000, "doc_count" :361296 },{ "key_as_string" : "2016-10-09-19" , "key" :1476039600000, "doc_count" :345926 },{ "key_as_string" : "2016-10-09-20" , "key" :1476043200000, "doc_count" :342214 },{ "key_as_string" :

优秀的个人博客,低调大师

MongoDB学习笔记(四)--索引 && 性能优化

索引 基础索引 用到ensureIndex方法建立索引,1为升序,-1为降序。 MongoDB数据库在创建集合的时候,默认会为_id创建索引。 注:当系统已有大量数据时,创建索引就是一个非常耗时的工作,只需要指定backgroud:true即可。 db.yyd.ensureIndex({"name":1},{"backgroud:true"}) 文档索引 注:下面的查询将不会用到索引,因为查询的顺序与索引建立的顺序不一样。 db.yyd.find({"address":{"short":"CD","city":"chengdu"}}) 组合索引 此时,无论city和short的顺序是前是后,都会用到索引。 删除索引 为了后面方便查看,先把刚才建立的索引删除掉。 唯一索引 这里的唯一索引就像非NoSQL数据库里面的主键一样。如果集合中有重复的值,系统会报错。 强制使用索引 先向里面插入数据,使name和age为索引,当查询的时候用explain()方法来查看查询所用到的索引indexBounds参数。由上图看出,没用到索引。 现在通过hint命令强制使用索引。 索引名称 索引的名称是默认取好了的,一般是“索引_1”这样的,当然也可以自己设置索引的名字。 db.yyd.ensureIndex({"name":1},{"name":"index_name"}) 优化器profiler profiler相当于MySQL中的慢查询日志,但是比慢查询日志更详细。 使用profiler优化器就必须先开启它,在mongoDB中有两种方法控制profiler的开关和级别。 方法一: 在启动服务的时候加上 –profiler 参数,通过这个参数来设置profiler的级别。 方法二: db.setProfilingLevel(level,slowms) 在客户端调用setProfilingLevel()方法。 level为级别,0代表不开启,1代表记录慢命令(默认100ms),3代表记录所有命令。 slowms代表慢命令执行的时间,超过这个时间将写入日志中。 查询profiler日志 db.system.profile.find() show profile 性能优化 创建索引 限定返回结果数 只查询使用到的字段,而不查询所以字段 采用capped集合 采用Server Side Code Execution hint 采用profiling mongosniff底层监控 mongosniff --source NET lo mongostat查看运行中的实例统计信息 mongostat db.serverStatus查看实例运行状态 >db.serverStatus() db.stats查看数据库状态 >db.stats() 本文转自我爱物联网博客园博客,原文链接:http://www.cnblogs.com/yydcdut/p/3563704.html,如需转载请自行联系原作者

资源下载

更多资源
腾讯云软件源

腾讯云软件源

为解决软件依赖安装时官方源访问速度慢的问题,腾讯云为一些软件搭建了缓存服务。您可以通过使用腾讯云软件源站来提升依赖包的安装速度。为了方便用户自由搭建服务架构,目前腾讯云软件源站支持公网访问和内网访问。

Nacos

Nacos

Nacos /nɑ:kəʊs/ 是 Dynamic Naming and Configuration Service 的首字母简称,一个易于构建 AI Agent 应用的动态服务发现、配置管理和AI智能体管理平台。Nacos 致力于帮助您发现、配置和管理微服务及AI智能体应用。Nacos 提供了一组简单易用的特性集,帮助您快速实现动态服务发现、服务配置、服务元数据、流量管理。Nacos 帮助您更敏捷和容易地构建、交付和管理微服务平台。

Spring

Spring

Spring框架(Spring Framework)是由Rod Johnson于2002年提出的开源Java企业级应用框架,旨在通过使用JavaBean替代传统EJB实现方式降低企业级编程开发的复杂性。该框架基于简单性、可测试性和松耦合性设计理念,提供核心容器、应用上下文、数据访问集成等模块,支持整合Hibernate、Struts等第三方框架,其适用范围不仅限于服务器端开发,绝大多数Java应用均可从中受益。

Rocky Linux

Rocky Linux

Rocky Linux(中文名:洛基)是由Gregory Kurtzer于2020年12月发起的企业级Linux发行版,作为CentOS稳定版停止维护后与RHEL(Red Hat Enterprise Linux)完全兼容的开源替代方案,由社区拥有并管理,支持x86_64、aarch64等架构。其通过重新编译RHEL源代码提供长期稳定性,采用模块化包装和SELinux安全架构,默认包含GNOME桌面环境及XFS文件系统,支持十年生命周期更新。

用户登录
用户注册