0、题记
1、什么是冷热架构?1.1 官方解读冷热架构
1.2 典型应用场景最大化解决客户反应的响应时间慢 的问题。每日增量6TB日志数据,高峰时段写入及查询频率都较高,集群压力较大,查询ES时,常出现查询缓慢问题。
2、最最核心的实现原理
https://www./guide/en/elasticsearch/reference/current/allocation-awareness.html https://www./guide/en/elasticsearch/reference/current/shard-allocation-filtering.html# 3、7.X版本ES实践一把节点层面设置节点类型。 热节点指定: 方案一:索引层面指定路由。 2{ 3 "settings": { 4 "index.routing.allocation.require.hotwarm_type": "hot", 5 "number_of_replicas": 0 6 } 7} 8 9 10PUT /logs_2019-08-01 11{ 12 "settings": { 13 "index.routing.allocation.require.hotwarm_type": "warm", 14 "number_of_replicas": 0 15 } 16} 2{ 3 "index_patterns": "logs_2019-08-*", 4 "settings": { 5 "index.number_of_replicas": "0", 6 "index.routing.allocation.require.hotwarm_type": "warm" 7 } 8} 9PUT _template/logs_2019-10-template 10{ 11 "index_patterns": "logs_2019-10-*", 12 "settings": { 13 "index.number_of_replicas": "0", 14 "index.routing.allocation.require.hotwarm_type": "hot" 15 } 16} 可以看出,两个索引分不到不同的节点上。 随着时间发展,当前数据会成为历史数据。 https://www./guide/en/elasticsearch/client/curator/5.8/command-line.html
2 1: 3 action: allocation 4 description: >- 5 Apply shard allocation routing to 'require' 'tag=cold' for hot/cold node 6 setup for logstash- indices older than 3 days, based on index_creation 7 date 8 options: 9 key: hotwarm_type 10 value: warm 11 allocation_type: require 12 disable_action: false 13 filters: 14 - filtertype: pattern 15 kind: prefix 16 value: logs_ 17 - filtertype: age 18 source: name 19 direction: older 20 timestring: "%Y-%m-%d" 21 unit: days 22 unit_count: 3
22019-10-13 22:28:31,662 INFO Preparing Action ID: 1, "allocation" 32019-10-13 22:28:31,662 INFO Creating client object and testing connection 42019-10-13 22:28:31,668 INFO Instantiating client object 52019-10-13 22:28:31,668 INFO Testing client connectivity 62019-10-13 22:28:31,675 INFO Successfully created Elasticsearch client object with provided settings 72019-10-13 22:28:31,677 INFO Trying Action ID: 1, "allocation": Apply shard allocation routing to 'require' 'tag=cold' f.... 82019-10-13 22:28:31,706 INFO Updating 2 selected indices: ['logs_2019-08-01', 'logs_2019-10-01'] 92019-10-13 22:28:31,706 INFO Updating index setting {'index.routing.allocation.require.hotwarm_type': 'warm'} 102019-10-13 22:28:32,559 INFO Action ID: 1, "allocation" completed. 112019-10-13 22:28:32,560 INFO Job completed. 4、坑5、线上使用场景
特点 是:
相对节约成本的前提下极大的提升性能 ,但是不能完全做到一种类型节点的故障对其他类型节点是无感的。6、小结7、Good 参考深入学习https://www./cn/blog/deploying-a-hot-warm-logging-cluster-on-the-elasticsearch-service https://www./cn/blog/hot-warm-architecture-in-elasticsearch-5-x https:///article/6127 更短时间更快习得更多干货! |
|