Elasticsearch Alerting with X-Pack (formerly known as Watcher), writes it watch executions in a daily indices. If you don’t keep an eye on that, you use a lot of shards on small indices. Curator offers the capability of the reindex action, i.e. write data from a daily index into a month or year index. This post contains an example for Elasticsearch v5.4.3 and Elasticsearch Curator v5.1.1.
The actionfile in yaml
actions: 1: description: "Create target index as named" action: create_index options: name: '.watcher-history-3-2017' 2: description: "Reindex daily watcher index into monthly index" action: reindex options: disable_action: False wait_interval: 9 max_wait: -1 request_body: source: index: REINDEX_SELECTION dest: index: .watcher-history-3-2017 filters: - filtertype: pattern kind: prefix value: .watcher-history-3-2017. 3: description: >- WATCHER: Delete indices older than 1 day action: delete_indices options: ignore_empty_list: True timeout_override: continue_if_exception: False disable_action: False filters: - filtertype: pattern kind: prefix value: .watcher-history-3-2017. exclude: - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 1 exclude:
The actions explained
- If the target index does not exists, it will be created. If it exists, nothing will happen .
- The reindex action will take all daily indices and reindex it to the target index.
- After the reindex the daily indices are deleted, since the data is then redundant.
Curator is a great tool to tend to Elasticsearch indices, but on the reindex action I miss a little bit of flexibility. So far no date pattern can be used for replacing the year or current month. If you reindex the data into a year index, you don’t have to touch the actionfile so often.