Loading...

Reindex data from remote cluster

Used:

  • elasticsearch v5.6.4

At work I still run the Elasticsearch Cluster in version 5.6.4. While I’m eager to upgrade and keep up the pace, I don’t always have the chance to upgrade immediately. A customer of mine needed a small set of data in Excel. Elasticsearch 6 or moreover Kibana 6 offers the CSV export in the X-Pack extensions. To use that functionality, I needed to export a fragment of desired data from my production cluster. Since the Reindex API allows us to read data from remote and write it, I simply ramped up my private cluster in v 6.1.1 with Docker and started the reindexing.

I ran into trouble. In order to do the export, the remote cluster addresses has to be added to the whitelist of my cluster as security precaution. Edit config/elasticsearch.yml:

reindex.remote.whitelist: ["dev:9200", "itu:9200", "etu:9200"]

Add it to all nodes and perform a rolling cluster restart.

Start reindex example with Kibana Console

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://etu:9200",
      "username": "admin",
      "password": "mapperking"
    },
    "index": "logs-2018.01.08",
    "query": {
      "match": {
        "terminal": "10041796"
      }
    }
  },
  "dest": {
    "index": "export"
  }
}