My company uses Elasticsearch and Kibana for various reasons. One of my responsibilities is to ensure the stability of our elasticsearch test and production cluster. My users and me have problems to distinguish the various environments (test and production). It happen more than once, that I executed an action with Sense/Console in the wrong environment . To make a better distinctive appearance, I dug a little in the Kibana source around and found it no so hard, to alter the appearance.
If you run into the situation, that documents were written to a wrong index, you can use the Reindex API to copy the documents to the desired index. You can remove them afterwards with the Delete By Query API.
If a field with its datatype in the mapping is defined, e.g. duration as Integer, Elasticsearch has a default behavior of coercion, if the value for the duration field is String. The String will be written, but interpreted as Integer. This can be a little bit misleading if you use only the document perspective.
Data is not always clean. Depending on how it is produced a number might be rendered in the JSON body as a true JSON number, e.g. 10, but it might also be rendered as a string, e.g. “10”. Some developers use MDC to pass meta data into Elasticsearch. If you have data as String and want to use Kibana for visualizations you need a fix. The only way to fix that is to reindex the data. Using the Reindex API with usage of pipelines ensures that the data have the correct data type.
If you run Elasticsearch and use Kibana for various reasons, you better ensure to perform automatic backups. The time spend in searches, visualizations and dashboard should be worth that. If an Elasticsearch upgrade goes south, you are happy to have a backup. The main advantages of an Elasticsearch cluster, that you can join and remove additional nodes, which may differ in their resources and capacity. That’s the situation I constantly deal at work. Shard Allocation Filtering helps to setup smart rules for example hot warm architecture.