Loading...

Blog

A Journey of a Thousand Miles Begins with a Single Step

Elasticsearch Date Processor Pipeline

I write some configuration documents with the Elasticsearch low level Java Rest Client. The documents are missing a timestamp, therefore I define a simple pipeline, which adds the @timestamp field to my documents.

Read more

Remove multiple RPMs

A small recipe how to remove multiple applications with yum.

Read more

Import Currency codes into Elasticsearch

Working in the financial business requires to have the currency code master-data accessible for various reasons. The ISO 4217 currency codes can be obtained from the ISO Organization website. This post uses Logstash and the csv plugin to process the data and import it into Elasticsearch. Elasticsearch itself provides the REST interface, so every micro-service or web service can access the desired data.

Read more

Reindex Watcher Indices with Curator

Elasticsearch Alerting with X-Pack (formerly known as Watcher), writes it watch executions in a daily indices. If you don’t keep an eye on that, you use a lot of shards on small indices. Curator offers the capability of the reindex action, i.e. write data from a daily index into a month or year index. This post contains an example for Elasticsearch v5.4.3 and Elasticsearch Curator v5.1.1.

Read more

Service Dependencies in Monit

Monit allows start, stop and restart program instructions in process checks. If you restart with monit, no alarm or warning is triggered then. For instance you need to restart an application like logstash for configuration changes.

Read more

Evaluating Elasticsearch Watcher Cron Expression

Working with Elasticsearch Watcher enables you to put a cron schedule into the trigger. This is no ordinary linux cron expression. It looks like Quartz. If you want to test the correctness of the cron, you use the shipped utility croneval in the installed X-Pack directory.

Read more

The Base64 tool for Linux

HTTP basic authentication does base64 encoding. base64 exists on nearly every Linux distribution.

Read more

Elasticsearch Nodes Memory Usage Watcher

TL,DR (Too long, Don’t read.). If you have a dedicated monitoring cluster for your Elasticsearch clusters, you should at least monitor the memory usage of each node. This is very helpful. Instead of fetching the data from the cluster within, we query the monitoring cluster remotely. This watch was created on Elasticsearch with X-Pack v5.4.3. Pay attention, that some values are tweaked and not realistic for production scenarios. The interval for instance is set to 6 hours, since we will execute this watch manually. We also choose 60% as threshold. 75% or 80% would be more realistic for warning scenarios.

Read more

Install gosu for Docker

gosu is an essential help for dockerized applications. Following recipe is from daily work. I always have some connectivity issues due to security precautions. This recipe works especially behind corporate firewalls with a http proxy.

Read more