Loading...

Ship Docker Container Logs to Elasticsearch with Fluentd

Used:

  • elasticsearch 6.1.2
  • fluentd 0.12
  • docker 17.12

By default, Docker captures the standard output (and standard error) of all your containers, and writes them in files using the JSON format. It is advised to set a max size, otherwise you will run out of disk space. Having unified logging with Elasticsearch allows you to investigate logs in a single point of view. Sending the logs to Elasticsearch from the Docker containers is quite easy. Fluentd is a data collector, which a Docker container can use by omitting the option --log-driver=fluentd.

docker run --log-driver=fluentd ubuntu echo 'Hello Fluentd!'

All we have to do, is to run Fluentd with the Elasticsearch output plugin. We can run Fluentd with Docker itself :smile:. Build fluentd Docker image with Elasticsearch plugin. This is a slight modified version of the official Docker image.

FROM fluent/fluentd:v0.12-onbuild

ENV http_proxy http://10.0.2.2:3128
ENV https_proxy https://10.0.2.2:3128
ENV no_proxy localhost,my-elasticsearch-cluster

USER root

RUN adduser -S fluent && apk add --update --virtual .build-deps \
        sudo build-base ruby-dev \
 && gem install \
        fluent-plugin-elasticsearch \
 && gem sources --clear-all \
 && apk del .build-deps \
 && rm -rf /var/cache/apk/* \
           /home/fluent/.gem/ruby/2.3.0/cache/*.gem

The fluentd configuration:

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match docker.consul>
  @type elasticsearch
  host my-es-cluster
  user cinhtau
  password "lemapper"
  port 9200
  tag_key @le_mapper
  flush_interval 5s # for testing
  logstash_format true
  logstash_prefix consul
  type_name doc
  pipeline consul
</match>

<match docker.*>
  @type elasticsearch
  host my-es-cluster
  user cinhtau
  password "lemapper"
  port 9200
  tag_key @le_mapper
  flush_interval 5s
  logstash_format true
  logstash_prefix docker
  type_name doc
</match>

Consul logs are ingested into the consul index name pattern which matches the docker.consul tag. Other output goes to the docker index.

The tag for fluentd is omitted by the run command:

docker run --log-driver=fluentd --log-opt fluentd-tag=docker.consul consul