DevOps Tools Introduction #13: Log Management and Analysis

While running, most applications, containers and virtual machines constantly generate information about numerous events. These events can be anything from severe errors to a simple notice that the server successfully answered a request. Collecting and analysing this log data become challenging in a dynamic architecture or microservice environment. The DevOps Tools Engineer exam covers log management and analysis in objective 705.2. The Elastic Stack, which includes the combination of tools Logstash, Elasticsearch and Kibana, is used as a reference implementation. From these tools, Logstash is the component which usually causes the most configuration effort and is the central focus of the objective.

Fortunately, the Logstash documentation is quite comprehensive. You should start with the first chapters, Logstash Introduction and Getting Started with Logstash. How Logstash Works summarizes the main elements of a Logstash pipeline.

Equipped with this knowledge, set up your first Elastic Stack. stack-docker provides a Docker Compose file which sets up the components of the Elastic Stack — and much more. Use this file to both gain more Docker experience and to set up Logstash, Elasticsearch, Kibana and, later, filebeat. Alternatively, follow Sébastien Pujadas’ elk-docker guide for setting up the Elastic Stack Docker.

Now that you have a playground, give a closer look to the Configure logstash guide. Follow all the subchapters, they cover important topics which are mentioned in the objectives.

A critical aspect of any logging infrastructure is collecting logging information and shipping it to the central log server. Filebeat is a modern approach to this need; it collects log data and sends it to another process, such as a Logstash server. The Filebeat documentation provides an overview of Filebeat along with the recommended getting started guide. The Logstash documentation describes Filebeat’s counterpart, the Beats input plugin.

Besides the modern Filebeat, LPI also expects you to use syslog to ship log data to Logstash. In case you’re not familiar with syslog, Aaron Leskiw’s introduction to syslog is a good place to start. You might also want to review the manpage of syslog.conf(5). To turn Logstash into a syslog server, the Syslog input should be configured.

In addition to the Beats and Syslog input plugins, Logstash’s functionality can be extended through the use of numerous Input, Output and Filter plugins. Browse through these indexes to learn more about the modules which are related to the technologies covered in the DevOps Tools Engineer exam. Take a special look at the Grok filter and Email alerting as they are explicitly mentioned in the exam objectives.

Elasticsearch is responsible for storing the log data. While this sounds unspectacular, indexes and data retention should be configured within Elasticsearch to support the analysis of log data. The Elasticsearch documenation’s getting started guide gives you an initial overview of Elastisearch itself. Afterwards, learn more about indexes and retiring data in Elasticsearch.

Once data is stored in Elasticsearch, Kibana provides a graphical way to access, aggregate and explore the logged information. The Kibana documentation explains how to interactively explore data, how to use visualization tools and how to create dashboards.

Finally, the exam objectives also expect an understanding of how applications can support logging. The internet contains a large number of tips and a comprehensive collection is ‘Beyond Application Monitoring’ by logomatic, ‘9 Logging Best Practices Based on Hands-on Experience’ by Liron Tal and ‘30 best practices for logging at scale’ by Jason Skowronski. Try to find the most useful best practices for you.

With these links, we have reached the end of the LPI DevOps Tools Engineer objectives. Next week we will take both a look back to what we’ve learned over the last few months as well as a look ahead on how to prepare for the exam and get certified.

Previous post | Next post