EIEIOTECHNOVATION
← Training programmes

EIEIO / AUTOMATION & EMERGING TECHNOLOGY

Log Aggregation and Search with ELK

Course outline

2 days

Duration

2 days

Overview

ELK, or ElasticSearch/LogStash/Kibana is a tool for centralizing disparate log files across servers into a single searchable store that can then be visualized and queried as necessary. ELK is actually 3 different pieces of software working together to provide this functionality. ElasticSearch is a search engine built on top of Lucene and allows for a searchable index. Logstash is an ETL tool that actually extracts the raw log files and directs them to ElasticSearch. Finally, Kibana allows for visualizing and querying the centralized store. With Kibana, understanding the aggregated log files becomes much easier.

What you'll learn

  • Understanding the installation and configuration of ELK
  • Learning the basics of curl, grok, JSON, and DSL
  • Importing, editing, and analysing data
  • Different types of searches
  • Developing graphs
  • Developing dashboards

Day 1

Basic Concepts and ElasticSearch

  • ELK overview – components
  • Installation and configuration of ElasticSearch

Logstash

  • Installing Logstash
  • Configuring Logstash
  • Inputs, Filters, and Output
  • Field reference
  • Conditionals

ElasticSearch Fundamentals

  • Indexing, updating, and deleting data
  • Searching data
  • Analyzing data
  • Aggregating data

Recap and Q&A

Kibana Fundamentals

  • Install Kibana
  • Configure Kabana
  • Kibana interface
  • Basic listing of data
  • Simple searches
  • Search with conditions
  • Saving and retrieving your searches

Kibana Advanced

  • Building graphs
  • Building additional graphs
  • Saving graphs
  • Retrieving graphs
  • Building dashboards
  • Building additional dashboards
  • Saving dashboards
  • Retrieving dashboards

Recap and Q&A

Day 2

Beats

  • What are Beats?
  • Filebeat
  • Packetbeat
  • Metricbeat
  • Winlogbeat
  • Auditbeat
  • Functionbeat
  • Configuring beats

ELK in Production

  • Monitoring Exceptions
  • Keeping up with growth and bursts
  • ELK Scalability in the Cloud

Recap and Q&A

  • Security
  • Maintainability
  • Upgrades
  • Mapping
  • Capacity Provisioning
  • Oversized Templates
  • Production fine-tuning

Use Cases

  • Development and Troubleshooting
  • Cloud operations
  • APM (Application Performance Monitoring)
  • Security and Compliance (Audit trail)
  • Business Intelligence (BI)
  • SEO

Hands-on Exercise

  • Ingest multiple log files and view the result in Kibana
  • Conduct simple searches against log files in Kibana
  • Create a simple dashboard in Kibana

Recap and Q&A

Programme scope, delivery arrangements and suitability are confirmed during enquiry.