EIEIOTECHNOVATION
← Training programmes

EIEIO / DATA SCIENCE

Apache Kafka and Integration with Spring (Advanced)

Course outline

1 day
Discuss this programme →

Duration

Apache Kafka and Integration with Spring (Advanced)

1 day

Overview

Apache Kafka is a distributed streaming platform that allows for the real-time handling of high volumes of data across various systems. It was originally developed by LinkedIn and later open-sourced as part of the Apache Software Foundation. Kafka is designed to address the challenges of high throughput, low latency, and scalability for data streaming applications.

The overall objectives of this Kafka course are to equip participants with a comprehensive understanding of Apache Kafka, a powerful distributed streaming platform used for real-time data processing.

Participants will gain foundational knowledge of Kafka's architecture, including its role as a key component in modern data infrastructures for handling high-throughput, low-latency data streams. They will learn how to install, configure, and manage Kafka clusters, enabling them to produce, consume.

Course Objectives

  • Gain an in-depth knowledge of Kafka’s architecture, core components (and its role as a distributed streaming platform in modern data ecosystems.
  • Learn to install, configure, and manage Kafka clusters and using Kafka Command Line Interface (CLI) tools for efficient cluster administration.
  • Develop expertise in using Kafka Producer and Consumer APIs to handle real-time data streams
  • Learn best practices for securing Kafka, ensuring fault tolerance, and optimizing performance for high availability.
  • Integrate Kafka with the Spring framework to build a powerful enterprise-level applications.

Pre-requisite

Knowledge in Java , Spring Boot and topics in Apache Kafka beginner is required for this course

Target Audience

Software developers, data engineers, DevOps professionals, data analysts, and enterprise architects, who need to harness real-time data streaming, whether to build scalable applications, manage data pipelines, or implement robust IT infrastructure.

Course Methodology

Theoretical lectures and hands-on labs to ensure comprehensive understanding and practical application of Apache Kafka concepts.

Course Outline

Integrating Spring Integration with Kafka

  • Spring Integration Kafka Support
  • Introduction to Spring Kafka project
  • Configuring Kafka Channels and Gateways
  • Kafka Inbound and Outbound Adapters
  • Lab: Creating Kafka Channels and Gateways
  • Hands-on exercise to create Kafka channels and gateways in Spring Integration

Advanced Messaging Patterns

  • Error Handling and Retries
  • Configuring error channels and retry mechanisms
  • Message Transformation and Enrichment
  • Using transformers and enrichers
  • Aggregation and Splitting
  • Implementing message aggregation and splitting patterns
  • Lab: Implementing Advanced Messaging Patterns
  • Hands-on exercise to implement error handling, message transformation, aggregation, and splitting

Transaction Management

  • Transactional Messaging with Kafka
  • Ensuring message delivery consistency
  • Configuring Transactions in Spring Integration
  • Using Kafka transactions with Spring Integration
  • Lab: Implementing Transactional Messaging
  • Hands-on exercise to implement transactional messaging in a Spring Integration flow

Monitoring and Management

  • Monitoring Kafka and Spring Integration Applications
  • Tools and techniques for monitoring
  • Kafka Lag Monitoring
  • Using tools like Kafka Exporter and Burrow
  • Lab: Setting Up Monitoring for Kafka and Spring Integration
  • Hands-on exercise to set up monitoring

Performance Tuning and Optimization

  • Kafka Performance Tuning
  • Optimizing Kafka producer and consumer performance
  • Spring Integration Performance Considerations
  • Best practices for optimizing Spring Integration flows
  • Lab: Performance Tuning Exercises
  • Hands-on exercise to optimize Kafka and Spring Integration performance

Security and Authentication

  • Securing Kafka
  • Configuring SSL and SASL for Kafka
  • Securing Spring Integration Flows
  • Implementing security measures in Spring Integration
  • Lab: Implementing Security
  • Hands-on exercise to secure Kafka and Spring Integration flows

Real-time Stream Processing with Kafka Streams and Spring Cloud Stream

  • Introduction to Kafka Streams
  • Key concepts and components of Kafka Streams
  • Integrating Kafka Streams with Spring Cloud Stream
  • Building stream processing applications
  • Lab: Implementing Real-time Stream Processing
  • Hands-on exercise to build a Kafka Streams application with Spring Cloud Stream

Programme scope, delivery arrangements and suitability are confirmed during enquiry.