Duration
2 days
Course Overview
In this course, you will delve into the intricacies of Apache Kafka, a powerful distributed event streaming platform. The journey begins with an introduction to big data and Kafka's architecture, covering essential components such as producers, consumers, brokers, topics, partitions, and replication. This foundational knowledge sets the stage for more advanced topics.
Next, you will explore Kafka installation using Kraft, learning key concepts and mastering CLI commands. You will also gain hands-on experience with Kafka producers and consumers. As you progress, the course delves deeper into Kafka brokers and partitions, emphasizing replication, reliability, fault tolerance, and guarantees.
Producer performance tuning is a critical aspect covered in detail. You will learn about producer configurations, API usage, message acknowledgements, and advanced features like idempotent and transactional producers. Similarly, consumer configurations are explored extensively to optimize performance and ensure efficient stream processing with Kafka Streams.
Monitoring and tools are vital for maintaining a healthy Kafka cluster. You will discover techniques for cluster monitoring using JMX metrics and various Kafka manager tools. Real-world case studies provide practical insights into performance tuning scenarios and best practices.
The course also covers Kafka cluster design for high availability and security. You will learn about deployment architectures, cluster setup, maintenance, upgrades, troubleshooting, and advanced monitoring techniques. Schema registry management is another crucial topic that ensures seamless schema compatibility and evolution.
Finally, you will dive into Kafka streams processing concepts and connectors. The course concludes with implementing a scalable Kafka solution from scratch. By the end of this comprehensive training program, you will be equipped with the skills needed to design, implement, tune, monitor, and secure robust Kafka clusters.
Target Audience
- Engineers
Methodologies
- Lecture
- Slides
- Case Studies
- Labs
- Q&A
What you'll learn
- Monitor Kafka clusters effectively using JMX metrics and manager tools.
- Develop scalable solutions by designing from scratch with performance testing.
- Optimize producer performance through advanced configurations and API usage.
- Design high-availability Kafka clusters with robust security measures.
- Enhance consumer efficiency by configuring offsets, groups, rebalancing strategies.
- Implement advanced stream processing techniques using KStream and KTable.
- Understand the architecture of Apache Kafka including producers, consumers, brokers, topics, partitions, and replication.
- Install and configure Kafka using Kraft while mastering CLI commands.
Outlines
- Introduction to Big Data and Apache Kafka
- Kafka Architecture
- Producers, consumers, brokers, topics, partitions, and replication
- Kafka Installation with Kraft
- Kafka Brokers and Partitions
- Producer Performance Tuning
- Consumer Configurations
- Kafka Monitoring and Tools
- Case Studies and Best Practices
- Kafka Cluster Design
- Administering Kafka with Kraft
- Advanced Monitoring Techniques
- Schema Registry and Management
- Kafka Streams and Processing
- Kafka Connectors
- Implementing a Scalable Kafka Solution
Programme scope, delivery arrangements and suitability are confirmed during enquiry.