Duration

Developing Apps using Kafka Core

4 days

Objectives

This course is designed for providing introductory knowledge about Apache Kafka. Initially you will focus on basics and architecture of Apache Kafka; you will also get familiar with Producer & Consumer side flows along with Managing Schema. You will also get an understanding about Kafka Connect Framework.

Who Should Attend?

This course is designed for Application Developers, QAs, Architects, DevOps Engineers who want to design or develop secure applications/products.

The intended audience for this course:

Application Developers
Quality Assurance
Architects
System Engineers
Technical Managers

Prerequisites

Attendees should preferably have prior Software Development experience along with basic knowledge of SQL and Unix commands.

Outline

Kafka Architecture

Background about Kappa Architecture
End to End Data Pipelines
Reference Architecture
Distributed Log Structure
Physical Architecture of Kafka
Partitions

Topics

Replicas
Producers & Consumers
Brokers
Roles and Responsibilities of various components
Key Terminologies

Kafka Internals

Producer API
Internals of Producer Side
Message Acknowledgement
Batching Messages
Keyed and Non-Keyed Messages
Compression
Batching
Consumer API
Replicas & High Watermarks
Ack
Retention
Rebalancing
Key configuration settings
Optimization Tips
Best Practices

Data Modeling & Schema Registry

Understanding Data Modeling
Determine no. of partitions
When to create a new topic
Avro Basics
Schema Evolution Concepts
Schema Registry
Hands-on Exercise - Integrating Confluent Kafka with PostgreSQL
Implementing CDC
Overview of Oracle database integration with Kafka
Using Oracle as a data source for Kafka producers
Configuring Kafka Connect for Oracle CDC using Debezium

KSQL DB

Overview

What is KSQLDB?
Why KSQLDB?
KSQL DB Architecture
KSQL DB Limitations
KSQL DB Key Syntax
Hands-on Exercise - Exploring KsqlDB

Strimzi Operator

Introduction to Strimzi
Features and benefits of Strimzi
Installing and setting up Strimzi
Managing Kafka clusters with Strimzi
Monitoring and scaling Kafka clusters with Strimzi

Kafka Streams Architecture

Kafka Streams’ Place in the Kafka Ecosystem
High Level Architecture Design
Kafka Streams Data Types

Streaming Analytics using KStreams

Streaming Introduction
Parallel Processing in KStreams
KStream
KTable
Caching in KTable
DSL API vs Processor API
Joins

Kafka Connect

Why Kafka Connect?
Physical Architecture
Key Configurations for Connect workers
Kafka Connect – Connectors
Kafka Connect - Tasks
Kafka Connect - Workers
Windowing related Concepts
Various types of Windowing
Hands-on Exercise - Working with KStreams

Monitoring

JMX
Confluent Control Center
ksqlDB-specific Considerations

Programme scope, delivery arrangements and suitability are confirmed during enquiry.