Apache Kafka Essential Training: Getting Started

Course details

One of the key components of a big data processing pipeline is a scalable and distributed message queue. Message queues enable real-time streaming capabilities with multiple producers and consumers of data. This enables real-time applications that can analyze data and produce insights in a scalable fashion. Apache Kafka provides these capabilities. As the de facto standard for open-source messaging, Apache Kafka is an essential skill for data scientists, big data engineers, data architects, and solution architects. In this course, instructor Kumaran Ponnambalam introduces Apache Kafka and explains its fundamental concepts and basic operations. Kumaran covers basic concepts like messages, topics, logs, and more. He shows you how to use the Kafka command line, as well as partitions and groups. He goes over Kafka Java programming, then concludes with a use case project.

Concepts

0. Introduction

01 - Getting started with Apache Kafka

1. Introduction to Kafka

02 - Message queues
03 - What is Kafka
04 - Benefits of Kafka
05 - Kafka use cases
06 - Setting up the exercise files
07 - Setting up Kafka

2. Basic Concepts

08 - Messages
09 - Topics
10 - Kafka brokers
11 - Logs in Kafka
12 - Producers and consumers

3. Using Kafka Command Line

13 - Kafka client scripts
14 - Creating a topic
15 - Exploring topics
16 - Publishing messages to topics
17 - Using console consumer
18 - Topics management
19 - Review Kafka setup

4. Kafka Partitions and Groups

20 - Intro to partitions
21 - Creating topics with partitions
22 - Publishing with keys
23 - Consumer groups
24 - Consumer offset management
25 - Consuming partitioned data

5. Kafka Java Programming

26 - Kafka client libraries
27 - Creating a producer in Java
28 - Publishing messages in Java
29 - Creating a consumer in Java
30 - Consuming messages in Java

6. Use Case Project

31 - Kafka basics use case - Problem definition
32 - Setting up topics
33 - Producing and consuming data with scripts
34 - Producing and consuming data with Java

Conclusion

35 - Extending your Kafka knowledge