Deploying and Running Apache Kafka on Kubernetes

Deploying and Running Apache Kafka on Kubernetes

1h 44mIntermediate2021-12-16

Authors

Kate Stanley

Kate Stanley

Software Engineer at IBM

Course details

Apache Kafka has become the de-facto event-streaming platform, and its popularity is growing every year. Did you know that you can even deploy Kafka on a Kubernetes cluster? Whether you’re deploying Kafka yourself or using a tool or a managed service, there are a few key points to keep in mind. In this course, instructor Kate Stanley talks you through how to deploy Kafka on Kubernetes effectively.

Explore the platform's internals and concepts and learn more about the Kubernetes features that fit best with Kafka. Kate covers the Kafka configuration options that are essential to a successful, healthy deployment. She teaches you how to configure Kafka to make it accessible from outside the Kubernetes cluster. Examples are drawn from existing open-source projects, taking you step by step through the deployment process and giving you tips to make it easier along the way.

Skills covered

KafkaKubernetesApacheDevOps ToolsData EngineeringDevOpsData ScienceOpen SourceDeep Dive (X:Y)

Concepts

Introduction

  • Introduction to this course

Running a Kafka Cluster on the Command Line

  • Running Apache Kafka on the command line
  • Sending Apache Kafka events

Deploying an Apache Kafka Cluster to Kubernetes

  • Apache Kafka brokers and their identity
  • Kubernetes StatefulSets for brokers
  • Installing Docker and Kubernetes
  • Deploying an Apache Kafka StatefulSet

Gaining Access to Apache Kafka

  • Listeners and advertised listeners
  • Multi-broker Apache Kafka cluster in containers
  • Multi-broker Apache Kafka cluster on Kubernetes in theory
  • Multi-broker Apache Kafka cluster on Kubernetes in practice
  • Deploying ZooKeeper

Keeping Apache Kafka on Kubernetes Healthy

  • Health probes in Kubernetes
  • Readiness probes in Kubernetes
  • Configuring Apache Kafka health
  • Configuring Apache Kafka readiness

Apache Kafka on Kubernetes in Production

  • High availability in Kubernetes
  • Automation for deploying Apache Kafka on Kubernetes
  • Deploying Apache Kafka with Strimzi Operators
40,000 Toman