Stream Processing Patterns in Apache Flink

Course details

Frameworks such as Apache Flink can help you build fast, scalable stream processing applications, but big data engineers still need to design smart use cases to achieve maximum efficiency. In this course, instructor Kumaran Ponnambalam demonstrates how to use Apache Flink and associated technologies to build stream-processing use cases leveraging popular patterns. Kumaran begins by highlighting the opportunities and challenges that stream processing brings to big data. He then goes over four popular patterns for stream processing: streaming analytics, alerts and thresholds, leaderboards, and real-time predictions. Along the way, he reviews example use cases and explains how to leverage Flink, as well as key technologies like MariaDB and Redis, to implement key examples.

Concepts

Introduction

Stream processing with Flink
What you should know

Stream Processing with Flink

What is stream processing
Streaming Opportunities and challenges
Streaming with Flink
Setting up the exercise files
Setting up Kafka
Setting up MariaDB and Redis

Streaming Analytics

Streaming analytics Pattern
Streaming analytics Use case design
Streaming analytics Helper classes
Streaming analytics Pipeline implementation
Streaming analytics Results review

Alerts and Thresholds

Alerts and thresholds Pattern
Alerts and thresholds Use case design
Alerts and thresholds Helper classes
Alerts and thresholds Pipeline implementation
Alerts and thresholds Review

Leaderboards

Leaderboards Pattern
Leaderboards Use case design
Leaderboards Helper classes
Leaderboards Pipeline implementation
Leaderboards Review

Real-Time Predictions

Real-time predictions Pattern
Real-time predictions Use case design
Real-time predictions Helper classes
Real-time predictions Pipeline implementation
Real-time predictions Review

Use Case Project

Use case definition
Design of the project
Code walkthrough
Execute and analyze

Conclusion

Next steps