Stream Processing Design Patterns with Spark

Stream processing is becoming more popular as more and more data is generated by websites, devices, and communications. Apache Spark is a leading platform that provides scalable and fast stream processing, but still requires smart design to achieve maximum efficiency. This course helps developers use best practices and validated design patterns to implement stream processing in Apache Spark. Instructor Kumaran Ponnambalam shows how to set up your environment and then walks through four design patterns and real-world use cases: streaming analytics, alerts and thresholds, leaderboards, and real-time predictions. In chapter six, he introduces a start-to-finish project that shows how to go from design to executed job using Spark, Apache Kafka, MariaDB, and Redis. By the end of the course, you'll understand all the capabilities of this powerful platform and be able to incorporate it in your own data engineering solutions.

Learning objectives
Streaming opportunities and challenges
Setting up the environment
Steaming analytics with Spark
Monitoring alerts and thresholds with Spark
Creating leaderboards with Spark
Generating real-time predictions with Spark
Hands-on Spark streaming project