Architecting Big Data Applications: Batch Mode Application Engineering

Course details

Big data applications allow data scientists and analysts to acquire, store, manage, and use big data to generate more consistent, data-driven results. In this course, instructor Kumaran Ponnambalam explores real-world business use cases and best practices for architecting big data applications using existing open-source technologies.

Learn how to architect both simple and complex batch processing applications, as you discover the basic principles of big data architectures such as horizontal scaling, distributed processing, technology selection and integration, and scheduling. Kumaran shows you how to minimize data volumes and distribute data loads uniformly, as well as how to use caches, reprocess data, troubleshoot errors, and more. Along the way, take your new skills to the next level with hands-on use cases that cover a variety of functional and technology domains.

Concepts

0. Introduction

01 - Architecting big data applications

1. Introduction to Big Data Applications

02 - Characteristics of batch processing
03 - Challenges building batch applications
04 - Technologies for batch big data engineering
05 - Use cases for batch big data
06 - Architecture process for data engineering

2. Big Data Architecture Principles

07 - Making the choice - Real-time vs. batch
08 - Horizontal scaling
09 - Distributed processing
10 - Technology selection
11 - Technology integrations

3. Batch Application Architecture Principles

12 - Schedule selection
13 - Minimizing data volumes
14 - Uniform load distribution
15 - Using caches
16 - Reprocessing

4. Use Case 1 - Audit Trail Data Archive

17 - Audit trail - Define the problem
18 - Audit trail - Study requirements
19 - Audit trail - Create a workflow
20 - Audit trail - Scale the workflow
21 - Audit trail - Select technologies
22 - Audit trail - Review final architecture

5. Use Case 2 - Advertising Analytics

23 - Advertising analytics - Define the problem
24 - Advertising analytics - Study requirements
25 - Advertising analytics - Create a workflow
26 - Advertising analytics - Scale the workflow
27 - Advertising analytics - Select technologies
28 - Advertising analytics - Review final architecture

6. Use Case 3 - Product Recommendations

29 - Product recommendations - Define the problem
30 - Product recommendations - Study requirements
31 - Product recommendations - Create a workflow
32 - Product recommendations - Scale the workflow
33 - Product recommendations - Select technologies
34 - Product recommendations - Review the final architecture

Conclusion

35 - Continuing to architect big data applications