Data Management with Apache NiFi (2023)

Course details

Apache NiFi is designed to automate the flow of data between software systems. If you are a data management professional, this course can give you the understanding you need to get started with this powerful tool. Janani Ravi guides you through getting started with Apache NiFi. Learn about core concepts and architecture, as well as how to download, start, and log into Apache NiFi on both macOS and Windows. Find out how to build and run data flows and how to integrate NiFi with PostgreSQL. Explore ways to configure and use NiFi processors and processing groups and integrate NiFi with Amazon S3. Plus, discover advanced features of NiFi dataflows like configuring back pressure, using a funnel in the dataflow, setting up monitoring and configuring alerts for a dataflow, and much more.

Concepts

0. Introduction

01 - Data management with Apache NiFi

1. Getting Started with Apache NiFi

02 - Prerequisites
03 - Apache NiFi core concepts
04 - Apache NiFi architecture
05 - macOS - Download Apache NiFi
06 - Windows - Download Apache NiFi
07 - macOS and Windows - Starting the NiFi server
08 - macOS - Logging into NiFi
09 - Windows - Logging into Apache NiFi
10 - Exploring the NiFi UI

2. Building and Running Data Flows

11 - Adding and configuring processors
12 - Running a dataflow
13 - Flow summaries and flow operations
14 - Creating, saving, and importing flow templates

3. Integrating NiFi with PostgreSQL

15 - Set up a PostgreSQL table
16 - Add GetFile and PutDatabase processors
17 - Configuring the CSVReader controller service
18 - Configuring the DBConnectionPool controller service
19 - Running and encountering dataflow errors
20 - Configuring the ReplaceText processor
21 - Running a dataflow to insert data into PostgreSQL
22 - Configuring the Run schedule
23 - Configuring task concurrency
24 - Configure processors to execute SQL queries

4. Exploring NiFi Processors and Process Groups

25 - Configure processors to convert data formats
26 - Updating attributes and writing files
27 - Running the end-to-end flow
28 - Modularize dataflows with process groups
29 - Running a dataflow with a process group

5. Integrating NiFi with Amazon S3

30 - Creating a PostgreSQL table, Amazon S3 bucket, and an IAM user
31 - Reading from files and storing in a database
32 - Configure the PutS3Bucket processor
33 - Running the flow to read from file and write to database and Amazon S3

6. Using Advanced Features of NiFi Dataflows

34 - Configuring back pressure
35 - Observing the effects of back pressure
36 - Configuring an InvokeHTTP processor
37 - Using a funnel in the dataflow
38 - Running the dataflow to read from URL and write to files
39 - Setting up monitoring for the dataflow
40 - Configuring alerting for the dataflow
41 - Email alerts on dataflow success
42 - Email alerts on change in activity status

Conclusion

43 - Summary and next steps