Introduction to Data Engineering on AWS: Data Sourcing and Storage
1h 51mIntermediate2022-10-07
Authors

Dipali Kulshrestha
AWS-Certified Software Programmer and Cloud Architect
Course details
Businesses need data experts—now more than ever before. As data-driven decision-making has risen to boardroom prominence, the role of the data expert has become essential to understanding and scaling a business. In this course—the first in a two-part series—instructor Dipali Kulshrestha shows you how to get started on your professional journey and grow your career as a data engineer with AWS.
Get an introduction to the field of data engineering and why it’s so important in today’s business world. Explore a variety of data types, data lakes, data sources, and how to use built-in AWS components such as DynamoDB, Kinesis, and S3 to store and manage your streams. Find out how to leverage the full power of an end-to-end data engineering pipeline, from selecting and configuring ingestion patterns, to storing data for analytic processing with S3. Test your new skills along the way in the hands-on data challenges at the end of each section.
Get an introduction to the field of data engineering and why it’s so important in today’s business world. Explore a variety of data types, data lakes, data sources, and how to use built-in AWS components such as DynamoDB, Kinesis, and S3 to store and manage your streams. Find out how to leverage the full power of an end-to-end data engineering pipeline, from selecting and configuring ingestion patterns, to storing data for analytic processing with S3. Test your new skills along the way in the hands-on data challenges at the end of each section.
Skills covered
Data EngineeringAmazon Web Services (AWS)AmazonCloud ServicesCloud PlatformsCloud ComputingData ScienceOne-Off
Concepts
0. Introduction
- 01 - Data engineering with AWS
1. Data Engineering
- 02 - Overview of data engineering
- 03 - Importance of data engineering
- 04 - Types of data
- 05 - Overview of data lakes
- 06 - Introduction to data engineering pipeline
2. Data Sources
- 07 - Overview of data sources
- 08 - DynamoDB overview
- 09 - Understanding DynamoDB partitions and streams
- 10 - Lab - Set up CLI to create DynamoDB tables
- 11 - Lab - Create DynamoDB tables via Python script
- 12 - Challenge - DynamoDB
- 13 - Solution - DynamoDB
3. Ingestion
- 14 - Data ingestion overview and design considerations
- 15 - Ingestion technology landscape
- 16 - Kinesis Data Streams overview
- 17 - Kinesis producers and consumers
- 18 - Kinesis Data Firehose
- 19 - Demo Firehose
4. Storage
- 20 - Data storage overview and design considerations
- 21 - Storage technology landscape
- 22 - Elastic cache overview
- 23 - Amazon S3 overview
- 24 - Amazon S3 storage classes and lifecycle rules
- 25 - Amazon S3 Versioning and encryption
- 26 - Amazon S3 security and event notifications
- 27 - Challenge - Amazon S3
- 28 - Solution - Amazon S3
Conclusion
- 29 - Next steps