Google Cloud Data Engineering Foundations
1h 57mIntermediate2024-09-04
Authors
Bhavani Ravi
Back-End Engineer, Speaker, and Community Leader
Course details
This course provides a high-level overview of all the tools and data engineering techniques employed with Google Cloud. Instructor Bhavani Ravi covers important aspects of data engineering concepts—like ingestion, transformation, and analytics—from the eyes of the Google Cloud Platform. Bhavani first goes over data engineering foundations, then dives into Google Cloud data storage concepts, explaining the various types of Google Cloud Storage options and when to utilize each. Then, learn about Google Cloud data pipelines—their respective use cases, and why you would use data pipelines in data engineering workflows. If you’re looking to learn more about the spectrum of data engineering—and how to address data problems by identifying the right Google tool for the job—join Bhavani in this course.
Skills covered
Google CloudData EngineeringSoftware Development ToolsGoogleFoundationsCloud PlatformsCloud ComputingData ScienceSoftware Development
Concepts
0. Introduction
- 01 - Data engineering on Google Cloud introduction
- 02 - A quick checklist
1. Data Engineering Foundations
- 03 - What is data engineering
- 04 - Data engineering ecosystem
- 05 - Overview of tools and terms
- 06 - What's in the course
- 07 - Repo set up
2. Google Cloud Data Storage
- 08 - Data storage options
- 09 - Google Cloud Storage (GCS)
- 10 - GCS with Python
- 11 - BigQuery
- 12 - BigQuery with Python
3. Google Cloud Data Pipelines
- 13 - Different kinds of data pipelines and their use cases
- 14 - Dataproc
- 15 - PySpark on Dataproc
- 16 - Google Cloud Pub Sub
- 17 - Cloud Dataflow
- 18 - What is Cloud Composer
- 19 - Writing an Airflow Pipeline
Conclusion
- 20 - Next steps