Python for Data Engineering: from Beginner to Advanced

Course details

Get up and running with the basics of Python before progressing to more advanced topics specific to data engineering. In this hands-on, interactive course, join instructor Deepak Goyal to practice performing a wide range of data engineering tasks in Python to boost your technical know-how, prepare for an interview, or land a new role. This course includes Code Challenges powered by CoderPad. Code Challenges are interactive coding exercises with real-time feedback, so you can get hands-on coding practice to advance your coding skills. Deepak helps you boost your skills as a Python programmer with six specific coding challenges. Explore language basics, Python collections, file handling, Pandas, NumPy, OOP, and advanced data engineering tools that use Python. The course ends with a capstone project focused on retail sales analysis.

Concepts

Introduction

Welcome to the course
What you should know
CoderPad tour

Python Basics

Introduction to Python and data engineering
Setting up your Python environment
Explore a Google Colab worksheet
Variables and data types
Operators and expressions
Control structures
Functions
Modules and packages
String manipulation
Error handling
Solution - Conditions

Python Collections

Collection overview
Python collections - Tuples
Python collections - Lists
Python collections - Sets
Python collections - Dictionaries
Solution - Collections

Python File Handling

File I O overview
Working with CSV files
Working with JSON files
Solution - File handling

pandas DataFrame API

Introduction to pandas
Read files as DataFrames
Data cleaning and preprocessing
Data manipulation and aggregation
Data visualization
Write DataFrames as files
Solution - pandas

NumPy

Introduction to NumPy
Array creation and attributes
Array operations
Indexing and slicing
Linear algebra and statistics
Write DataFrames as files
Solution - NumPy

OOP with Python

Understanding classes and objects
Implementation - Classes and objects in Python
Understand OOP features - Abstraction, inheritance, and more
Solution - OOP

Advanced Data Engineering

Tips to write efficient Python code
What is ETL in the data engineering world
What is Hadoop
Understand PySpark for data engineering
Importance of visualization tools in DE
On-prem vs. cloud data engineering

Capstone Project

Capstone project - Retail sales analysis
Solution - Capstone project

Conclusion

Next steps