Complete Guide to Parallel and Concurrent Programming in Python

Course details

Parallel programming unlocks a program’s ability to execute multiple instructions simultaneously, increases the overall processing throughput, and is key to writing faster and more efficient applications. In this course, join instructors Barron and Olivia Chiu Stone as they introduce the basics of parallel programming in Python, providing the foundational knowledge you need to write more efficient, performant code. Barron and Olivia explain concepts like threading and mutual exclusion in a fun and informative way, relating them to everyday activities you perform in the kitchen. To cement the ideas, they demo them in action using Python. Each lesson is short and practical, driving home the theory with hands-on techniques.

Concepts

Introduction

Learn parallel programming basics
What you should know
Exercise files

Parallel Computing Hardware

Sequential vs. parallel computing
Parallel computing architectures
Shared vs. distributed memory

Threads and Processes

Thread vs. process
Concurrent vs. parallel execution
Global interpreter lock - Python demo
Multiple threads - Python demo
Multiple processes - Python demo
Execution scheduling
Execution scheduling - Python demo
Thread lifecycle
Thread lifecycle - Python demo
Daemon thread
Daemon thread - Python demo

Mutual Exclusion

Data race
Data race - Python demo
Mutual exclusion
Mutual exclusion - Python demo

Locks

Reentrant lock
RLock - Python demo
Try lock
Non-blocking acquire - Python demo
Read-write lock
Read-write lock - Python demo

Liveness

Deadlock
Deadlock - Python demo
Abandoned lock
Abandoned lock - Python demo
Starvation
Starvation - Python demo
Livelock
Livelock - Python demo

Synchronization

Condition variable
Condition variable - Python demo
Producer-consumer
Producer-consumer threads - Python demo
Producer-consumer processes - Python demo
Semaphore
Semaphore - Python demo

Barriers

Race condition
Race condition - Python demo
Barrier
Barrier - Python demo

Asynchronous Tasks

Computational graph
Thread pool
Thread pool - Python demo
Process pool - Python demo
Future
Future - Python demo
Divide and conquer
Divide and conquer - Python demo

Evaluating Parallel Performance

Speedup, latency, and throughput
Amdahl's law
Measure speedup
Measure speedup - Python demo

Designing Parallel Programs

Partitioning
Communication
Agglomeration
Mapping

Challenge Problems

Welcome to the challenges
Challenge - Matrix multiply in Python
Solution - Matrix multiply in Python
Challenge - Merge sort in Python
Solution - Merge sort in Python
Challenge - Download images in Python
Solution - Download images in Python

Conclusion

Additional resources
Next steps