Azure Spark Databricks Essential Training

Course details

Apache Spark and Microsoft Azure are two of the most in-demand platforms and technology sets in use by today's data science teams. These two platforms join forces in Azure Databricks‚ an Apache Spark-based analytics platform designed to make the work of data analytics easier and more collaborative. In this course, Lynn Langit digs into patterns, tools, and best practices that can help developers and DevOps specialists use Azure Databricks to efficiently build big data solutions on Apache Spark. Lynn covers how to set up clusters and use Azure Databricks notebooks, jobs, and services to implement big data workloads. She also explores data pipelines with Azure Databricks—including how to use ML Pipelines—as well as architectural patterns for machine learning.

Learning objectives
Business scenarios for Apache Spark
Setting up a cluster
Using Python, R, and Scala notebooks
Scaling Azure Databricks workflows
Data pipelines with Azure Databricks
Machine learning architectures
Using Azure Databricks for data warehousing

Concepts

0. Introduction

01 - Optimize data pipelines
02 - What you should know
03 - About using cloud services

1. Big Data on Azure Databricks

04 - Meet Databricks Apache Spark clusters
05 - Business scenarios for Spark
06 - Understand Spark key components
07 - Azure Databricks concepts
08 - Quick start - Use a notebook
09 - Set up Databricks AI Playground
10 - Use Databricks AI Playground

2. Core Azure Databricks Workloads

11 - Review Databricks Azure cluster setup
12 - Use a Python notebook with dashboards
13 - Use an R notebook
14 - Use a Scala notebook for visualization
15 - Use a notebook with scikit-learn
16 - Use a Spark Streaming notebook
17 - Use an external Scala library - variant-spark

3. Scaling Azure Databricks Workloads

18 - Understand data engineering workload steps
19 - Understand cluster configurations
20 - Understand Spark job execution overhead
21 - Explore optimization control planes
22 - Optimize a cluster and job
23 - Run a production-size job

4. Data Pipelines with Azure Databricks

24 - Use Databricks jobs and role-based control
25 - Use Databricks Runtime ML
26 - Understand ML Pipelines API
27 - Use ML Pipelines API
28 - Use distributed ML training
29 - Understand Databricks Delta
30 - Use Databricks Delta
31 - Use Azure Blob storage
32 - Understand MLflow

5. Machine Learning Architectures

33 - Azure Databricks pipeline considerations
34 - Azure Databricks for data warehousing
35 - Azure Databricks and machine learning
36 - Azure Databricks for churn analysis
37 - Azure Databricks for intrusion detection

Conclusion

38 - Next steps