Spark for Machine Learning & AI

Course details

Apache Spark is one of the most widely used and supported open-source tools for machine learning and big data. In this course, discover how to work with this powerful platform for machine learning. Instructor Dan Sullivan discusses MLlib—the Spark machine learning library—which provides tools for data scientists and analysts who would rather find solutions to business problems than code, test, and maintain their own machine learning libraries. He shows how to use DataFrames to organize data structure, and he covers data preparation and the most commonly used types of machine learning algorithms: clustering, classification, regression, and recommendations. By the end of the course, you will have experience loading data into Spark, preprocessing data as needed to apply MLlib algorithms, and applying those algorithms to a variety of machine learning problems.

Topics include:
Machine learning workflows
Organizing data in DataFrames
Preprocessing and data preparation steps for machine learning
Clustering data
Classification algorithms
Regression methods available in Spark MLlib
Common approaches to designing recommendation systems

Concepts

Introduction

Welcome

Introduction to Spark and MLlib

Introduction to Spark
Steps in the machine learning process
Install Spark
Organizing data in DataFrames
Components of Spark MLlib

Data Preparation and Transformation

Introduction to preprocessing
Normalize numeric data
Standardize numeric data
Bucketize numeric data
Tokenize text data
TF-IDF
Summary of preprocessing

Clustering

Introduction to clustering
K-means clustering
Hierarchical clustering
Summary of clustering techniques

Classification

Introduction to classification
Preprocessing the Iris data set
Naive Bayes classification
Multilayer perceptron classification
Decision trees classification
Summary of classification algorithms

Regression

Introduction to regresssion
Preprocessing regression data
Linear regression
Decision tree regression
Gradient-boosted tree regression
Summary of regression algorithms

Recommendations

Understand recommendation systems
Collaborative filtering

Conclusion

Tips for using Spark MLlib