Deep Learning: Model Optimization and Tuning

Course details

Deep learning as a technology has grown leaps and bounds in the last few years. More and more AI solutions use deep learning as their foundational technology. Studying this technology, however, presents several challenges. IT professionals from varying backgrounds need a simplified resource to learn the concepts and build models quickly. In this course, instructor Kumaran Ponnambalam provides a simplified path to understand various optimization and tuning options available for deep learning models and shows you how to use these options to improve models. He begins by reviewing Deep Learning, including artificial neural networks and architectures. Next, Kumaran discusses the process of hyper parameter tuning. He examines the building blocks of neural networks and the levers available to tune them. Kumaran offers recommendations and best practices. Then he concludes with an end-to-end tuning exercise.

Learning objectives
Understand the architecture of deep learning including artificial neural networks.
Examine the building blocks of neural networks and the levers available to tune them.
Learn best practices and the workflow for end-to-end fine tuning.

Concepts

Introduction

Optimizing neural networks
Setting up exercise files

Introduction to Deep Learning Optimization

Review of artificial neural networks
An artificial neural network (ANN) model
Model optimization and tuning
The deep learning tuning process
Experiment setups for the course

Tuning the Deep Learning Network

Epoch and batch size tuning
Epoch and batch size experiment
Hidden layers tuning
Determining nodes in a layer
Choosing activation functions
Initializing weights

Tuning Back Propagation

Vanishing and exploding gradients
Batch normalization
Optimizers
Optimizer experiment
Learning rate
Learning rate experiment

Overfitting Management

Overfitting in ANNs
Regularization
Regularization experiment
Dropouts
Dropout experiment

Model Tuning Exercise

Tuning exercise - Problem statement
Acquire and process data
Tuning the network
Tuning backpropagation
Avoiding overfitting
Building the final model