Large Language Models: Text Classification for NLP using BERT
55mAdvanced2022-06-03
Authors
Jonathan Fernandes
Consultant focusing on data science, AI, and big data
Course details
Transformers are taking the natural language processing (NLP) world by storm. In this course, instructor Jonathan Fernandes teaches you all about this go-to architecture for NLP and computer vision tasks and must-have skill in your Artificial Intelligence toolkit. Jonathan uses a hands-on approach to show you the basics of working with transformers in NLP and production. He goes over BERT model sizes, bias in BERT, and how BERT was trained. Jonathan explores transfer learning, shows you how to use the BERT model and tokenization, and covers text classification. After thoroughly explaining the transformer model architecture, he finishes up with some additional training runs.
Skills covered
Natural Language Processing (NLP)Generative AIPythonArtificial Intelligence (AI)Open SourceDeep Dive (X:Y)
Concepts
0. Introduction
- 01 - Natural language processing with transformers
- 02 - How to use the exercise files
1. NLP and Transformers
- 03 - How transformers are used in NLP
- 04 - Transformers in production
- 05 - Transformers history
- 06 - Challenge - BERT model sizes
- 07 - Solution - BERT model sizes
2. BERT and Transfer Learning
- 08 - Bias in BERT
- 09 - How was BERT trained
- 10 - Transfer learning
3. Transformer Architecture and BERT
- 11 - Transformer - Architecture overview
- 12 - BERT model and tokenization
- 13 - Positional encodings and segment embeddings
- 14 - Tokenizers
- 15 - Self-attention
- 16 - Multi-head attention and feedforward network
4. Text Classification
- 17 - BERT and text classification
- 18 - The Datasets library
- 19 - Overview of IMDb dataset
- 20 - Using a tokenizer
- 21 - Tiny IMDb
- 22 - A training run
Conclusion
- 23 - Additional training runs