Processing Text with R Essential Training

Processing Text with R Essential Training

56mIntermediate2019-09-19

Authors

Kumaran Ponnambalam

Kumaran Ponnambalam

Working with data for 20+ years

Course details

Today’s big data and analytics pipelines are consuming more and more text data generated through websites, social media, and private communications. But deriving insights from text isn't straightforward; it requires a series of techniques and forms for preparing text for analytics and machine learning. In this course, learn the essential techniques for cleansing and processing text in R, and discover how to convert text to a form that's ready for analytics and predictions. Kumaran Ponnambalam begins by reviewing techniques for extracting, cleansing, and processing text. He then shows how to convert text into an analytics-ready form, including how to use n-grams and TF-IDF. Throughout the course, he provides examples for exercising these techniques using the R and tm libraries.

Learning objectives
Acquiring text from various sources
Cleansing and transforming text data
Preparing TF-IDF matrices for machine learning
Building n-grams databases for text predictions
Best practices for scalability and storing text

Skills covered

RStatisticsEssential TrainingProgramming LanguagesData ScienceOpen SourceSoftware Development

Concepts

Introduction

  • The emergence of text analytics

Introduction to Text Mining

  • Purpose
  • Document
  • Corpus
  • R text processing libraries
  • Setting up the environment

Corpus in R

  • PCorpus and VCorpus
  • Reading files with CorpusReader
  • Exploring the corpus
  • Persisting the corpus

Text Cleansing and Extraction

  • Setup for processing
  • Cleansing text
  • Stop word removal
  • Stemming
  • Managing metadata

TF-IDF

  • Introduction to tf-idf
  • Generating term frequency matrix
  • Improving term frequency matrix
  • Plotting term frequency
  • Generating tf-idf

N-Grams

  • N-grams concepts
  • Using RWeka NGramTokenizer
  • Creating an n-gram text frequency matrix
  • Extracting n-gram pairs

Best Practices

  • Storing text
  • Processing text data
  • Scalability

Conclusion

  • Next steps
40,000 Toman