String Cleaning with pandas 2.0

String Cleaning with pandas 2.0

1h 2mAdvanced2023-10-10

Authors

Seth Berry

Seth Berry

Associate Teaching Professor and MSBA Academic Co-Director at the University of Notre Dame

Course details

Cleaning strings is no easy task, but it’s important, necessary work. From basic find-and-replace operations to more complex regular expressions, this course provides an overview of how to clean strings with pandas 2.0, the latest release of the open-source library for the Python programming language.

Gather insights from University of Notre Dame professor Seth Berry as he explores the fundamentals of string cleaning using pandas expressions, functions, operations, and methods, including PyArrow backends, regular expressions to find and replace text for complex patterns, speed tests, text extractions, match and split functions, quantifiers, backreferences, lookarounds, boundaries, chains, and more. By the end of this course, you’ll be ready to leverage a wide variety of pandas solutions to start cleaning strings on your own.

Skills covered

pandasData AnalysisData ScienceBusiness Analysis and StrategyBusiness Software and ToolsOpen SourceOne-Off

Concepts

Introduction

  • Cleaning strings with pandas
  • What you need to know

Foundations for String Cleaning

  • Using the PyArrow backend
  • Testing speed
  • Removing special characters
  • Matching patterns
  • Extracting patterns
  • Counting strings
  • Splitting strings
  • Creating new columns from text

Taking Action in pandas

  • Finding and replacing patterns
  • Using quantifiers
  • Reordering strings with backreferences
  • Naming backreference groups
  • Extracting strings with lookarounds
  • Using boundaries and sets
  • Chaining pandas string methods
  • Regular expression golf

Conclusion

  • Advancing regular expressions
40,000 Toman