Class 7
The ETL Process: Regular Expressions
Description
This session introduces regular expressions as a method for identifying, extracting, and validating patterns within text. Regular expressions are important because many business datasets contain useful information embedded within identifiers, descriptions, documents, and other fields that are not consistently structured.
Motivation
How do we extract structure from unstructured information?
Review
Previous sessions focused primarily on transforming information that already has recognizable rows, columns, and variables. This session begins the transition toward unstructured data by showing how patterns can be used to impose structure on textual information.
Class plan
- We will start with a brief review and opening polleverywhere question on regex (based on the Regex EYARC Experience Regex fundamentals primer)
- We will undertake an exercise using regex101.com to understand some of the fundamentals of using regular expressions
- We will then move to Google Colab to run regular expressions using python through the Enron email dataset.
Team format
Discussion Teams
Materials
-
Class 07 Slides
View slides: Class 07 Slides
Links
Tools
- regular expressions
- python
Concepts
- structuring unstructured text data