The ETL Process: Introduction to Transforming data

Description

This session introduces common data transformations used to clean, standardize, reshape, and prepare data for analysis. Transformation is important because analytical results depend on converting raw observations into consistent variables and structures that accurately represent the underlying business activity.

Motivation

How do we make messy data usable?

Review

The preceding sessions focused on acquiring data and understanding relationships among datasets. This session moves into the transformation stage of ETL by considering how extracted and combined data must be changed before they can provide useful analytical evidence.

Class plan

• We will briefly review the ETL material we are covering and where we are at.
• We will quickly wrap-up some of the SQL material from last class, and check-in with the SQL material we are working through online.
• There are two data exercises for this class, they are based on transforming messier data, using approaches in different software.

Team format

Discussion Teams

Materials

• Slides

  Class 05 Slides

  View slides: Class 05 Slides

Tools

• PowerBI
• VSCode with SQLite extension for SQL automation

Concepts

• Data cleaning and transformation
• Automation of data cleaning

Data

Use a git pull to extract the case data for this class.