COMP3260Data Science Foundations

Module 2 · Unit 6 · 8 tutorials · Week 6

Data Transformation

Normalisation, standardisation, encoding, aggregation and feature creation. Shaping clean data into the form your analysis requires.

Start t-01 Module overview

Overview

A cleaned dataset is not yet an analysis-ready dataset. Values may be on incompatible scales, categorical variables must be encoded, individual records may need aggregating into meaningful summaries, and sometimes the right variable does not exist yet and must be engineered from the ones that do.

This unit covers each of these transformations and the judgements that go with them — which scaling method is appropriate, whether to use one-hot or ordinal encoding, and how to construct a feature that actually captures what you intend.

Learning outcomes

By the end of this unit you will be able to

  • Normalise and standardise numeric features appropriately.
  • Encode categorical variables using suitable techniques.
  • Aggregate and group data to produce summaries.
  • Engineer new features that capture domain-relevant information.
  • Document transformation choices and their rationale.

Lesson sequence

t-01

Transformation Overview

Why transformation matters and where it sits in the pipeline.

 t-02

Normalization

Rescaling values to a fixed range, and when it helps.

 t-03

Standardization

Z-scores, centring and scaling to unit variance.

 t-04

Encoding Categorical Data

One-hot, ordinal, target and binary encoding, and their trade-offs.

 t-05

Aggregation and Grouping

Group-by operations, pivot tables and summary statistics.

 t-06

Feature Creation

Deriving new variables from existing ones: ratios, bins, dates.

 t-07

Practical Lab

A full transformation pipeline on a real dataset.

 t-08

Unit Summary

Consolidated key points and a forward look to Module 3.

Key concepts

Normalisation
Rescaling values onto a fixed range, typically [0, 1], so that features measured on different scales contribute comparably.
Standardisation
Transforming values to have mean zero and standard deviation one, preserving the shape of the distribution.
One-hot encoding
Representing a categorical variable as a set of binary columns, one per category.
Feature engineering
Creating new variables from existing ones in order to expose patterns more clearly to analysis.

Assessment alignment

Prepares Assignment 2 and the capstone

The transformation decisions you make in Assignment 2 (Data Cleaning and Preparation) are assessed on justification as well as implementation. The capstone requires the same discipline applied end to end.

👤