Module 4 of 6
Every module so far has assumed you have labels to learn from. Module 4 removes that assumption entirely. Here you will find structure in data without being told what to look for — grouping, compressing, associating, and flagging the unusual — using methods that underpin much of modern data analysis and signal detection.
Learning outcomes
By the end of Module 4 you will be able to explore an unlabelled dataset, identify its internal structure using four different techniques, and judge whether the structure you found is real or an artefact of your method.
Unsupervised learning removes the two things you have relied on so far: labels and accuracy. You will not be told the right answer, and no metric will tell you definitively whether you found it. The habits this module builds — sceptical interpretation, multiple methods, honest visualisation — are exactly what working with unlabelled data demands.
This module assumes you are comfortable with the Python data stack from Module 1, Unit 3 — NumPy, Pandas, Matplotlib, and the Scikit-Learn estimator API. You will be using all four heavily, and increasingly without explicit instruction on how they work.
Units & tutorials
Module 4 covers four distinct unsupervised tasks. Units 10 and 11 are the core — grouping and compressing. Units 12 and 13 apply unsupervised ideas to two specific, practically important problems.
Group data points by similarity, without being told what the groups are. K-Means, hierarchical clustering, and the hard question of how many clusters to keep.
Too many features makes everything harder. Learn to compress high-dimensional data into fewer, more informative dimensions — and to visualise it honestly.
Finding things that occur together. Market basket analysis, the Apriori algorithm, and the rules that power recommendation systems.
Finding the rare, the unusual, and the suspicious. Outlier detection, fraud detection, and intrusion monitoring — where the interesting cases are always the ones you have fewest examples of.
Datasets
Each unit uses a dataset that makes the specific unsupervised task concrete. Units 10 and 11 deliberately share a dataset, so you can cluster it and then visualise the same clusters via PCA.
How to work through it
Unsupervised learning has no test set to tell you whether you got the right answer. The habits this module rewards are different from anything earlier in the course.
Before clustering or reducing dimensions, visualise the raw data. Patterns your eye can see are patterns worth understanding before an algorithm rediscovers them.
No single unsupervised method tells the whole story. If K-Means and hierarchical clustering disagree on the same data, that disagreement is itself information.
Unsupervised methods will always produce clusters. Always produce components. Always produce rules. The question is never "did it work?" but "is this structure real, or did the method impose it?"
Each unit ends with a short quiz. Score it honestly before looking at the feedback.
Assessment
Module 4 carries four unit quizzes — the most of any module — and the unsupervised learning assignment.
Assignment 4: Unsupervised Learning Techniques asks you to apply at least two of this module's four techniques to a single dataset of your choosing and to interpret the results without the benefit of ground-truth labels. Its deadline falls in week 10, roughly the midpoint of Module 4. Plan to begin it as soon as you have completed Unit 11, rather than waiting until the end of the module.
Module 5 returns to supervised learning — but with a dramatic increase in model capacity. Everything you have built so far has been limited to models with a few dozen parameters. Neural networks have thousands, sometimes millions. The evaluation tools from Module 2 carry over unchanged; the intuitions about how models behave need rebuilding.