Unit 10 — Clustering | COMP328: Fundamentals of Machine Learning

01 Learning outcomes

By the end of Unit 10 you should be able to do the following. Each outcome is assessed somewhere — the mapping is given in Section 05.

Knowledge — explain

Define unsupervised learning and distinguish it from supervised and reinforcement learning.
Describe the difference between partitional, hierarchical, and density-based clustering paradigms.
Explain how the choice of distance metric changes the clusters an algorithm discovers.
State the objective function that k-means minimises, and explain why it converges.
Distinguish internal from external cluster-validity measures.

Skills — apply

Scale and prepare feature matrices for distance-based methods.
Implement and tune k-means and agglomerative clustering with scikit-learn.
Select k using the elbow method and silhouette analysis.
Read and interpret a dendrogram.
Quantify cluster quality with silhouette, Davies–Bouldin, and adjusted Rand scores.
Communicate a segmentation result to a non-technical audience.

Professional competencies — evaluate

Recognise when clustering is the wrong tool for a question.
Identify how clustering can encode or amplify bias in grouping people.
Report uncertainty honestly — a cluster label is a hypothesis, not a fact.
Document clustering decisions so that another analyst can reproduce them.

02 Why clustering matters

Labelled data is expensive. Someone has to sit down and tag every example, and for many problems no correct label exists in the first place — what is the "right" customer segment, or the "correct" grouping of genes? Clustering is how machine learning answers questions when the answer is not given to you in advance.

The idea is simple to state and surprisingly hard to do well: find groups of observations such that points in the same group are more similar to each other than to points in other groups. The difficulty is entirely in the details — what "similar" means, how many groups there are, and whether the groups you found are real or merely an artefact of the algorithm you chose.

Three visually obvious clusters — plus one point that belongs to none of them. Which algorithm you choose determines whether that point becomes its own cluster, gets absorbed into a neighbour, or is labelled noise.

Common misconception

Clustering always returns clusters. It never tells you that there are no groups in your data. If you run k-means with k=5 on pure random noise, you will get five clusters back — and they will be meaningless. Validating your result is not an optional extra step; it is the step that separates analysis from decoration.

03 Reading roadmap

The nine tutorials below are designed to be read in order. The first three build the conceptual and mathematical foundation; the middle three cover the three main algorithm families; the last three deal with evaluation, implementation, and a full worked case study.

T-01

What unsupervised learning is

The three learning paradigms revisited, and the questions clustering can and cannot answer.

T-02

Distance and similarity

Euclidean, Manhattan, cosine and Jaccard distance; feature scaling; the curse of dimensionality previewed.

T-03

K-means

The assignment–update loop, the within-cluster sum of squares objective, convergence and its limitations.

T-04

Choosing k

The elbow method, silhouette analysis, gap statistic, and why there is rarely one right answer.

T-05

Hierarchical clustering

Agglomerative merging, linkage criteria, dendrograms, and cutting the tree at the right height.

T-06

DBSCAN and density-based clustering

Core points, ε-neighbourhoods, arbitrary shapes, and principled noise handling.

T-07

Evaluating cluster quality

Silhouette, Davies–Bouldin, Calinski–Harabasz, and external measures such as adjusted Rand index.

T-08

Clustering in scikit-learn

A complete implementation workflow: pipelines, scaling, fitting, and visualising results.

T-09

Case study: customer segmentation

End-to-end analysis of a retail dataset, from feature engineering to a stakeholder-ready report.

Study tip

Tutorials T-03, T-04 and T-05 reward slow reading. Work through each step of the k-means loop and each merge in the dendrogram by hand on a five-point dataset before you touch code. The algorithms are short; the intuition is what takes time.

04 Learning activities

Each tutorial contains a worked example and a self-check. The activities below are the ones that require you to write and run code.

Activity 10.1 — Reproduce the k-means loop

Using the five-point dataset from T-03, run two iterations of k-means by hand with k=2 and initial centroids at (1, 1) and (4, 4). Record the assignment of each point and the new centroid positions at every step. Then run KMeans(n_init=1, random_state=0) and compare.

Activity 10.2 — Distance matters

Take the wine dataset from scikit-learn. Cluster it three times with k-means: once unscaled with Euclidean distance, once standardised, and once using cosine distance via normalize() plus k-means. Plot all three silhouette scores. Explain in three sentences why the results differ.

Activity 10.3 — Read a dendrogram

Run agglomerative clustering on the iris dataset with linkage='ward', 'complete', and 'single'. Render all three dendrograms. Identify which linkage produces the most balanced tree and which produces the longest chain.

unit-10-clustering.ipynb Notebook containing starter code, datasets, and the plotting helpers used across T-02 to T-09. Open workflow

05 Assessment activities

10%

Unit 10 quiz

12 questions covering distance metrics, k-means mechanics, dendrogram interpretation, and validity measures. Unlimited attempts; highest score counts.

15%

Assignment 4

Unsupervised learning. You will cluster a supplied dataset, justify your choice of k, and defend your clusters against a sceptical reviewer.

—

Self-checks

Embedded in every tutorial. Ungraded, but they are the fastest way to find out whether you have actually understood the previous section.

Outcome-to-assessment mapping

Where each learning outcome is assessed
Outcome	Quiz	Assignment 4	Capstone
Explain clustering paradigms	Q1–Q4	Report §1	Methodology
Select and justify distance metric	Q5–Q6	Report §2	Methodology
Apply k-means and hierarchical clustering	Q7–Q9	Code + results	Model building
Evaluate cluster quality	Q10–Q12	Report §4	Evaluation
Communicate findings responsibly	—	Report §5	Ethics analysis

Quiz 10 Clustering fundamentals — 12 questions, 30 minutes Revise

Assignment 4 Unsupervised Learning: clustering, dimensionality reduction, and association rules Open brief

06 Key terms

Unsupervised learning: Learning structure from data that has no labelled target variable.
Clustering: Partitioning observations into groups such that within-group similarity is high and between-group similarity is low.
Centroid: The mean position of all points assigned to a cluster; the representative point used by k-means.
Within-cluster sum of squares (WCSS): The total squared distance from each point to its cluster centroid — the quantity k-means minimises.
Dendrogram: A tree diagram recording the order and distance at which clusters were merged in hierarchical clustering.
Linkage: The rule that defines the distance between two clusters: single, complete, average, or Ward.
Silhouette coefficient: A per-point score in [−1, 1] comparing how close a point is to its own cluster versus the nearest other cluster.
Core point: In DBSCAN, a point with at least min_samples neighbours within radius ε.
Noise point: A point that is neither a core point nor reachable from one; labelled −1 by DBSCAN.

07 Summary

What you should take away

Clustering is exploratory, not predictive. It generates hypotheses about structure; it does not confirm them.
Every clustering result depends on three choices you make before any algorithm runs: the features, the distance metric, and the number or scale of clusters. Change any one and the answer changes.
K-means is fast and simple but assumes roughly spherical, similarly sized clusters and forces every point into one.
Hierarchical clustering gives you a full nested structure instead of a single partition, at a higher computational cost.
DBSCAN finds arbitrarily shaped clusters and explicitly labels noise, but is sensitive to its ε and min_samples parameters.
Always scale your features before using a distance-based method, and always validate your clusters with more than one measure.

Ready to move on?

Once you are comfortable with the tutorials and have scored at least 80% on the unit quiz, continue to Unit 11 — Dimensionality Reduction, where you will learn how to reduce the number of features before clustering, and how to visualise high-dimensional cluster structure in two dimensions.

08 Frequently asked questions

Do I need to know linear algebra for this unit?

Only lightly. You need to be comfortable with the idea of a point in n-dimensional space and with Computing a distance between two points. Everything else — centroids, WCSS, linkage — is built from those two ideas. The mathematics is developed from scratch in T-02 and T-03.

Which algorithm should I use for my assignment?

There is no single correct answer, and choosing badly is not penalised — failing to justify your choice is. A reasonable default workflow is:

Scale the features.
Try k-means across a range of k and plot the silhouette score.
Run hierarchical clustering with Ward linkage and inspect the dendrogram.
If you suspect non-spherical clusters or outliers, run DBSCAN and compare.
Report which method you preferred and why, including what the others got wrong.

My silhouette score is 0.28. Is that good?

It is not automatically bad. Silhouette values are dataset-dependent and have no universal threshold. What matters is the comparison: is 0.28 better than the 0.24 you got with k=3 and the 0.19 you got with k=6? Also inspect the per-cluster silhouette plot — a respectable average can hide one cluster with negative scores, which usually means that cluster should not exist.

Is clustering used in industry, or is it just academic?

Heavily used. Customer segmentation, document grouping, image compression, anomaly detection, biological taxonomy, recommender pre-processing, and log analysis all rely on clustering. The case study in T-09 is modelled on a real retail segmentation workflow.