COMP328 Fundamentals of Machine Learning

Module 3 of 6

Classical Machine Learning Algorithms

Three algorithms that still dominate applied machine learning in industry: decision trees, random forests, and support vector machines. Each takes a different route to the same goal — drawing a boundary between classes — and each makes a different trade-off between interpretability, accuracy, and computational cost.

3 units 27 tutorials 3 unit quizzes 1 assignment

Module at a glance

Units 3 — trees, forests, and margin methods
Tutorials 27 — nine per unit
Quizzes 3 — one at the end of each unit
Assignment 1 — Classification Analysis
Study time ~20 hours plus assignment work

Learning outcomes

What you will be able to do

By the end of Module 3 you will be able to build, tune, and compare three fundamentally different classification algorithms — and justify which one you would deploy for a given problem.

By the end of this module you will be able to

  • Explain how a decision tree recursively partitions feature space, and interpret the resulting tree as a readable set of decision rules.
  • Compute entropy and information gain, and explain why a tree chooses one split over another.
  • Diagnose overfitting in a decision tree and apply pruning and depth limits to control it.
  • Describe ensemble learning and explain how bagging reduces variance without increasing bias.
  • Interpret feature importance from a random forest, and recognise its limitations as an explanation of causality.
  • Explain the maximum-margin principle, identify support vectors, and describe the role of the C parameter in controlling the margin.
  • Apply the kernel trick to classify non-linearly separable data, and choose between linear, polynomial, and RBF kernels.
  • Select an appropriate algorithm for a given problem based on dataset size, dimensionality, interpretability requirements, and expected accuracy.

Three algorithms, one comparison

These three methods are not a sequence to work through and forget. Module 3 builds toward a single question: given a new classification problem, which of these would you reach for, and why? The tutorials return to that question repeatedly.

Prerequisite reminder

This module assumes fluency with the evaluation vocabulary from Module 2, Unit 6 — confusion matrices, precision, recall, F1-score, ROC/AUC, and cross-validation. Every algorithm here is judged with those tools, and the assignments expect you to use them without prompting.

Units & tutorials

Three units, twenty-seven tutorials

Unit 7 builds a single tree and shows you exactly how it decides. Unit 8 turns many trees into a forest. Unit 9 abandons trees entirely and takes a geometric approach to the same problem.

Unit 7 9 tutorials · ~7 hours

Decision Trees

The most interpretable model in machine learning. Learn how a tree partitions feature space, how it chooses each split, and why it overfits so readily.

Unit overview Start T-01
Unit 9 9 tutorials · ~6 hours

Support Vector Machines

A geometric approach to classification. Find the widest possible street between the classes — then bend the space when a straight line will not do.

Unit overview Start T-01

Datasets

What you will work with

Each unit uses a dataset chosen to make that algorithm’s behaviour visible. By the end of the module you will have applied all three algorithms to overlapping problems, so you can compare them directly.

Iris Species Unit 7 · Decision Trees
A small, visual dataset for seeing exactly where a tree draws its splits.
Titanic Survival Units 7 & 8 · Trees and Forests
Mixed categorical and numeric features; a natural bridge from one tree to many.
Credit Card Default Unit 8 · Random Forests
A class-imbalanced dataset where feature importance and OOB error matter.
Breast Cancer Diagnosis Unit 9 · Support Vector Machines
Thirty numeric features and a clear binary outcome — ideal for margin methods.

How to work through it

Build it, break it, then compare it

The value of Module 3 is not in learning three algorithms in isolation. It is in learning how they differ, and in developing the judgement to choose between them.

  1. Build the model by hand first

    Before reaching for Scikit-Learn, work through the arithmetic on a tiny example. Entropy, information gain, and margins only stick when you have computed them once yourself.

  2. Push it until it fails

    Deliberately overfit a tree. Give an SVM a linear kernel on data that needs a curve. Watching a model break is faster than reading about why it might.

  3. Compare on the same data

    Whenever a new algorithm is introduced, apply it to a dataset you already used in an earlier unit. The comparison is where the learning is.

  4. Finish with the unit quiz

    Each unit ends with a short quiz. Score it honestly before looking at the feedback.

Assessment

How this module is assessed

Module 3 carries three unit quizzes and the classification assignment that draws directly on all three algorithms.

  • Quiz 7 Decision Trees — 15 questions, counts toward your quiz grade Unit 7
  • Quiz 8 Random Forests — 15 questions, counts toward your quiz grade Unit 8
  • Quiz 9 Support Vector Machines — 15 questions, counts toward your quiz grade Unit 9
  • Assignment 3 Classification Analysis Open

Assignment 3 is released during this module

Assignment 3: Classification Analysis draws on all three algorithms in this module and asks you to compare them on a single dataset. It is released while you are working through Unit 8, and its deadline falls in week 9 — just after this module ends. Start it as soon as it is released rather than leaving it until Module 4 begins.

Looking ahead

Module 4 marks a fundamental shift. Everything so far has assumed you have labelled data to learn from. From Unit 10 onward, you will work with datasets that have no labels at all — which means the entire evaluation toolkit you built in Module 2 needs to be rethought.

👤