COMP328 Fundamentals of Machine Learning

Module 5 of 6

Neural Networks and Deep Learning

The most demanding module in the course. Everything before this has involved models with a few dozen parameters. Neural networks have thousands, sometimes millions — and that jump in capacity changes how you build, train, and reason about models. You will start with a single artificial neuron and finish with convolutional architectures that recognise images.

3 units 27 tutorials 3 unit quizzes 1 assignment

Module at a glance

Units 3 — neurons, deep networks, convolutions
Tutorials 27 — nine per unit
Quizzes 3 — one at the end of each unit
Assignment 1 — Neural Network Modeling
Study time ~24 hours plus assignment work

Learning outcomes

What you will be able to do

By the end of Module 5 you will be able to build, train, and evaluate neural networks of your own — including convolutional networks for image data — and to diagnose why a network that should be learning is not.

By the end of this module you will be able to

  • Describe the artificial neuron as a weighted sum followed by a non-linear activation, and explain why the non-linearity is essential.
  • Explain backpropagation at a conceptual level: how a network attributes error to individual weights and adjusts them accordingly.
  • Choose appropriate activation functions for hidden and output layers, and explain the trade-offs between sigmoid, tanh, ReLU, and softmax.
  • Build and train a multi-layer neural network in Keras, and interpret the loss curves that result.
  • Diagnose common training failures — underfitting, overfitting, vanishing gradients, and unstable learning rates — and apply the appropriate remedy.
  • Apply regularisation techniques including dropout and early stopping to control overfitting in deep networks.
  • Explain how convolution extracts local features from images, and how pooling reduces spatial dimensions while preserving information.
  • Build a convolutional neural network for an image classification task, and interpret its learned feature maps.

This module is a step up

Module 5 is the most demanding in the course. Neural networks introduce mathematical depth and engineering complexity that earlier modules do not. Plan to spend longer per tutorial than you have in previous units, and expect to re-read the backpropagation material more than once. That is normal — it is also what everyone else is doing.

What carries over from earlier modules

The evaluation toolkit from Module 2, Unit 6 applies here unchanged — precision, recall, F1, ROC/AUC, and cross-validation all work the same way on a neural network as they do on a logistic regression. The Python workflow from Module 1, Unit 3 also carries over, extended by Keras. Only the model itself is new.

Units & tutorials

Three units, twenty-seven tutorials

Unit 14 builds a single neuron and then a small network by hand. Unit 15 scales that network up and teaches you to keep it training well. Unit 16 adds the architectural ideas that make image recognition possible.

Unit 15 9 tutorials · ~8 hours

Deep Learning

Scale the network up and learn to keep it training well. Optimisation, regularisation, dropout, and the practical craft of getting a deep network to converge.

Unit overview Start T-01
Unit 16 9 tutorials · ~8 hours

Convolutional Neural Networks

The architecture that made computer vision practical. Convolution, pooling, and the feature hierarchies that allow a network to recognise images.

Unit overview Start T-01

Tools

A second framework joins the stack

Everything from Module 1 still applies. Module 5 adds TensorFlow and Keras for defining and training neural networks, replacing Scikit-Learn for this module's modelling work.

TensorFlow 2.x The numerical engine for neural network training.
Keras High-level API for defining network architectures.
NumPy & Pandas Data preparation, exactly as in earlier modules.
Matplotlib Plotting loss curves, accuracy curves, and feature maps.
Scikit-Learn Evaluation metrics and dataset splitting.
JupyterLab Same notebook environment as the rest of the course.

Datasets

What you will work with

Module 5 deliberately starts with small datasets so that training runs in seconds, not hours. The architectures you build are real; the data is modest by design.

MNIST Digits Units 14 & 15
Handwritten digit images — the standard starting point for neural networks.
Fashion-MNIST Unit 15
Clothing images, harder than MNIST and a realistic step up for deep networks.
CIFAR-10 Unit 16
Ten-class colour images for convolutional networks and feature-map inspection.
Tabular Datasets Units 14 & 15
Familiar tabular data from earlier modules, now modelled with small neural networks.

On compute

Every dataset and architecture in this module is chosen so that training completes on a standard laptop CPU in seconds to a few minutes. No GPU is required, and no cloud compute is needed. The point of Module 5 is understanding, not scale.

How to work through it

Build small, train often, watch the curves

Neural network training is an empirical craft. The habits that matter here are different from every earlier module — you will spend as much time watching training curves and adjusting as you will reading theory.

  1. Work through the arithmetic by hand

    The forward pass and backpropagation are not exercises in memorisation. Compute one forward pass and one backward pass on a two-neuron network yourself before touching Keras.

  2. Always plot the loss curve

    A neural network that is not learning produces a loss curve that tells you why. A flat curve, a diverging curve, a curve that falls then rises — each has a specific diagnosis.

  3. Change one thing at a time

    When a model underperforms, resist the urge to change architecture, learning rate, and regularisation simultaneously. Adjust one variable, retrain, observe. Neural network debugging is experimental design.

  4. Finish with the unit quiz

    Each unit ends with a short quiz. Score it honestly before looking at the feedback.

Assessment

How this module is assessed

Module 5 carries three unit quizzes and the neural network assignment, which asks you to build, train, and evaluate models of your own design.

  • Quiz 14 Neural Networks — 15 questions, counts toward your quiz grade Unit 14
  • Quiz 15 Deep Learning — 15 questions, counts toward your quiz grade Unit 15
  • Quiz 16 Convolutional Neural Networks — 15 questions, counts toward your quiz grade Unit 16
  • Assignment 5 Neural Network Modeling Open

Assignment 5 is the last assignment before the capstone

Assignment 5: Neural Network Modeling is released during Unit 15 and asks you to design, train, and evaluate a neural network on a dataset of your choosing. Its deadline falls in week 13, and once submitted you will move straight into the capstone. Begin it as soon as you have finished Unit 15 — do not leave it until the module ends.

Looking ahead to the capstone

The capstone project allows you to use any technique from the course, including the neural networks you build here. If your capstone dataset is image-based or involves a large number of features, the architectures from Unit 16 are likely to be directly applicable. The submission requirements are identical regardless of which technique you choose.

👤