Unit 14 — Neural Networks

01 Unit at a glance

Where this unit sits

Units 4–9 gave you models that learn a fixed functional form — a line, a logistic curve, a tree, a margin. Unit 14 replaces that fixed form with a learnable network of simple units. Everything here is the foundation for Units 15 and 16, and for Assignment 5.

Unit 14 summary
Module	Module 5 — Neural Networks and Deep Learning
Estimated study time	4.5 – 6 hours (9 tutorials, quiz and practice)
Format	9 tutorial pages · 1 unit quiz · 1 practical notebook · 1 assignment component
Prerequisites	Units 4–6 (regression, logistic regression, model evaluation) and Unit 3 (Python ML ecosystem). Comfort with derivatives at the level of “the slope of a curve” is assumed.
Tools	Python, NumPy, scikit-learn, Jupyter, Matplotlib

Neural networks Backpropagation Optimisation Applied ML

02 Learning outcomes

By the end of Unit 14 you should be able to do all of the following. These outcomes are assessed in the unit quiz, in Assignment 5, and again in the capstone project.

Knowledge — explain

Describe what an artificial neuron computes and why a bias term is needed.
Explain why a network without non-linear activation functions collapses into a single linear model.
Trace forward propagation through a multi-layer network by hand.
Explain backpropagation as the chain rule applied layer by layer.

Skills — apply

Implement a forward pass and a weight update in NumPy.
Choose an appropriate activation function for a hidden or output layer.
Train a classifier with MLPClassifier from scikit-learn.
Diagnose underfitting and overfitting from training curves.

Professional competencies — evaluate

Justify when a neural network is — and is not — the right tool for a problem.
Report model performance honestly, including the cost of training and tuning.
Recognise that opaque models raise accountability questions addressed later in Unit 18.

03 Reading roadmap

Work through the nine tutorials in order. Each one builds directly on the previous, and each ends with a self-check you should complete before moving on.

Tutorial 01

Why neural networks?

The limits of linear models, and the intuition behind learning representations.

Tutorial 02

The artificial neuron

Weighted sums, bias, and the perceptron as the atom of a network.

Tutorial 03

Activation functions

Sigmoid, tanh, ReLU and softmax — what each one does and when to use it.

Tutorial 04

Architecture & forward propagation

Layers, matrix notation, and Computing a prediction end to end.

Tutorial 05

Loss functions

Mean squared error and cross-entropy, and what "error" means for a network.

Tutorial 06

Backpropagation

The chain rule, layer by layer — the algorithm that makes deep learning possible.

Tutorial 07

Weight updates & learning rate

Gradient descent, batch sizes, epochs, and what happens when the rate is wrong.

Tutorial 08

Building a network in scikit-learn

A complete worked example with MLPClassifier, from split to score.

Tutorial 09

Diagnosing & improving training

Learning curves, overfitting, early stopping, and a first look at regularisation.

Suggested pacing
Session	Tutorials	Focus	Time
1	01 – 03	Neurons and activations — the building blocks	90 min
2	04 – 05	Forward propagation and measuring error	75 min
3	06 – 07	Backpropagation and gradient descent	105 min
4	08 – 09	Implementation, evaluation and tuning	90 min
Total guided study			≈ 6 h

04 Learning activities

Reading alone will not make this unit stick. Complete the following activities as you go — each one maps to a specific tutorial.

Activity 14.1 — Build a neuron by hand

After Tutorial 02, take three inputs and one set of weights, and compute the output of a single neuron on paper. Then change the bias by ±1 and describe how the decision boundary moves.

Activity 14.2 — Activation function explorer

After Tutorial 03, plot sigmoid, tanh and ReLU on the same axes in a Jupyter notebook. Annotate where each saturates and explain why saturation slows learning.

Activity 14.3 — Forward pass in NumPy

After Tutorial 04, implement a two-layer forward pass using only matrix multiplication and your chosen activation. Verify it against a hand calculation for a single input vector.

Activity 14.4 — Trace the gradient

After Tutorial 06, work through the backpropagation derivation for a network with one hidden layer. Write out each partial derivative explicitly before looking at the worked solution.

Activity 14.5 — Learning rate experiment

After Tutorial 07, train the same network three times with learning rates of 0.001, 0.1 and 10. Plot the loss curves together and write two sentences describing what each curve tells you.

Unit 14 practical notebook All five activities in one runnable notebook — unit-14-practical.ipynb

05 Assessment activities

10 q

Unit 14 quiz

Multiple choice and short answer, covering neuron mechanics, activation functions, forward propagation and backpropagation. Unlimited attempts; highest score counts.

A5

Assignment 5 — Neural Network Modeling

Train and evaluate a neural network on a provided dataset. Report your architecture, hyperparameters, results and a short ethical reflection.

Cap

Capstone contribution

If your capstone problem is non-linear, you may use a neural network as your primary model — justify the choice against a simpler baseline.

Assignment 5 at a glance

A5 · Part 1 Data preparation and baseline model (logistic regression) Instructions →
A5 · Part 2 Design, train and tune a neural network Instructions →
A5 · Part 3 Evaluation, error analysis and comparison to baseline Rubric →
A5 · Part 4 Reflection: interpretability, cost and responsible use Submit →

06 Key terms

Artificial neuron: A unit that computes a weighted sum of its inputs plus a bias, then applies an activation function.
Activation function: A non-linear function applied to a neuron's pre-activation value. Without it, stacked layers remain a linear model.
Forward propagation: Passing input data through the network layer by layer to produce an output prediction.
Loss function: A scalar measure of how wrong the network's prediction is for the current batch of data.
Backpropagation: The algorithm that applies the chain rule backwards through the network to compute the gradient of the loss with respect to every weight.
Learning rate: The step size used when updating weights in the direction of the negative gradient.
Epoch: One complete pass of the training algorithm over the entire training dataset.

07 Summary

What you have covered

Unit 14 introduced the neural network as a composition of simple, learnable units. You began with a single neuron and its weighted sum, added non-linear activation functions to give the network expressive power, and assembled neurons into layers. You then traced a forward pass to produce a prediction, measured the error with a loss function, and used backpropagation to assign blame to each weight before updating it with gradient descent.

The essential idea to carry forward is this: a neural network is not magic — it is a differentiable function whose parameters are adjusted by repeatedly applying the chain rule. Every architecture in Unit 15 and Unit 16 is a variation on that same loop.

Before you move on

You should be able to derive the weight update for a single-layer network without notes. If you cannot, revisit Tutorials 06 and 07 before starting Unit 15.

Common misconception

More layers do not automatically mean a better model. Without enough data, appropriate regularisation and honest validation, a deeper network simply overfits faster.