Unit 16 — Convolutional Neural Networks

01 Unit at a glance

Where this unit sits

Unit 15 gave you the machinery to train a deep network: optimisers, initialisation and regularisation. Unit 16 asks what happens when the data has spatial structure. A dense network on a modest 256×256 colour image would need roughly 200,000 weights in its first layer alone. A convolutional layer does the same job with a few hundred, by reusing the same filter at every location. This unit is the final piece of Module 5 and the foundation for any work you do with image, audio or time-series data in the capstone.

Unit 16 summary
Module	Module 5 — Neural Networks and Deep Learning
Estimated study time	5 – 7 hours (9 tutorials, quiz, practical notebook and augmentation experiments)
Format	9 tutorial pages · 1 unit quiz · 1 practical notebook · 1 assignment component
Prerequisites	Unit 14 (forward propagation, backpropagation, activation functions) and Unit 15 (optimisers, regularisation, dropout, diagnosing training curves). Familiarity with NumPy array indexing and broadcasting is assumed.
Tools	Python, NumPy, Jupyter, Matplotlib, plus a deep-learning framework (Keras/TensorFlow or PyTorch — introduced in this unit). Conceptual work and hand-built convolution stay in NumPy.

Convolution Feature maps Pooling Parameter sharing Transfer learning

02 Learning outcomes

By the end of Unit 16 you should be able to do all of the following. These outcomes are assessed in the unit quiz, in Assignment 5, and again in the capstone project.

Knowledge — explain

Explain why a fully-connected layer is a poor fit for image data, in terms of parameter count and loss of spatial structure.
Describe the convolution operation and how a learnable filter produces a feature map.
Compute the spatial dimensions of a convolutional or pooling layer's output from its input size, kernel size, stride and padding.
Distinguish translation equivariance (convolution) from translation invariance (pooling plus global aggregation).
Summarise the design ideas behind LeNet, AlexNet, VGG and ResNet, and why residual connections made very deep networks trainable.

Skills — apply

Implement a 2-D convolution over a small image by hand and in NumPy.
Select kernel size, stride, padding and pooling to shape a feature map to a target size.
Build and train a small CNN classifier on an image dataset, with augmentation and dropout.
Use a pretrained backbone for transfer learning when the available dataset is small.

Professional competencies — evaluate

Quantify the parameter and compute savings of a convolutional design against a dense baseline.
Recognise dataset bias and representation harms in image data, and state their limits on deployment.
Report accuracy alongside the training cost, data requirements and known failure modes of a CNN.

03 Reading roadmap

Work through the nine tutorials in order. Tutorials 01–05 build the mechanics of convolution and pooling from the ground up; 06–07 assemble those parts into real architectures; 08–09 put them to work on a dataset.

Tutorial 01

Why images break fully-connected networks

Parameter explosion, discarded spatial structure, and the two properties a vision model needs.

Tutorial 02

The convolution operation

Sliding a kernel across an image, the elementwise product and sum, and a hand-worked example.

Tutorial 03

Filters, kernels & feature maps

Edge detectors, learned filters, channels, and what the network discovers for itself.

Tutorial 04

Stride, padding & output dimensions

The output-size formula, valid versus same padding, and dilated convolutions.

Tutorial 05

Pooling layers

Max and average pooling, downsampling, receptive fields and global average pooling.

Tutorial 06

Assembling a CNN architecture

Stacking convolution, activation and pooling blocks, then flattening to a classifier head.

Tutorial 07

Classic architectures: LeNet to ResNet

Thirty years of design lessons, and why skip connections changed everything.

Tutorial 08

Building a CNN in code

A complete worked example: data pipeline, model definition, training loop and evaluation.

Tutorial 09

Data augmentation & transfer learning

Stretching a small dataset, and fine-tuning a pretrained backbone.

Suggested pacing
Session	Tutorials	Focus	Time
1	01 – 03	Motivation, the convolution operation and feature maps	105 min
2	04 – 05	Geometry of convolution, stride, padding and pooling	75 min
3	06 – 07	Architecture design and landmark CNN models	90 min
4	08 – 09	Implementation, augmentation and transfer learning	120 min
Total guided study			≈ 6.5 h

04 Learning activities

Convolution is easiest to understand with your hands on the numbers. Complete each activity as you reach the corresponding tutorial.

Activity 16.1 — Count the parameters

After Tutorial 01, calculate the number of weights in a fully-connected layer taking a 128×128 grayscale image to 64 units. Then calculate the weights in a convolutional layer with 64 filters of size 3×3 over the same image. Write one sentence on the ratio.

Activity 16.2 — Convolve by hand

After Tutorial 02, take the 5×5 grid below and a 3×3 kernel, and compute the output feature map with stride 1 and no padding. Verify your answer in NumPy using a double loop before using any library function.

Input patch (5×5) and kernel (3×3)
Row	c1	c2	c3	c4	c5
r1	1	1	1	0	0
r2	0	1	1	1	0
r3	0	0	1	1	1
r4	0	0	1	1	0
r5	0	1	1	0	0
Kernel	1	0	1	0	1

Activity 16.3 — Visualise learned filters

After Tutorial 03, train a small CNN and extract the first-layer filters. Plot them as images. Do they resemble edge detectors, colour blobs, or something harder to name?

Activity 16.4 — Output shape drill

After Tutorial 04, complete the following table using the formula (W − F + 2P) / S + 1. Then check each answer by building the layer and printing its output shape.

Compute the output width
Input W	Kernel F	Padding P	Stride S	Output
32	3	0	1	?
32	3	1	1	?
32	3	1	2	?
64	5	0	2	?
28	2	0	2	?

Activity 16.5 — Max versus average pooling

After Tutorial 05, replace every max-pooling layer in your model with average pooling and retrain. Record the change in validation accuracy and the change in training time, and offer a hypothesis for the difference.

Activity 16.6 — Augmentation ablation

After Tutorial 09, train the same CNN three times: with no augmentation, with horizontal flips only, and with flips plus random crops. Plot the three validation curves on one axis and state which regulariser helped most.

Unit 16 practical notebook All six activities in one runnable notebook — unit-16-practical.ipynb

05 Assessment activities

10 q

Unit 16 quiz

Multiple choice and short answer covering the convolution operation, output dimensions, pooling, receptive fields and architecture design. Unlimited attempts; highest score counts.

A5

Assignment 5 — Neural Network Modeling

If your chosen dataset is image-like, you may satisfy the modelling component with a CNN. If not, compare a dense network against a convolutional design on a small image benchmark as an additional study.

Cap

Capstone contribution

A CNN capstone must document the data pipeline, augmentation strategy, training cost and known failure modes of the model on under-represented classes.

Where Unit 16 connects to your assessments

A5 · Part 2 Model design — convolution, pooling and classifier head Instructions →
A5 · Part 3 Evaluation — per-class accuracy and confusion matrix Rubric →
A5 · Part 4 Reflection — dataset bias and representation harms Submit →
Capstone Data acquisition, cleaning and ethical review of image data Capstone brief →

06 Key terms

Convolution: An operation that slides a small learnable filter across an input, Computing an elementwise product and sum at each position.
Kernel (filter): The small weight matrix that is learned and reused at every spatial location. Its values determine what pattern it responds to.
Feature map: The 2-D output produced by applying one filter across the whole input; each filter produces one feature map (one channel).
Stride: The number of pixels the filter moves between positions. Larger strides downsample the output.
Padding: Extra border pixels added around the input, usually zeros, to control the output size and preserve edge information.
Pooling: A fixed, non-learned downsampling operation — typically max or average — applied over small spatial windows.
Receptive field: The region of the original input that influences a given unit's activation. It grows with each convolutional and pooling layer.
Parameter sharing: Using the same weights at every spatial position, which is what makes convolutional layers dramatically cheaper than dense layers.
Translation equivariance: The property that shifting the input shifts the feature map by the same amount — a direct consequence of parameter sharing.
Residual connection: A skip connection that adds a layer's input to its output, allowing gradients to flow directly through very deep networks.
Data augmentation: Generating additional training examples by applying label-preserving transformations such as flips, crops and colour jitter.
Transfer learning: Reusing a model pretrained on a large dataset as the starting point for a new task, typically by fine-tuning the later layers.

07 Summary

What you have covered

Unit 16 introduced the convolutional network as the answer to a specific problem: dense layers ignore the spatial structure of images and explode in parameter count. Convolution solves both at once. A small filter is reused at every location, so the layer learns a pattern detector rather than a position-specific weight, and the number of parameters depends on the kernel size rather than the image size.

You learned to compute the output dimensions of a convolutional or pooling layer from its input size, kernel size, stride and padding; you saw how stacking convolution, activation and pooling blocks builds a hierarchy in which early layers detect edges and later layers respond to object parts; and you traced the design lineage from LeNet through AlexNet and VGG to ResNet, where residual connections made hundred-layer networks trainable.

Finally, you applied the regularisation instincts from Unit 15 to vision: augmentation as a data-space constraint, dropout in the classifier head, and transfer learning as the pragmatic choice when your own dataset is small.

Before you move on

You should be able to take any convolutional architecture diagram and, for each layer, state its input shape, output shape, parameter count and purpose. If that feels uncertain, redo Activity 16.4 before starting Module 6.

Common misconception

A CNN does not "see" images the way you do. It responds to statistical regularities in pixel values, which is why a small, imperceptible perturbation can change its prediction entirely — and why augmentation with realistic transformations matters more than adding layers.

Ethical note

Image datasets encode historical and social bias. A model can reach high overall accuracy while failing badly on under-represented groups. Report per-class results, not just aggregate accuracy — this theme returns in Unit 18.