Supervised Learning Linear Regression

01 Why this unit

Linear regression is the smallest complete machine learning system. It has a model, a loss function, an optimisation procedure and an evaluation story — and every one of those parts reappears, at larger scale, in the deep networks you will meet in Module 5.

Because the mathematics stays simple, this is the unit where you can see the whole machine at once. If you understand precisely what least squares is minimising, why the loss surface is shaped like a bowl, and what a residual plot is telling you, then the later algorithms in this course stop being a list of tricks and start being variations on a theme you already know.

02 Learning outcomes

By the end of Unit 4 you should be able to do everything listed below without referring back to the notes. These outcomes are the ones the quiz and Assignment 2 are written against.

03 Reading roadmap

Work through the tutorials in order. Each one assumes the vocabulary introduced by the previous one, and Tutorials 7 and 8 assume you can already fit and evaluate a model.

Suggested study sequence

1. Read the concept, don't skim it Sections 1–5 of each tutorial carry the idea. Slow down on the worked example.
2. Run the code yourself Every notebook is short. Type it out at least once before you copy-paste it.
3. Do the self-check before the answer reveal Section 7 exists to catch the misunderstanding while it is still cheap to fix.
4. Attempt the practice exercises and problem set Sections 8 and 9 are where the skill actually forms.
5. Write two sentences in the discussion Section 10 asks you to reason about the model, not just run it.

04 Tutorials

Eight self-contained pages. Each follows the same twelve-section structure — objectives, motivating example, concept, visual, worked example, interactive activity, self-check, practice exercises, problem set, discussion, summary and further reading.

05 Learning activities

Reading alone will not get you to the outcomes. These are the activities that will.

import numpy as np
from sklearn.linear_model import LinearRegression

# Features: [size_m2, bedrooms], target: price in thousands
X = np.array([[62, 2], [85, 3], [104, 3], [128, 4], [150, 4]])
y = np.array([210, 275, 330, 402, 455])

model = LinearRegression().fit(X, y)

print("weights :", model.coef_.round(2))
print("bias    :", round(model.intercept_, 2))
print("R²      :", round(model.score(X, y), 4))

weights : [ 2.86  5.71]
bias    : 36.9
R²      : 0.9979

The weight on size_m2 says: holding bedrooms constant, each extra square metre is associated with about 2.86 thousand more in price. That sentence — not the R² — is usually what a stakeholder actually wants.

06 Assessment

Unit 4 contributes to your grade through one quiz and one assignment. Both are mapped directly onto the learning outcomes in Section 02.

What the quiz will ask you to do

Assignment 2 — Regression Modelling

• Part A Data exploration and preparation — features, target, missing values, leakage check Unit 4
• Part B Model fitting and evaluation — train/test split, metrics, residual analysis Unit 6
• Part C Interpretation and limitations — coefficient reading, fairness and reporting Unit 6

07 Summary

Key terms

Hypothesis function

The function h(x) that maps a feature vector to a predicted target value. For linear regression, h(x) = w·x + b.

Residual

The difference between an observed target and the model's prediction for that example: eᵢ = yᵢ − ŷᵢ.

Residual sum of squares (RSS)

The sum of all squared residuals across the training set. Least squares minimises this quantity.

Cost function

A function of the model parameters that measures how wrong the model is. Training means finding the parameters that minimise it.

Convexity

A property of the MSE loss surface for linear regression: any local minimum is the global minimum, so gradient descent cannot get stuck.

Normal equation

The closed-form solution w = (XᵀX)⁻¹Xᵀy, which gives the least-squares weights directly, without iteration.

R² (coefficient of determination)

The proportion of variance in the target explained by the model, relative to predicting the mean. Not a measure of correctness.