Supervised Learning Logistic Regression

01 Why this unit

Most of the decisions machine learning is asked to support are binary. Will this customer leave? Is this transaction fraudulent? Should this loan be approved? Logistic regression is the workhorse model for exactly these questions — and it remains one of the most widely deployed classifiers in industry.

The unit is also where a subtle but important shift happens. In Unit 4 the model output was the prediction. Here the model output is a score, which is then converted into a probability, which is then converted into a decision by comparing it against a threshold. Three distinct steps, and each one can be evaluated and tuned separately. Understanding that separation is the main conceptual payoff of the unit.

02 Learning outcomes

By the end of Unit 5 you should be able to do everything listed below without referring back to the notes. These outcomes are the ones the quiz and Assignment 3 are written against.

03 Reading roadmap

Work through the tutorials in order. Tutorials 5 and 6 are the conceptual centre of the unit — do not rush them, because the threshold decision in Tutorial 5 is the part practitioners most often get wrong.

Suggested study sequence

1. Read the concept, don't skim it Sections 1–5 of each tutorial carry the idea. Slow down on the worked example.
2. Run the code yourself Every notebook is short. Type it out at least once before you copy-paste it.
3. Do the self-check before the answer reveal Section 7 exists to catch the misunderstanding while it is still cheap to fix.
4. Attempt the practice exercises and problem set Sections 8 and 9 are where the skill actually forms.
5. Write two sentences in the discussion Section 10 asks you to reason about the classifier, not just run it.

04 Tutorials

Eight self-contained pages. Each follows the same twelve-section structure — objectives, motivating example, concept, visual, worked example, interactive activity, self-check, practice exercises, problem set, discussion, summary and further reading.

05 Learning activities

Reading alone will not get you to the outcomes. These are the activities that will.

import numpy as np
from sklearn.linear_model import LogisticRegression

# Features: [months_as_customer, monthly_charge], target: churned?
X = np.array([[24, 45.0], [3, 92.5], [48, 38.0], [7, 88.0], [36, 52.5]])
y = np.array([0, 1, 0, 1, 0])

clf = LogisticRegression().fit(X, y)

print("probabilities:", clf.predict_proba(X)[:, 1].round(3))
print("labels @ 0.5  :", clf.predict(X))
print("odds ratio    :", np.exp(clf.coef_[0]).round(3))

probabilities: [0.062 0.918 0.041 0.865 0.103]
labels @ 0.5  : [0 1 0 1 0]
odds ratio    : [0.943 1.121]

The odds ratio on monthly_charge is 1.121: each additional dollar of monthly charge multiplies the odds of churn by about 1.12, holding tenure constant. Note that this is a multiplicative effect on odds — not an additive effect on probability.

06 Assessment

Unit 5 contributes to your grade through one quiz and one assignment. Both are mapped directly onto the learning outcomes in Section 02.

What the quiz will ask you to do

Assignment 3 — Classification Analysis

• Part A Problem framing and data preparation — target definition, class balance, leakage check Unit 5
• Part B Model fitting and evaluation — probabilities, metrics, confusion matrix Unit 6
• Part C Threshold justification and fairness analysis — costs, trade-offs, proxy variables Unit 5

07 Summary

Key terms

Binary classification

A supervised learning task in which the target has exactly two possible values, conventionally encoded as 0 and 1.

Sigmoid (logistic) function

The function σ(z) = 1 / (1 + e^−z), which maps any real number to a value strictly between 0 and 1.

Odds

The ratio p / (1 − p). If the probability of churn is 0.8, the odds are 4 to 1.

Log-odds (logit)

The natural logarithm of the odds, ln(p / (1 − p)). Logistic regression is a linear model in this quantity.

Odds ratio

exp(wⱼ), the factor by which the odds change for a one-unit increase in feature j, holding all other features constant.

Decision boundary

The set of points where the predicted probability equals the threshold — for a two-feature model, a straight line.

Decision threshold

The probability cut-off above which the model predicts the positive class. A modelling choice, not a property of the data.

Maximum likelihood

The fitting principle that chooses the parameters which make the observed labels most probable under the model.

Cross-entropy (log-loss)

The loss function obtained by taking the negative log-likelihood. It penalises confident wrong predictions very heavily.