What is Machine Learning?

01 Learning Objectives

02 Motivating Example

Imagine you have been asked to write a program that decides whether an email is spam. This is a classic problem, and the obvious approach is to write rules.

if email contains "free money"  → mark as spam
if email contains "click here"  → mark as spam
if sender is in blocklist       → mark as spam
if email contains "unsubscribe" → mark as legitimate

This works for a while. Then reality intervenes:

• Spammers learn to write fr33 m0ney and cl1ck here, and your keyword rules stop matching.
• New senders appear every day, and your blocklist is always behind.
• A legitimate newsletter you subscribed to contains the word unsubscribe — and gets marked as spam.
• Your rules grow into thousands of lines, and nobody can explain why any single one exists.

The rules approach fails not because any individual rule is wrong, but because the problem changes faster than you can write rules to keep up with it.

03 Concept Explanation

The most widely cited formal definition of machine learning comes from Tom Mitchell’s 1997 textbook:

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

— Tom M. Mitchell, Machine Learning (1997)

The definition is dense, so let us unpack it with the spam example:

Task T

Classifying an email as spam or not spam.

Experience E

A large set of emails, each already labelled as spam or not spam.

Performance measure P

The fraction of emails the system classifies correctly.

For a program to be learning, all three must be present and linked. If performance P does not improve as experience E grows, the program is not learning — it is just following rules.

The three ingredients of a learning system

Stripping away the formality, every machine learning system contains the same three components:

1. Data — the examples the system learns from. This is what Mitchell calls experience.
2. A model — a mathematical structure with adjustable parameters. The model is what does the learning.
3. An objective — a precise definition of what “better” means. This is Mitchell’s performance measure, expressed as something the algorithm can optimise.

Every algorithm you will meet in this course — from linear regression in Module 2 to convolutional networks in Module 5 — is a specific combination of these three. Different algorithms differ in how they represent the model and what objective they optimise, but the underlying structure is always the same.

Classical programming versus machine learning

The clearest way to understand what is new about machine learning is to compare it with the classical approach you already know.

In classical programming, you write the rules and the computer applies them to data. The rules are the input; the answers are the output.

In machine learning, you provide the data and the answers, and the computer works out the rules. The rules are now the output, not the input.

This reversal — rules as output rather than input — is the single most important idea in this tutorial. It is worth reading twice.

Two broad categories

Machine learning is usually divided into two large categories, depending on what kind of data you have:

• Supervised learning — you have labelled examples. The system learns to map inputs to outputs. Spam filtering is supervised learning: every training email has a known label.
• Unsupervised learning — you have data but no labels. The system finds structure on its own. Grouping customers into segments based on purchasing behaviour, without being told what the segments should be, is unsupervised learning.

There is a third category, reinforcement learning, in which a system learns by interacting with an environment and receiving rewards. We will mention it in passing but the course focuses on the first two.

When is machine learning a good fit?

Machine learning is often described as a solution in search of problems. It is powerful, but it is not always the right tool. It tends to work well when:

• The problem is too complex to write rules for by hand, or the rules would be too numerous to maintain.
• There are patterns in the data, even if we cannot articulate what they are.
• The problem changes over time, so static rules would need constant rewriting.
• We have, or can obtain, examples to learn from.

It tends to be a poor fit when:

• The problem has clean, stable rules — use classical programming instead.
• There is no data to learn from, and none can be reasonably obtained.
• The cost of an unexplainable error is so high that a simple, auditable rule is preferable.
• The problem is fundamentally about reasoning or computation, not about finding patterns.

04 Visual Illustration

The diagram below contrasts the two approaches side by side. Read it slowly: the boxes on the left are the inputs, the boxes on the right are the outputs, and the difference between the two panels is where the rules come from.

05 Worked Example

Let us apply Mitchell’s definition to a concrete problem: predicting whether a student will pass a course.

Suppose a university has records for 200 previous students. For each student, the record contains their average hours of study per week, their attendance rate, and their prior GPA. It also records whether they ultimately passed the course.

We want to build a system that, given a new student’s study hours, attendance, and prior GPA, predicts whether they will pass.

The three ingredients of the learning system map cleanly onto this problem:

• Data: 200 rows, each with three features and one label.
• Model: For a first attempt, we might use logistic regression — a model we will meet in Module 2 that produces a probability between 0 and 1.
• Objective: Minimise the number of incorrect predictions on the training data, or equivalently maximise accuracy.

Training the model means adjusting its internal parameters until the objective is as good as it can be. Once trained, we check the model on a fresh set of students it has never seen, to make sure it learned something general rather than memorising the training data.

What we have just described is the full lifecycle of a supervised learning project, compressed into three paragraphs. Everything else in this course is a refinement of these ideas: better models, better objectives, better ways of measuring whether the model has actually learned something useful.

06 Interactive Activity

07 Self-Check Questions

Try each question before revealing the answer. If you get one wrong, re-read the section it refers to before continuing.

08 Practice Exercises

These exercises build fluency with the vocabulary and concepts from this tutorial. They are short by design. If any feel difficult, revisit the relevant section before moving on.

1. Identify the three components. For each scenario below, state the task (T), the experience (E), and the performance measure (P) in Mitchell's definition.
   • A system that predicts tomorrow's rainfall from today's weather readings.
   • A system that sorts incoming customer emails into “urgent” and “routine”.
   • A system that recommends songs to a listener based on their past listening history.
2. Classify the problem. For each of the following, decide whether it is better suited to classical programming or machine learning, and give one sentence of justification.
   • Converting a Fahrenheit temperature into Celsius.
   • Predicting whether a customer will click on an advertisement.
   • Checking whether a password meets a set of complexity rules.
   • Diagnosing whether a photograph shows a benign or malignant skin lesion.
3. Match the terms. Match each term on the left with its definition on the right.

   Term	Definition
   Feature	What we want the model to predict
   Label	A number inside the model that is adjusted during training
   Parameter	An input variable the model uses
   Loss	A measure of how wrong the model's predictions are

4. True or false, with justification. State whether each claim is true or false and give a one-sentence reason.
   • Every system that improves over time is a machine learning system.
   • A lookup table that stores the answer for every possible input is a model.
   • If a program's performance at a task improves as it sees more data, it is learning in Mitchell's sense.

09 Problem Sets

Analytical problems

1. A hospital is considering two systems to help triage patients arriving in an emergency department. System A is a rule-based system built from a physician's decision protocol. System B is a machine learning system trained on five years of triage decisions made by experienced nurses.

   (a) Which system is more likely to perform well on unusual cases the protocol was not designed for? Justify your answer. (b) Which system is more likely to explain why it made a particular decision? (c) If the hospital had only six months of triage data, how would your recommendation change?

2. Consider the following claim: “Any task that a human can perform can in principle be performed by a machine learning system.”

   Give two distinct counterexamples. For each, explain what specifically about the task makes it unsuitable for a learning approach.

3. The performance measure P in Mitchell's definition is chosen by whoever is building the system. Describe a scenario in which two reasonable people would disagree about what P should be for the same task, and explain what practical consequences that disagreement would have.

Technical problems

1. Write pseudocode for a system that learns to predict a student's exam score from hours studied. Your pseudocode should include all four steps of the training loop from T-05 and should identify which variables are features, which are labels, and which are parameters.

2. The following table records four training examples for a system that predicts whether a customer will make a purchase.

   Age	Previous purchases	Made a purchase?
   25	0	No
   42	8	Yes
   31	2	No
   55	12	Yes

   (a) Identify the features and the label. (b) A model makes the following predictions on this data: No, Yes, Yes, Yes. Compute the accuracy of the model. (c) Is this dataset sufficient to train a reliable model? Explain your reasoning.

3. A learning system stores its knowledge as a set of 47 numbers. If you replaced all 47 numbers with random values, would the resulting program still be a model? Would it still be learning? Justify your answers with reference to the definition of machine learning from this tutorial.

Theoretical questions

1. Why does Mitchell's definition require that performance improve with experience? What would be lost if the definition were weakened to simply “performance correlates with experience”?
2. Classical programming is sometimes described as encoding knowledge explicitly and machine learning as encoding knowledge implicitly. What does this distinction mean, and is it a useful way to characterise the difference between the two approaches?
3. Every machine learning system contains a model, and every model has parameters. Is it possible to imagine a learning system whose parameters are not numbers? If so, what would such a system look like, and would it still fit Mitchell's definition?
4. The tutorial claims that machine learning is not always the right tool for a problem, even when a learning system could in principle be built. What kinds of reasons would justify choosing a simpler non-learning approach for a problem that machine learning could handle?

10 Discussion and Projects

Discussion questions

1. A news article claims that a machine learning system has “learned to detect lies with 85% accuracy.” What questions would you want answered before deciding whether to believe this claim?
2. A university considers replacing its rule-based plagiarism detection system with a machine learning one trained on past cases of confirmed plagiarism. What are the potential advantages? What concerns would you raise?
3. Some computer scientists argue that the term “machine learning” is misleading because these systems do not learn the way humans do. Do you think the terminology is problematic? If so, what would you change it to?
4. Consider a system that recommends which candidates to interview for a job. The system is a machine learning model trained on ten years of hiring data. Discuss: should this system be built at all? What conditions would need to be met for it to be acceptable?

Case study: the Amazon hiring algorithm

Critical-thinking activity: evaluating an AI claim

Homework assignment

Mini-project: build a decision tree by hand

11 Summary

12 Further Reading

The following sources expand on the ideas in this tutorial. None are required reading, but each is worth returning to as the course progresses.

• Mitchell, T. M. (1997). Machine Learning. McGraw-Hill. Chapter 1.
  The source of the formal definition used in this tutorial. Chapter 1 is short and accessible.
• Domingos, P. (2012). “A Few Useful Things to Know About Machine Learning.” Communications of the ACM, 55(10), 78–87.
  A widely cited article that pairs well with this tutorial. It offers a practitioner’s view of what matters and what does not.
• Jordan, M. I., & Mitchell, T. M. (2015). “Machine learning: Trends, perspectives, and prospects.” Science, 349(6245), 255–260.
  A broader survey of the field, useful for understanding where machine learning sits within computer science and beyond.
• Google Developers. Machine Learning Crash Course.
  A free online resource with a gentler, more applied introduction. Useful as a companion to the first few units.