Unit 12 — Association Learning | COMP328: Fundamentals of Machine Learning

01 Learning outcomes

By the end of Unit 12 you should be able to do the following. Each outcome is assessed somewhere — the mapping is given in Section 05.

Knowledge — explain

Define association rule mining and describe the structure of a rule X → Y.
Explain what support, confidence, and lift each measure, and why no single one is sufficient.
State the downward closure (anti-monotonicity) property and explain why it makes Apriori efficient.
Describe how Apriori generates and prunes candidate itemsets level by level.
Explain how FP-Growth avoids candidate generation, and when it is preferable to Apriori.

Skills — apply

Encode transactional data as a one-hot / boolean matrix suitable for rule mining.
Compute support, confidence, and lift by hand on a small transaction table.
Run Apriori and FP-Growth with mlxtend and tune min_support and min_threshold.
Filter and rank rules by lift rather than confidence alone.
Interpret a rule set as an actionable business or engineering recommendation.
Report rule counts, threshold choices, and their trade-offs.

Professional competencies — evaluate

Recognise that association rules describe co-occurrence, not causation.
Identify rules that are statistically strong but commercially useless, and say why.
Assess the ethical risks of mining behavioural data at scale.
Document thresholds and parameters so that a rule set can be reproduced.

02 Why association learning matters

Clustering and dimensionality reduction both operate on a table of numeric features. But a great deal of real-world data is not numeric at all — it is a list of things that happened together. A shopping basket. A set of symptoms in a patient record. The pages a user visited in one session. The services a microservice called while handling a request. Association learning is the tool for this shape of data.

The output is unusually readable. A decision tree gives you a tree, a neural network gives you weights, but association learning gives you sentences: "If a customer buys bread and butter, they buy milk 68% of the time, and they are 2.3× more likely to buy milk than a random customer." That readability is why the technique has survived for three decades while far more sophisticated methods have come and gone — and why it remains one of the standard tools for recommender pre-processing, cross-sell analysis, and diagnostic rule discovery.

The itemset lattice for three items, with a minimum support of 0.20. Bread and Milk are frequent on their own, so {Bread, Milk} is a candidate worth counting. Eggs is not frequent, so by downward closure every itemset containing Eggs is pruned without ever being counted — three of the six candidate pairs and the full triple are eliminated before a single database pass. This single observation is what makes Apriori tractable.

Common misconception

"Customers who buy X also buy Y" does not mean X causes Y. It may mean Y causes X, or that both are caused by a third thing — a promotion, a seasonal event, or the fact that both items sit on the same shelf. Lift corrects for how common Y is in general, but it still cannot establish direction or causation. Treat every rule as a hypothesis to be tested, not a conclusion.

03 Reading roadmap

The nine tutorials below are designed to be read in order. T-01 and T-02 set up the problem and the data representation; T-03 develops the three core measures; T-04 to T-06 cover the algorithms; T-07 to T-09 deal with evaluation, implementation, and a full worked case study.

T-01

What association learning is

Co-occurrence as a pattern type, market basket analysis, and how this differs from clustering.

T-02

Transactions and itemsets

Representing baskets as sets, one-hot encoding, and the combinatorial explosion of candidate itemsets.

T-03

Support, confidence, and lift

The three measures every rule is judged by, worked through on a small transaction table.

T-04

The Apriori algorithm

Level-wise candidate generation, downward closure, and why the pruning step is the whole point.

T-05

Rule generation and pruning

Turning frequent itemsets into rules, and the confidence-based pruning that keeps the output manageable.

T-06

FP-Growth and scalability

The FP-tree, divide-and-conquer mining, and when to abandon Apriori.

T-07

Evaluating and filtering rules

Lift versus confidence, conviction, redundancy, and practical thresholds for real datasets.

T-08

Mining rules with mlxtend

A complete implementation workflow: encoding transactions, running Apriori, generating and filtering rules.

T-09

Case study: retail baskets

End-to-end analysis of a grocery transaction log, from raw receipts to a cross-sell recommendation.

Study tip

Association learning is unusually easy to self-check, because you can count. For every rule you meet in T-03 onward, count the relevant transactions by hand on the small example table and verify the support, confidence, and lift figures yourself. If your numbers match, you understand the definitions. If they do not, the discrepancy will tell you exactly which one you have misread.

04 Learning activities

Each tutorial contains a worked example and a self-check. The activities below are the ones that require you to write and run code.

Activity 12.1 — The three measures by hand

Using the ten-transaction grocery table from T-03, compute support, confidence, and lift for the rules {Bread} → {Milk} and {Milk} → {Bread}. Confirm that the two rules have identical support and identical lift, but different confidence. Explain in two sentences why that asymmetry is the reason confidence alone is a dangerous filter.

Activity 12.2 — Threshold sensitivity

Load a grocery transaction dataset and run Apriori with min_support values of 0.05, 0.02, 0.01, and 0.005. Record the number of frequent itemsets and the number of rules generated at each threshold, along with the runtime. Plot rules against threshold on a log scale. At what point does the output stop being reviewable by a human?

Activity 12.3 — Confidence is not enough

From your 0.01-support rule set, take the top ten rules by confidence and the top ten rules by lift. Compare the two lists. For each rule in the confidence list that is not in the lift list, identify the high-support consequent that is inflating its confidence, and write one sentence explaining why the rule is uninteresting.

unit-12-association-learning.ipynb Starter notebook containing transaction datasets, encoding helpers, and scaffolded cells for activities 12.1–12.3. Open workflow

05 Assessment activities

10%

Unit 12 quiz

12 questions covering support, confidence, lift, downward closure, Apriori's level-wise structure, and FP-Growth. Unlimited attempts; highest score counts.

15%

Assignment 4

Unsupervised learning. Your submission must include an association-rule analysis with justified thresholds and an interpretation of the top rules by lift.

—

Self-checks

Embedded in every tutorial. Ungraded, but they are the fastest way to find out whether you have actually understood the previous section.

Outcome-to-assessment mapping

Where each learning outcome is assessed
Outcome	Quiz	Assignment 4	Capstone
Define rules and their structure	Q1–Q2	Report §1	Methodology
Compute support, confidence, lift	Q3–Q5	Report §2	Evaluation
Explain downward closure and Apriori	Q6–Q8	Report §2	Methodology
Apply Apriori / FP-Growth in code	Q9–Q10	Code + results	Model building
Filter and interpret rules responsibly	Q11–Q12	Report §3	Ethics analysis

Quiz 12 Association learning — 12 questions, 30 minutes Revise

Assignment 4 Unsupervised Learning: clustering, dimensionality reduction, and association rules Open brief

06 Key terms

Market basket analysis: The application of association rule mining to retail transaction data, in order to discover which products are purchased together.
Itemset: A set of one or more items occurring together in a transaction. A k-itemset contains exactly k items.
Association rule: An implication of the form X → Y, where X and Y are disjoint itemsets. Read as "if X, then Y".
Antecedent / consequent: The left-hand side (X) and right-hand side (Y) of a rule, respectively.
Support: The proportion of all transactions that contain a given itemset. For a rule, support is the proportion containing both X and Y.
Confidence: The proportion of transactions containing X that also contain Y — formally, support(X ∪ Y) / support(X).
Lift: The ratio of a rule's confidence to the support of its consequent. Lift > 1 means X and Y co-occur more often than chance; lift = 1 means independence.
Conviction: An alternative to lift that is sensitive to the direction of the rule and is undefined (infinite) for perfectly deterministic rules.
Frequent itemset: An itemset whose support meets or exceeds the minimum support threshold.
Downward closure (anti-monotonicity): The property that every subset of a frequent itemset is itself frequent — equivalently, if an itemset is infrequent, all of its supersets are infrequent. The basis of Apriori's pruning.
Apriori: A level-wise algorithm that finds frequent itemsets by generating candidates of size k, counting them, pruning, and repeating with size k+1.
FP-Growth: Frequent Pattern Growth — an algorithm that compresses the database into an FP-tree and mines it recursively, avoiding explicit candidate generation.
Redundant rule: A rule whose information is already implied by a more general rule with equal or greater confidence.

07 Summary

What you should take away

Association learning finds co-occurrence patterns in transactional data. Its output is a set of readable if–then rules.
Every rule is judged by three numbers. Support says how common the pattern is. Confidence says how reliable the implication is. Lift says whether the implication is any better than chance.
Confidence alone is misleading. A rule pointing at a very popular item will have high confidence and a lift near 1. Always filter by lift.
Downward closure is the key insight behind Apriori: if an itemset is infrequent, none of its supersets can be frequent, so they need never be counted.
Apriori is simple and interpretable but repeatedly scans the database. FP-Growth builds a compressed tree once and mines it recursively, and is usually the better choice on large datasets.
The number of rules explodes as you lower the support threshold. Choose thresholds that leave you with a human-reviewable output, and rank by lift.
Rules describe co-occurrence, not causation. Report them as hypotheses.

Ready to move on?

Once you are comfortable with the tutorials and have scored at least 80% on the unit quiz, continue to Unit 13 — Anomaly Detection, where you will learn how to find the rare events that clustering and association rules are designed to ignore.

08 Frequently asked questions

Is association learning the same as clustering?

No, though both are unsupervised. Clustering groups rows — transactions or customers — into similar sets. Association learning groups columns — items — into co-occurring sets, and then turns those sets into rules. A clustering result tells you "these customers are alike". An association result tells you "these products are bought together". They answer different questions and are often used on the same dataset for different purposes.

What counts as a good lift value?

There is no universal threshold, but the interpretation is straightforward: lift = 1 means independence, lift < 1 means the items avoid each other, and lift > 1 means they attract. In retail practice, lift above roughly 1.5 is where rules start to be interesting, and lift above 3 is usually worth investigating closely. The exact number depends entirely on how sparse your item set is — with a hundred thousand SKUs, even a lift of 5 can occur by chance somewhere in the data. Combine lift with a minimum support floor, and validate the rule on held-out transactions.

My Apriori run returned zero rules. What went wrong?

Almost always, the thresholds are too high. This is the single most common error in this unit. Try, in order:

Lower min_support by a factor of five.
Lower min_threshold (the confidence floor) to 0.3 or below.
Check that your transaction matrix is boolean, not counts — Apriori implementations expect True/False.
Check that you did not accidentally pass the one-hot matrix transposed, which would mine item co-occurrence across items instead of across baskets.

Apriori or FP-Growth for my assignment?

For the dataset sizes used in this course, either works and the results will be identical — the algorithms find the same frequent itemsets. Apriori is easier to explain in a report because its level-wise structure is intuitive. FP-Growth is faster and worth using if your dataset has more than a few thousand transactions, or if you want to demonstrate awareness of scalability. If you use both, report that they agreed — that is a useful correctness check.

Can I use association rules to make predictions?

Only cautiously. You can use a rule as a simple recommender: if a basket contains X, recommend Y. But rules mined from historical data encode historical behaviour, including historical promotions and stockouts. A rule like {phone} → {case} may reflect genuine preference or may simply reflect the fact that the two were bundled. Before deploying a rule as a recommendation, you need an experiment — an A/B test — not just a high lift score on past data.