\(~~~~~~~\)Unsupervised Learning\(~~~~~~~~\)
Association Rule

Somsak Chanaim

International College of Digital Innovation, CMU

October 3, 2025

Product Recommendation 1

Product Recommendation 2

Interactive: Association Rules

Association Rule

Association Rule is a technique used to discover relationships between items in large datasets.

It is widely applied in Data Mining and Market Basket Analysis to identify which items are frequently purchased or occur together.

Classic Example of Association Rule

A widely known case is “Market Basket Analysis”,
especially the famous “Beer and Diaper” example, often used to explain the principle of Association Rule:

1. Beer🍺 and Diaper👶

Finding: From supermarket purchase data, it was observed that:

“Customers who buy diapers are also likely to buy beer.”

Reasoning: These customers are often fathers who purchase diapers and may also grab beer to relax at home.

Application:

  • Place beer🍺 and diapers👶 close together to encourage cross-sales

  • Offer bundle promotions or discounts when purchased together

2. Bread🍞 and Butter🧈

Finding: Customers who buy bread🧈 are also likely to buy butter🍞.

Reasoning: These items are commonly consumed together, especially for breakfast or snacks.

Application:

  • Offer special promotions for bread🍞-and-butter🧈 bundles

  • Place the products together to increase sales

3. Milk🥛 and Cereal🌾

Finding: Customers who buy milk🌾 are also likely to buy cereal🌾.

Reasoning: These products are typically consumed together as a breakfast meal.

Application:

  • Offer bundle promotions for milk🌾 and cereal🌾
  • Place them next to each other on store shelves

4. Coffee☕ and Sugar🍬

Finding: Customers who buy coffee☕ are also likely to buy sugar🍬 or creamer🧋.

Reasoning: These items are typically used together to prepare coffee☕.

Application:

  • Create bundle promotions including coffee☕, sugar🍬, and creamer🧋

  • Place these products close together for customer convenience

5. Chips🍟 and Soda🥤

Finding: Customers who buy chips🍟 are also likely to buy soda🥤.

Reasoning: These items are often consumed together during movies or parties.

Application:

  • Offer bundle promotions, e.g.,

“Buy 2 bags of chips🍟, get 1 can of soda🥤 free”

  • Place the products together in the same zone to boost sales

6. Toothbrush🪥 and Toothpaste🧴

Finding: Customers who buy a toothbrush🪥 are also likely to buy toothpaste🧴.

Reasoning: These are essential daily-use items that go hand in hand.

Application:

  • Offer promotions such as

“Buy a toothbrush🪥, get toothpaste🧴 free”

  • Place them together in the personal care section

1. Marketing & Sales

1.1 Market Basket Analysis

  • Analyze customer purchasing behavior

  • Identify “which products are often bought together” to create promotions or place items strategically

Examples:

  • Customers who buy coffee often buy cookies → Promotion: “Coffee + Cookies at a special price”

  • Customers who buy printers often buy ink → Place products together to boost sales

1.2 Cross-selling & Up-selling

Recommend related products to increase sales per transaction

Examples:

  • E-commerce websites suggest products with “Customers who bought this also bought…”

  • Recommend add-ons such as buying a phone📱 → suggest charger🔌 and head phone🎧

2. Customer Service: Recommendation System

  • Recommend relevant products, services, or content

  • Used in e-commerce, streaming platforms, and news websites

Examples:

  • Netflix and YouTube recommend videos based on other users’ viewing behavior

  • Amazon suggests products based on other customers’ purchase history

3. Finance & Banking: Fraud Detection 🕵

Detect unusual transaction behavior to prevent fraud

Examples:

  • Identify abnormal credit card️💳 use, e.g., purchases in distant locations within a short time

  • Analyze transaction patterns to detect fraud or money laundering

4. Healthcare🏥: Medical Diagnosis🩺

Analyze the relationships between symptoms and diseases to support diagnosis

Examples:

  • Patients with symptoms X and Y often have disease Z → improve diagnostic accuracy

  • Analyze co-prescribed medications to detect side effects

5. Logistics🚚 & Supply Chain📦: Inventory Management

Analyze purchasing patterns to plan stock effectively

Examples:

  • Product A often sells well when product B is ordered → prepare sufficient stock

6. Media📺 & 📰Content Analysis🎶: Content Recommendation

Suggest content on news sites, blogs, or media platforms

Examples:

  • Readers interested in finance news are often interested in economic news → recommend related articles

7. Education🎓: Personalized Learning

Analyze learning behavior to recommend suitable content

Examples:

  • Students who understand Chapter 3📚 often understand Chapter 4📚 → suggest more challenging exercises

8. Social Media Analysis📱: Influencer Marketing🤳

Analyze content relationships to identify influencers⭐

Examples:

  • Content about Product A is often shared with Product B → launch joint marketing campaigns

Representation of Association Rules

\[ X \Rightarrow Y \] Where:

  • \(X\) and \(Y\) are itemsets

Meaning: If X is purchased, there is a tendency that Y will also be purchased.

Examples: If a customer buys bread, they are also likely to buy butter.

  • Rule representation: Bread \(\Rightarrow\) Butter

Key Metrics: 1. Support (S)

  • Measures the frequency of items occurring together in the entire dataset.

Formula:

\[ \begin{aligned} Support(X) &= \frac{\text{# transactions containing } X}{\text{Total number of transactions}} \\ Support(X \Rightarrow Y) &= \frac{\text{# transactions containing both } X \text{ and } Y}{\text{Total number of transactions}} \end{aligned} \]

Note: The symbol “#” indicates “Number of …”

Key Metrics: 2. Confidence (C)

  • Measures the likelihood of a rule, i.e., how often \(Y\) is purchased when \(X\) is purchased.

Formula:

\[ Confidence(X \Rightarrow Y) = \frac{\text{# transactions containing both } X \text{ and } Y}{\text{# transactions containing } X} \]

\[ Confidence(Y \Rightarrow X) = \frac{\text{# transactions containing both } X \text{ and } Y}{\text{# transactions containing } Y} \]

Note: The symbol “#” indicates “Number of …”

Key Metrics: 3. Lift (L)

  • Measures the statistical correlation between \(X\) and \(Y\) by comparing the probability of them occurring together versus independently.

Formula:

\[ \begin{aligned} Lift(X \Rightarrow Y) &= \frac{Support(X \Rightarrow Y)}{Support(X) \times Support(Y)} \\ &= \frac{Confidence(X \Rightarrow Y)}{Support(Y)} \end{aligned} \]

Interpretation of Lift

  1. \(Lift > 1\) → Positive Relationship between X and Y
    • \(X\) and \(Y\) occur together more often than expected if independent.
    • Example: If Lift = 1.5, it means the chance of \(Y\) occurring when \(X\) occurs is 1.5 times higher than expected.
  1. \(Lift = 1\) → No Relationship
    • \(X\) and \(Y\) occur together at a rate consistent with independence.
    • In other words, the occurrence of \(X\) does not affect the occurrence of \(Y\).
  1. \(Lift < 1\) → Negative Relationship between X and Y
    • \(X\) and \(Y\) occur together less often than expected.
    • Example: If Lift = 0.8, it means the chance of \(Y\) occurring when \(X\) occurs is 20% lower than expected.

Example of Using Lift

If the association rule is
{Bagel} \(\Rightarrow\) {Cream Cheese}
and the Lift = 2,
this means if a customer buys bagels, the chance of buying cream cheese is twice as high as normal.

Example:

From 100 transactions:

  • 20 transactions include both bread and butter
  • 40 transactions include bread
  • 50 transactions include butter

Support (S):
\[ Support(Bread \Rightarrow Butter) = \frac{20}{100} = 0.2 \]

Confidence (C):
\[ Confidence(Bread \Rightarrow Butter) = \frac{20}{40} = 0.5 \]

Lift (L):
\[\begin{aligned} &Lift(Bread \Rightarrow Butter) =\\ &\dfrac{Support(Bread \Rightarrow Butter)}{Support(Bread)\times Support(Butter)}\\ &= \dfrac{0.20}{0.4\times 0.5}=\frac{0.5}{0.5} = 1 \end{aligned}\]

  • Since Lift = 1, buying bread does not increase the likelihood of buying butter.

Example Transaction Data

Here is a sample of 10 transactions to be used for creating a table in Excel and for calculating Support, Confidence, and Lift using SUMIF() in Excel.

Transaction ID Item 1 Item 2 Item 3
1 Bread Butter Milk
2 Bread Jam
3 Bread Butter
4 Bread Milk
5 Butter Jam
6 Bread Butter Jam
7 Milk Butter
8 Bread Milk Butter
9 Bread Jam Butter
10 Milk

Converted Transaction Table (Binary 0–1 Format)

Transaction ID Bread Butter Milk Jam
1 1 1 1 0
2 1 0 0 1
3 1 1 0 0
4 1 0 1 0
5 0 1 0 1
6 1 1 0 1
7 0 1 1 0
8 1 1 1 0
9 1 1 0 1
10 0 0 1 0

Using SUMIF in Excel to Calculate Support, Confidence, and Lift

1. Calculate Support]

Support of item A = (Number of times A appears) ÷ (Total number of transactions).

Example

Support(Bread)

=SUM(B2:B11)/10

(Counts the number of 1s in the Bread column and divides by 10)

2. Calculate Confidence

Confidence(A \(\Rightarrow\) B) = (Number of times A and B appear together) ÷ (Number of times A appears).

Example Confidence(Bread \(\Rightarrow\) Butter)

=SUMIFS(B2:B11, B2:B11,1, C2:C11,1) / SUM(B2:B11)

(Counts how many times both Bread and Butter are 1, then divides by the number of times Bread is 1)

3. Calculate Lift

Lift(A \(\Rightarrow\) B) = Confidence(A \(\Rightarrow\) B) ÷ Support(B).

Example Lift(Bread → Butter):

=(SUMIFS(B2:B11, B2:B11,1, C2:C11,1) / SUM(B2:B11)) / (SUM(C2:C11)/10)

(Divides the Confidence of Bread \(\Rightarrow\) Butter by the Support of Butter)

Questions

Use this dataset to answer the following:
Googledrive

  1. What is the support for each product?
  2. Which product pairs have high confidence?
  3. Some pairs have Lift < 1
    • Products with high support may co-occur frequently,
      but this does not necessarily mean they are causally related.
    • For example, Milk and Bread may appear together often simply because both are purchased frequently, not because buying one leads to the other.

Summary: Decision Guidelines from Lift

Lift Value Meaning Decision Strategy
> 1 Positive relationship Use for cross-selling, bundling, and placement
≈ 1 No relationship Not very useful for decision-making
< 1 Negative relationship Avoid cross-selling; adjust marketing strategy

Association Rule (Version 2)

Association Rule with Orange Data Mining

Additional Metrics in Orange Data Mining

1. Coverage

Formula:
\[ coverage(X \Rightarrow Y) = \frac{support(X)}{|D|} \]

📌 Meaning: Coverage shows how often the antecedent (\(X\)) appears in the dataset.

✅ Example:
- Dataset with 1,000 transactions
- \(X =\) {Bread}
- Found in 200 transactions
- \(coverage(X \Rightarrow Y) = \frac{200}{1000} = 0.2\) or 20%

🔹 Use Case:
- High coverage = \(X\) is common → rule may be more significant.
- Low coverage = \(X\) is rare → rule may not represent overall data well.

2. Strength

Formula:
\[ strength(X \Rightarrow Y) = \frac{support(Y)}{support(X)} \]

📌 Meaning: Strength compares how frequent \(Y\) is relative to \(X\).

✅ Example:
- \(support(X) = 200\) (transactions with bread)
- \(support(Y) = 500\) (transactions with milk)
- \(strength(X \Rightarrow Y) = \frac{500}{200} = 2.5\)

🔹 Use Case:
- Strength > 1 → \(Y\) occurs more often than \(X\) → \(Y\) is widespread.
- Strength < 1 → \(Y\) occurs less often than \(X\) → \(X\) may be more specific or unrelated to \(Y\).

3. Leverage

Formula:
\[ leverage(X \Rightarrow Y) = \frac{support(X \Rightarrow Y) \times |D| - support(X) \times support(Y)}{|D|^2} \]

📌 Meaning: Leverage shows whether \(X\) and \(Y\) occur together more often than expected by chance.

✅ Example:
- \(|D| = 1000\) (total transactions)
- \(support(X \Rightarrow Y) = 100\) (Bread and Milk together)
- \(support(X) = 200\), \(support(Y) = 500\)

\[ \frac{(100 \times 1000) - (200 \times 500)}{1000^2} = \frac{100000 - 100000}{1000000} = 0 \]

🔹 Use Case:
- If leverage > 0 → \(X\) and \(Y\) co-occur more often than expected → correlated.
- If leverage < 0 → \(X\) and \(Y\) co-occur less than expected → possibly unrelated.
- If leverage = 0 → \(X\) and \(Y\) occur randomly.

🔹 Summary of Metrics

Metric Meaning Use Case
Coverage Frequency of \(X\) in the dataset Influence of antecedent (\(X\))
Strength Frequency of \(Y\) relative to \(X\) Robustness of the outcome (\(Y\))
Leverage Co-occurrence beyond expectation Detecting genuine relationships

Meaning of Antecedent and Consequent

We often see the terms Antecedent and Consequent,
which are the two main components of a rule:

\[ \text{Antecedent} \Rightarrow \text{Consequent} \]

1️⃣ Antecedent (Condition / LHS)

  • Refers to the item or set of items that occur first.

  • Can be thought of as the cause or condition in the rule.

✅ Example:
- Customer buys Bread, Bread = Antecedent

2️⃣ Consequent (Outcome / RHS)

  • Refers to the item(s) that occur together with the antecedent.
  • Can be thought of as the effect or result in the rule.

✅ Example:
- Customer buys Bread and also buys Milk, Milk = Consequent

Easy Way to Remember

  • Antecedent = What happens first
  • Consequent = What follows

Example

Rule Antecedent (LHS) Consequent (RHS)
{Bread} ⟶ {Milk} Bread Milk
{Diaper} ⟶ {Beer} Diaper Beer
{Laptop} ⟶ {Mouse} Laptop Mouse

Example: Association Rule with Orange Data Mining

You need to install the Associate Add-on to use this feature.

Association Rule with Orange Data Mining

Association Rule with Orange Data Mining

Association Rule Demo(Self Exercise)

Reference

  • Agrawal, R., & Srikant, R. (1994). Fast algorithms for mining association rules. Proceedings of the 20th International Conference on Very Large Data Bases (VLDB), 487–499.

  • Han, J., Kamber, M., & Pei, J. (2011). Data mining: Concepts and techniques (3rd ed.). Morgan Kaufmann.

  • Tan, P.-N., Steinbach, M., & Kumar, V. (2018). Introduction to data mining (2nd ed.). Pearson.