diff --git a/_config.yml b/_config.yml
index ea7d07e6d..026bd5afd 100644
--- a/_config.yml
+++ b/_config.yml
@@ -8,22 +8,22 @@ plugins:
#-------------------------------
# General Site Settings
-title: Johnny Hopkins
-description: "Hi I'm Johnny, and I'm a Data Scientist. My portfolio focuses on interesting projects I've recently undertaken, with a strong emphasis on business impact. Please visit my Github & LinkedIn pages (or download my Resume) by using the links below!"
+title: Charles Goodaker
+description: "Hi I'm Charles, and I'm a Data Scientist. My portfolio focuses on projects I've recently undertaken, with a strong emphasis on business impact. Please visit my Github & LinkedIn pages (or download my Resume) by using the links below!"
baseurl: "" # the subpath of your site, e.g. /blog
url: "" # the base hostname & protocol for your site, e.g. http://example.com
#-------------------------------
# About Section
subtitle: Data Science Portfolio
-location: "London, UK"
+location: "Canton, GA"
resume_url: /docs/resume.docx
-avatar_image: /img/profile_picture.JPG
+avatar_image: /img/CAF9BD2A-23C8-4235-BE2F-89315A9439E7_1_105_c.jpeg
#-------------------------------
# Contact links
-linkedln: "https://linkedln.com/#" # Add your linkedln handle
-github: "https://github.com/#" # Add your github handle
+linkedln: "https://linkedin.com/in/charles-goodaker-1b9a86b" # LinkedIn handle
+github: "https://github.com/cgoodaker" # Add your github handle
paginate: 6
paginate_path: "/page/:num"
@@ -52,3 +52,4 @@ defaults:
# - vendor/gems/
# - vendor/ruby/
+
diff --git a/_posts/2026-03-10-coffee-and-python.md b/_posts/2026-03-10-coffee-and-python.md
new file mode 100644
index 000000000..786e67a24
--- /dev/null
+++ b/_posts/2026-03-10-coffee-and-python.md
@@ -0,0 +1,66 @@
+---
+layout: post
+title: Coffee & Python
+image: "/posts/coffee_python.jpg"
+tags: [Python, Coffee]
+---
+
+# My first project
+## is all about
+### how much
+#### I LOVE
+##### Python & Coffee!
+
+---
+
+Firstly, I love Python so much, here is some code!
+
+```
+my_love_for_python = 0
+my_python_knowledge = 0
+
+for day in lifetime:
+ my_love_for_python += 1
+ my_python_knowledge += 1
+```
+
+Just so you really see how much I love Python, here is some code BUT with some colours for keywords & functionality!
+
+```python
+my_love_for_python = 0
+my_python_knowledge = 0
+
+for day in lifetime:
+ my_love_for_python += 1
+ my_python_knowledge += 1
+```
+
+Here is an **unordered list** showing some things I love about Python
+
+* For my work
+ * Data Analysis
+ * Data Visualisation
+ * Machine Learning
+* For fun
+ * Deep Learning
+ * Computer Vision
+ * Projects about coffee
+
+Here is an _ordered list_ showing some things I love about coffee
+
+1. The smell
+ 1. Especially in the morning, but also at all times of the day!
+2. The taste
+3. The fact I can run the 100m in approx. 9 seconds after having 4 cups in quick succession
+
+I love Python & Coffee so much, here is that picture from the top of my project AGAIN, but this time, in the BODY of my project!
+
+
+
+The above image is just linked to the actual file in my Github, but I could also link to images online, using the URL!
+
+A line break, like this one below - helps me make sense of what I'm reading, especially when I've had so much coffee that my vision goes a little blurry
+
+---
+
+I could also add things to my project like links, tables, quotes, and HTML blocks - but I'm starting to get a cracking headache. Must be coffee time.
diff --git a/_posts/2021-06-09-Finding-Prime-Numbers-With-Python.md b/_posts/2026-04-30-Finding-Prime-Numbers-With-Python.md
similarity index 100%
rename from _posts/2021-06-09-Finding-Prime-Numbers-With-Python.md
rename to _posts/2026-04-30-Finding-Prime-Numbers-With-Python.md
diff --git a/_posts/2026-05-19-chi-square-test.md b/_posts/2026-05-19-chi-square-test.md
new file mode 100644
index 000000000..11f2902fa
--- /dev/null
+++ b/_posts/2026-05-19-chi-square-test.md
@@ -0,0 +1,322 @@
+---
+layout: post
+title: Assessing Campaign Performance Using Chi-Square Test For Independence
+image: "/posts/ab-testing-title-img.png"
+tags: [AB Testing, Hypothesis Testing, Chi-Square, Python]
+---
+
+In this project we apply Chi-Square Test For Independence (a Hypothesis Test) to assess the performance of two types of mailers that were sent out to promote a new service!
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results & Discussion](#overview-results)
+- [01. Concept Overview](#concept-overview)
+- [02. Data Overview & Preparation](#data-overview)
+- [03. Applying Chi-Square Test For Independence](#chi-square-application)
+- [04. Analysing The Results](#chi-square-results)
+- [05. Discussion](#discussion)
+
+___
+
+# Project Overview
+
+### Context
+
+Earlier in the year, our client, a grocery retailer, ran a campaign to promote their new "Delivery Club" - an initiative that costs a customer $100 per year for membership, but offers free grocery deliveries rather than the normal cost of $10 per delivery.
+
+For the campaign promoting the club, customers were put randomly into three groups - the first group received a low quality, low cost mailer, the second group received a high quality, high cost mailer, and the third group were a control group, receiving no mailer at all.
+
+The client knows that customers who were contacted, signed up for the Delivery Club at a far higher rate than the control group, but now want to understand if there is a significant difference in signup rate between the cheap mailer and the expensive mailer. This will allow them to make more informed decisions in the future, with the overall aim of optimising campaign ROI!
+
+
+
+### Actions
+
+For this test, as it is focused on comparing the *rates* of two groups - we applied the Chi-Square Test For Independence. Full details of this test can be found in the dedicated section below.
+
+**Note:** Another option when comparing "rates" is a test known as the *Z-Test For Proportions*. While, we could absolutely use this test here, we have chosen the Chi-Square Test For Independence because:
+
+* The resulting test statistic for both tests will be the same
+* The Chi-Square Test can be represented using 2x2 tables of data - meaning it can be easier to explain to stakeholders
+* The Chi-Square Test can extend out to more than 2 groups - meaning the client can have one consistent approach to measuring significance
+
+From the *campaign_data* table in the client database, we isolated customers that received "Mailer 1" (low cost) and "Mailer 2" (high cost) for this campaign, and excluded customers who were in the control group.
+
+We set out our hypotheses and Acceptance Criteria for the test, as follows:
+
+**Null Hypothesis:** There is no relationship between mailer type and signup rate. They are independent.
+**Alternate Hypothesis:** There is a relationship between mailer type and signup rate. They are not independent.
+**Acceptance Criteria:** 0.05
+
+As a requirement of the Chi-Square Test For Independence, we aggregated this data down to a 2x2 matrix for *signup_flag* by *mailer_type* and fed this into the algorithm (using the *scipy* library) to calculate the Chi-Square Statistic, p-value, Degrees of Freedom, and expected values
+
+
+
+
+### Results & Discussion
+
+Based upon our observed values, we can give this all some context with the sign-up rate of each group. We get:
+
+* Mailer 1 (Low Cost): **32.8%** signup rate
+* Mailer 2 (High Cost): **37.8%** signup rate
+
+However, the Chi-Square Test gives us the following statistics:
+
+* Chi-Square Statistic: **1.94**
+* p-value: **0.16**
+
+The Critical Value for our specified Acceptance Criteria of 0.05 is **3.84**
+
+Based upon these statistics, we retain the null hypothesis, and conclude that there is no relationship between mailer type and signup rate.
+
+In other words - while we saw that the higher cost Mailer 2 had a higher signup rate (37.8%) than the lower cost Mailer 1 (32.8%) it appears that this difference is not significant, at least at our Acceptance Criteria of 0.05.
+
+Without running this Hypothesis Test, the client may have concluded that they should always look to go with higher cost mailers - and from what we've seen in this test, that may not be a great decision. It would result in them spending more, but not *necessarily* gaining any extra revenue as a result
+
+Our results here also do not say that there *definitely isn't a difference between the two mailers* - we are only advising that we should not make any rigid conclusions *at this point*.
+
+Running more A/B Tests like this, gathering more data, and then re-running this test may provide us, and the client more insight!
+
+
+
+
+___
+
+# Concept Overview
+
+
+#### A/B Testing
+
+An A/B Test can be described as a randomised experiment containing two groups, A & B, that receive different experiences. Within an A/B Test, we look to understand and measure the response of each group - and the information from this helps drive future business decisions.
+
+Application of A/B testing can range from testing different online ad strategies, different email subject lines when contacting customers, or testing the effect of mailing customers a coupon, vs a control group. Companies like Amazon are running these tests in an almost never-ending cycle, testing new website features on randomised groups of customers...all with the aim of finding what works best so they can stay ahead of their competition. Reportedly, Netflix will even test different images for the same movie or show, to different segments of their customer base to see if certain images pull more viewers in.
+
+
+#### Hypothesis Testing
+
+A Hypothesis Test is used to assess the plausibility, or likelihood of an assumed viewpoint based on sample data - in other words, it helps us assess whether a certain view we have about some data is likely to be true or not.
+
+There are many different scenarios we can run Hypothesis Tests on, and they all have slightly different techniques and formulas - however they all have some shared, fundamental steps & logic that underpin how they work.
+
+
+**The Null Hypothesis**
+
+In any Hypothesis Test, we start with the Null Hypothesis. The Null Hypothesis is where we state our initial viewpoint, and in statistics, and specifically Hypothesis Testing, our initial viewpoint is always that the result is purely by chance or that there is no relationship or association between two outcomes or groups
+
+
+**The Alternate Hypothesis**
+
+The aim of the Hypothesis Test is to look for evidence to support or reject the Null Hypothesis. If we reject the Null Hypothesis, that would mean we’d be supporting the Alternate Hypothesis. The Alternate Hypothesis is essentially the opposite viewpoint to the Null Hypothesis - that the result is *not* by chance, or that there *is* a relationship between two outcomes or groups
+
+
+**The Acceptance Criteria**
+
+In a Hypothesis Test, before we collect any data or run any numbers - we specify an Acceptance Criteria. This is a p-value threshold at which we’ll decide to reject or support the null hypothesis. It is essentially a line we draw in the sand saying "if I was to run this test many many times, what proportion of those times would I want to see different results come out, in order to feel comfortable, or confident that my results are not just some unusual occurrence"
+
+Conventionally, we set our Acceptance Criteria to 0.05 - but this does not have to be the case. If we need to be more confident that something did not occur through chance alone, we could lower this value down to something much smaller, meaning that we only come to the conclusion that the outcome was special or rare if it’s extremely rare.
+
+So to summarise, in a Hypothesis Test, we test the Null Hypothesis using a p-value and then decide its fate based on the Acceptance Criteria.
+
+
+**Types Of Hypothesis Test**
+
+There are many different types of Hypothesis Tests, each of which is appropriate for use in differing scenarios - depending on a) the type of data that you’re looking to test and b) the question that you’re asking of that data.
+
+In the case of our task here, where we are looking to understand the difference in sign-up *rate* between two groups - we will utilise the Chi-Square Test For Independence.
+
+
+#### Chi-Square Test For Independence
+
+The Chi-Square Test For Independence is a type of Hypothesis Test that assumes observed frequencies for categorical variables will match the expected frequencies.
+
+The *assumption* is the Null Hypothesis, which as discussed above is always the viewpoint that the two groups will be equal. With the Chi-Square Test For Independence we look to calculate a statistic which, based on the specified Acceptance Criteria will mean we either reject or support this initial assumption.
+
+The *observed frequencies* are the true values that we’ve seen.
+
+The *expected frequencies* are essentially what we would *expect* to see based on all of the data.
+
+**Note:** Another option when comparing "rates" is a test known as the *Z-Test For Proportions*. While, we could absolutely use this test here, we have chosen the Chi-Square Test For Independence because:
+
+* The resulting test statistic for both tests will be the same
+* The Chi-Square Test can be represented using 2x2 tables of data - meaning it can be easier to explain to stakeholders
+* The Chi-Square Test can extend out to more than 2 groups - meaning the business can have one consistent approach to measuring significance
+
+___
+
+
+# Data Overview & Preparation
+
+In the client database, we have a *campaign_data* table which shows us which customers received each type of "Delivery Club" mailer, which customers were in the control group, and which customers joined the club as a result.
+
+For this task, we are looking to find evidence that the Delivery Club signup rate for customers that received "Mailer 1" (low cost) was different to those who received "Mailer 2" (high cost) and thus from the *campaign_data* table we will just extract customers in those two groups, and exclude customers who were in the control group.
+
+In the code below, we:
+
+* Load in the Python libraries we require for importing the data and performing the chi-square test (using scipy)
+* Import the required data from the *campaign_data* table
+* Exclude customers in the control group, giving us a dataset with Mailer 1 & Mailer 2 customers only
+
+
+```python
+
+# install the required python libraries
+import pandas as pd
+from scipy.stats import chi2_contingency, chi2
+
+# import campaign data
+campaign_data = ...
+
+# remove customers who were in the control group
+campaign_data = campaign_data.loc[campaign_data["mailer_type"] != "Control"]
+
+```
+
+A sample of this data (the first 10 rows) can be seen below:
+
+
+
+| **customer_id** | **campaign_name** | **mailer_type** | **signup_flag** |
+|---|---|---|---|
+| 74 | delivery_club | Mailer1 | 1 |
+| 524 | delivery_club | Mailer1 | 1 |
+| 607 | delivery_club | Mailer2 | 1 |
+| 343 | delivery_club | Mailer1 | 0 |
+| 322 | delivery_club | Mailer2 | 1 |
+| 115 | delivery_club | Mailer2 | 0 |
+| 1 | delivery_club | Mailer2 | 1 |
+| 120 | delivery_club | Mailer1 | 1 |
+| 52 | delivery_club | Mailer1 | 1 |
+| 405 | delivery_club | Mailer1 | 0 |
+| 435 | delivery_club | Mailer2 | 0 |
+
+
+In the DataFrame we have:
+
+* customer_id
+* campaign name
+* mailer_type (either Mailer1 or Mailer2)
+* signup_flag (either 1 or 0)
+
+___
+
+
+# Applying Chi-Square Test For Independence
+
+
+#### State Hypotheses & Acceptance Criteria For Test
+
+The very first thing we need to do in any form of Hypothesis Test is state our Null Hypothesis, our Alternate Hypothesis, and the Acceptance Criteria (more details on these in the section above)
+
+In the code below we code these in explicitly & clearly so we can utilise them later to explain the results. We specify the common Acceptance Criteria value of 0.05.
+
+```python
+
+# specify hypotheses & acceptance criteria for test
+null_hypothesis = "There is no relationship between mailer type and signup rate. They are independent"
+alternate_hypothesis = "There is a relationship between mailer type and signup rate. They are not independent"
+acceptance_criteria = 0.05
+
+```
+
+
+#### Calculate Observed Frequencies & Expected Frequencies
+
+As mentioned in the section above, in a Chi-Square Test For Independence, the *observed frequencies* are the true values that we’ve seen, in other words the actual rates per group in the data itself. The *expected frequencies* are what we would *expect* to see based on *all* of the data combined.
+
+The below code:
+
+* Summarises our dataset to a 2x2 matrix for *signup_flag* by *mailer_type*
+* Based on this, calculates the:
+ * Chi-Square Statistic
+ * p-value
+ * Degrees of Freedom
+ * Expected Values
+* Prints out the Chi-Square Statistic & p-value from the test
+* Calculates the Critical Value based upon our Acceptance Criteria & the Degrees Of Freedom
+* Prints out the Critical Value
+
+```python
+
+# aggregate our data to get observed values
+observed_values = pd.crosstab(campaign_data["mailer_type"], campaign_data["signup_flag"]).values
+
+# run the chi-square test
+chi2_statistic, p_value, dof, expected_values = chi2_contingency(observed_values, correction = False)
+
+# print chi-square statistic
+print(chi2_statistic)
+>> 1.94
+
+# print p-value
+print(p_value)
+>> 0.16
+
+# find the critical value for our test
+critical_value = chi2.ppf(1 - acceptance_criteria, dof)
+
+# print critical value
+print(critical_value)
+>> 3.84
+
+```
+
+Based upon our observed values, we can give this all some context with the sign-up rate of each group. We get:
+
+* Mailer 1 (Low Cost): **32.8%** signup rate
+* Mailer 2 (High Cost): **37.8%** signup rate
+
+From this, we can see that the higher cost mailer does lead to a higher signup rate. The results from our Chi-Square Test will provide us more information about how confident we can be that this difference is robust, or if it might have occurred by chance.
+
+We have a Chi-Square Statistic of **1.94** and a p-value of **0.16**. The critical value for our specified Acceptance Criteria of 0.05 is **3.84**
+
+**Note** When applying the Chi-Square Test above, we use the parameter *correction = False* which means we are not applying what is known as the *Yates' Correction* which is applied when your Degrees of Freedom is equal to one. This correction helps to prevent overestimation of statistical signficance in this case.
+
+___
+
+
+# Analysing The Results
+
+At this point we have everything we need to understand the results of our Chi-Square test - and just from the results above we can see that, since our resulting p-value of **0.16** is *greater* than our Acceptance Criteria of 0.05 then we will _retain_ the Null Hypothesis and conclude that there is no significant difference between the signup rates of Mailer 1 and Mailer 2.
+
+We can make the same conclusion based upon our resulting Chi-Square statistic of **1.94** being _lower_ than our Critical Value of **3.84**
+
+To make this script more dynamic, we can create code to automatically interpret the results and explain the outcome to us...
+
+```python
+
+# print the results (based upon p-value)
+if p_value <= acceptance_criteria:
+ print(f"As our p-value of {p_value} is lower than our acceptance_criteria of {acceptance_criteria} - we reject the null hypothesis, and conclude that: {alternate_hypothesis}")
+else:
+ print(f"As our p-value of {p_value} is higher than our acceptance_criteria of {acceptance_criteria} - we retain the null hypothesis, and conclude that: {null_hypothesis}")
+
+>> As our p-value of 0.16351 is higher than our acceptance_criteria of 0.05 - we retain the null hypothesis, and conclude that: There is no relationship between mailer type and signup rate. They are independent
+
+
+# print the results (based upon p-value)
+if chi2_statistic >= critical_value:
+ print(f"As our chi-square statistic of {chi2_statistic} is higher than our critical value of {critical_value} - we reject the null hypothesis, and conclude that: {alternate_hypothesis}")
+else:
+ print(f"As our chi-square statistic of {chi2_statistic} is lower than our critical value of {critical_value} - we retain the null hypothesis, and conclude that: {null_hypothesis}")
+
+>> As our chi-square statistic of 1.9414 is lower than our critical value of 3.841458820694124 - we retain the null hypothesis, and conclude that: There is no relationship between mailer type and signup rate. They are independent
+
+```
+
+As we can see from the outputs of these print statements, we do indeed retain the null hypothesis. We could not find enough evidence that the signup rates for Mailer 1 and Mailer 2 were different - and thus conclude that there was no significant difference.
+
+___
+
+
+# Discussion
+
+While we saw that the higher cost Mailer 2 had a higher signup rate (37.8%) than the lower cost Mailer 1 (32.8%) it appears that this difference is not significant, at least at our Acceptance Criteria of 0.05.
+
+Without running this Hypothesis Test, the client may have concluded that they should always look to go with higher cost mailers - and from what we've seen in this test, that may not be a great decision. It would result in them spending more, but not *necessarily* gaining any extra revenue as a result
+
+Our results here also do not say that there *definitely isn't a difference between the two mailers* - we are only advising that we should not make any rigid conclusions *at this point*.
+
+Running more A/B Tests like this, gathering more data, and then re-running this test may provide us, and the client more insight!
diff --git a/_posts/2026-05-25-predicting-customer-loyalty.md b/_posts/2026-05-25-predicting-customer-loyalty.md
new file mode 100644
index 000000000..bca6d7057
--- /dev/null
+++ b/_posts/2026-05-25-predicting-customer-loyalty.md
@@ -0,0 +1,1181 @@
+---
+layout: post
+title: Predicting Customer Loyalty Using ML
+image: "/posts/regression-title-img.png"
+tags: [Customer Loyalty, Machine Learning, Regression, Python]
+---
+
+My client, a grocery retailer, hired a market research consultancy to append market level customer loyalty information to the database. However, only around 50% of the client's customer base could be tagged, thus the other half did not have this information present. I'll use ML to solve this!
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results](#overview-results)
+ - [Growth/Next Steps](#overview-growth)
+ - [Key Definition](#overview-definition)
+- [01. Data Overview](#data-overview)
+- [02. Modeling Overview](#modelling-overview)
+- [03. Linear Regression](#linreg-title)
+- [04. Decision Tree](#regtree-title)
+- [05. Random Forest](#rf-title)
+- [06. Modeling Summary](#modelling-summary)
+- [07. Predicting Missing Loyalty Scores](#modeling-predictions)
+- [08. Growth & Next Steps](#growth-next-steps)
+
+___
+
+# Project Overview
+
+### Context
+
+My client, a grocery retailer, hired a market research consultancy to append market level customer loyalty information to the database. However, only around 50% of the client's customer base could be tagged, thus the other half did not have this information present.
+
+The overall aim of this work is to accurately predict the *loyalty score* for those customers who could not be tagged, enabling my client a clear understanding of true customer loyalty, regardless of total spend volume - and allowing for more accurate and relevant customer tracking, targeting, and comms.
+
+To achieve this, I looked to build out a predictive model that will find relationships between customer metrics and *loyalty score* for those customers who were tagged, and use this to predict the loyalty score metric for those who were not.
+
+
+### Actions
+
+I firstly needed to compile the necessary data from tables in the database, gathering key customer metrics that may help predict *loyalty score*, appending on the dependent variable, and separating out those who did and did not have this dependent variable present.
+
+As I am predicting a numeric output, I tested three regression modeling approaches, namely:
+
+* Linear Regression
+* Decision Tree
+* Random Forest
+
+
+
+### Results
+
+My testing found that the Random Forest had the highest predictive accuracy.
+
+
+**Metric 1: Adjusted R-Squared (Test Set)**
+
+* Random Forest = 0.955
+* Decision Tree = 0.886
+* Linear Regression = 0.754
+
+
+**Metric 2: R-Squared (K-Fold Cross Validation, k = 4)**
+
+* Random Forest = 0.925
+* Decision Tree = 0.871
+* Linear Regression = 0.853
+
+As the most important outcome for this project was predictive accuracy, rather than explicitly understanding weighted drivers of prediction, I chose the Random Forest as the model to use for making predictions on the customers who were missing the *loyalty score* metric.
+
+
+### Growth/Next Steps
+
+While predictive accuracy was relatively high - other modelling approaches could be tested, especially those somewhat similar to Random Forest, for example XGBoost, LightGBM to see if even more accuracy could be gained.
+
+From a data point of view, further variables could be collected, and further feature engineering could be undertaken to ensure that we have as much useful information available for predicting customer loyalty
+
+
+### Key Definition
+
+The *loyalty score* metric measures the % of grocery spend (market level) that each customer allocates to the client vs. all of the competitors.
+
+Example 1: Customer X has a total grocery spend of $100 and all of this is spent with our client. Customer X has a *loyalty score* of 1.0
+
+Example 2: Customer Y has a total grocery spend of $200 but only 20% is spent with our client. The remaining 80% is spend with competitors. Customer Y has a *customer loyalty score* of 0.2
+
+
+___
+
+# Data Overview
+
+We will be predicting the *loyalty_score* metric. This metric exists (for approximately half of the customer base) in the *loyalty_scores* table of the client database.
+
+The key variables hypothesized to predict the missing loyalty scores will come from the client database, namely the *transactions* table, the *customer_details* table, and the *product_areas* table.
+
+Using pandas in Python, I merged these tables together for all customers, creating a single dataset that we can use for modeling.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+
+# import required data tables
+loyalty_scores = pd.read_excel("data/grocery_data.xlsx", sheet_name = "loyalty_scores")
+customer_details = pd.read_excel("data/grocery_data.xlsx", sheet_name = "customer_details")
+transactions = pd.read_excel("data/grocery_data.xlsx", sheet_name = "transactions")
+# loyalty_scores returns 400 rows, customer_details returns 870 rows
+
+# merge loyalty score data and customer details data, at customer level. customer_details is the base.
+# Later we'll split this into 2 datasets for training and testing
+data_for_regression = pd.merge(customer_details, loyalty_scores, how = "left", on = "customer_id")
+
+# aggregate sales data from transactions table. Create 5 sales metrics.
+sales_summary = transactions.groupby("customer_id").agg({"sales_cost" : "sum",
+ "num_items" : "sum",
+ "transaction_id" : "count",
+ "product_area_id" : "nunique"}).reset_index()
+
+# rename columns for clarity
+sales_summary.columns = ["customer_id", "total_sales", "total_items", "transaction_count", "product_area_count"]
+
+# engineer an average basket value column for each customer
+sales_summary["average_basket_value"] = sales_summary["total_sales"] / sales_summary["transaction_count"]
+
+# Overwrite data_for_regression and merge the sales summary with the overall customer data
+data_for_regression = pd.merge(data_for_regression, sales_summary, how = "inner", on = "customer_id")
+# Our new "data_for_regression" has 10 columns including customer_id
+
+# split out data for modeling (loyalty score is present or not)
+regression_modeling = data_for_regression.loc[data_for_regression["customer_loyalty_score"].notna()]
+
+# split out data for scoring post-modeling (loyalty score is missing)
+regression_scoring = data_for_regression.loc[data_for_regression["customer_loyalty_score"].isna()]
+
+# for scoring set, drop the loyalty score column (as it is blank/redundant)
+regression_scoring.drop(["customer_loyalty_score"], axis = 1, inplace = True)
+
+# save our datasets for future use
+pickle.dump(regression_modeling, open("data/customer_loyalty_modeling.p", "wb"))
+pickle.dump(regression_scoring, open("data/customer_loyalty_scoring.p", "wb"))
+
+```
+
+After this data pre-processing in Python, we have a dataset for modeling that contains the following fields...
+
+
+
+| **Variable Name** | **Variable Type** | **Description** |
+|---|---|---|
+| loyalty_score | Dependent | The % of total grocery spend that each customer allocates to ABC Grocery vs. competitors |
+| distance_from_store | Independent | "The distance in miles from the customers home address, and the store" |
+| gender | Independent | The gender provided by the customer |
+| credit_score | Independent | The customers most recent credit score |
+| total_sales | Independent | Total spend by the customer in ABC Grocery within the latest 6 months |
+| total_items | Independent | Total products purchased by the customer in ABC Grocery within the latest 6 months |
+| transaction_count | Independent | Total unique transactions made by the customer in ABC Grocery within the latest 6 months |
+| product_area_count | Independent | The number of product areas within ABC Grocery the customers has shopped into within the latest 6 months |
+| average_basket_value | Independent | The average spend per transaction for the customer in ABC Grocery within the latest 6 months |
+
+___
+
+# Modeling Overview
+
+I will build a model that looks to accurately predict the “loyalty_score” metric for those customers that were able to be tagged, based upon the customer metrics listed above.
+
+If that can be achieved, I can use this model to predict the customer loyalty score for the customers that were unable to be tagged by the agency.
+
+As I am predicting a numeric output, we tested three regression modeling approaches, namely:
+
+* Linear Regression
+* Decision Tree
+* Random Forest
+
+___
+
+# Linear Regression
+
+I utilized the scikit-learn library within Python to model our data using Linear Regression. The code sections below are broken up into 4 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+
+
+### Data Import
+
+Since I saved our modeling data as a pickle file, I'll import it. I will ensure the id column is removed, and I'll also ensure the data is shuffled.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+from sklearn.linear_model import LinearRegression
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import r2_score
+from sklearn.preprocessing import OneHotEncoder
+from sklearn.feature_selection import RFECV
+
+# import modeling data
+data_for_model = pickle.load(open("data/customer_loyalty_modeling.p", "rb"))
+
+# drop uneccessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+```
+
+### Data Preprocessing
+
+For Linear Regression there are certain data preprocessing steps that need to be addressed, including:
+
+* Missing values in the data
+* The effect of outliers (for Linear)
+* Encoding categorical variables to numeric form
+* Multicollinearity & Feature Selection
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) I'll just drop those rows
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Outliers
+
+The ability for a Linear Regression model to generalize well across *all* data can be hampered if there are outliers present. There is no right or wrong way to deal with outliers, but it is always something worth very careful consideration - just because a value is high or low, does not necessarily mean it should not be there!
+
+In this code section, I'll use **.describe()** from Pandas to investigate the spread of values for each of our predictors. The results of this can be seen in the table below.
+
+
+
+| **metric** | **distance_from_store** | **credit_score** | **total_sales** | **total_items** | **transaction_count** | **product_area_count** | **average_basket_value** |
+|---|---|---|---|---|---|---|---|
+| mean | 2.02 | 0.60 | 1846.50 | 278.30 | 44.93 | 4.31 | 36.78 |
+| std | 2.57 | 0.10 | 1767.83 | 214.24 | 21.25 | 0.73 | 19.34 |
+| min | 0.00 | 0.26 | 45.95 | 10.00 | 4.00 | 2.00 | 9.34 |
+| 25% | 0.71 | 0.53 | 942.07 | 201.00 | 41.00 | 4.00 | 22.41 |
+| 50% | 1.65 | 0.59 | 1471.49 | 258.50 | 50.00 | 4.00 | 30.37 |
+| 75% | 2.91 | 0.66 | 2104.73 | 318.50 | 53.00 | 5.00 | 47.21 |
+| max | 44.37 | 0.88 | 9878.76 | 1187.00 | 109.00 | 5.00 | 102.34 |
+
+
+Based on this investigation, some *max* column values for several variables are much higher than the *median* value.
+
+This is for columns *distance_from_store*, *total_sales*, and *total_items*
+
+For example, the median *distance_to_store* is 1.645 miles, but the maximum is over 44 miles!
+
+Because of this, I'll apply some outlier removal in order to facilitate generalization across the full dataset.
+
+I'll do this using the "boxplot approach" where I'll remove any rows where the values within those columns are outside of the interquartile range multiplied by 2.
+
+
+```python
+
+outlier_investigation = data_for_model.describe()
+outlier_columns = ["distance_from_store", "total_sales", "total_items"]
+
+# boxplot approach
+for column in outlier_columns:
+
+ lower_quartile = data_for_model[column].quantile(0.25)
+ upper_quartile = data_for_model[column].quantile(0.75)
+ iqr = upper_quartile - lower_quartile
+ iqr_extended = iqr * 2
+ min_border = lower_quartile - iqr_extended
+ max_border = upper_quartile + iqr_extended
+
+ outliers = data_for_model[(data_for_model[column] < min_border) | (data_for_model[column] > max_border)].index
+ print(f"{len(outliers)} outliers detected in column {column}")
+
+ data_for_model.drop(outliers, inplace = True)
+
+```
+
+
+##### Split Out Data For Modeling
+
+In the next code block I'll do two things, firstly split the data into an **X** object which contains only the predictor variables, and a **y** object that contains only the dependent variable.
+
+Once I have done this, I split the data into "training" and "test" sets to ensure I can fairly validate the accuracy of the predictions on data that was not used in training. In this case, we have allocated 80% of the data for training, and the remaining 20% for validation.
+
+
+```python
+
+# split data into X and y objects for modeling
+X = data_for_model.drop(["customer_loyalty_score"], axis = 1)
+y = data_for_model["customer_loyalty_score"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)
+
+```
+
+
+##### Categorical Predictor Variables
+
+In the dataset, there is one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+The Linear Regression algorithm can't deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it, in other words, Male isn't higher or lower than Female and vice versa - one appropriate approach is to apply "One Hot Encoding" to the categorical column.
+
+One Hot Encoding can be thought of as a way to represent categorical variables as binary vectors, in other words, a set of *new* columns for each categorical value with either a 1 or a 0 saying whether that value is true or not for that observation. These new columns would go into our model as input variables, and the original column is discarded.
+
+I also drop one of the new columns using the parameter *drop = "first"*. This is done to avoid the *dummy variable trap* where the newly created encoded columns perfectly predict each other, running the risk of breaking the assumption that there is no multicollinearity - a requirement or at least an important consideration for some models, Linear Regression being one of them! Multicollinearity occurs when two or more input variables are *highly* correlated with each other. It is a scenario to attempt to avoid. While it won't necessarily affect the predictive accuracy of our model, it can make it difficult to trust the statistics around how well the model is performing, and how much each input variable is truly having.
+
+In the code, I also make sure to apply *fit_transform* to the training set, but only *transform* to the test set. This means the One Hot Encoding logic will *learn and apply* the "rules" from the training data, but only *apply* them to the test data. This is important in order to avoid *data leakage* where the test set *learns* information about the training data, and means we can't fully trust model performance metrics!
+
+For ease, after applying One Hot Encoding, training and test objects can be turned back into Pandas Dataframes, with the column names applied.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse_output=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns (array)
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to Pandas DataFrame
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+
+# Create a new DataFrame by concatonating the original to the encoded DataFrame
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+# Drop original input 2 and input 3 variables
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+# Test data
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+# Drop old "gender"
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+##### Feature Selection using RFE (Recursive Feature Elimination)
+
+Feature Selection is the process used to select the input variables that are most important to your Machine Learning task. It can be a very important addition or at least, consideration, in certain scenarios. The potential benefits of Feature Selection are:
+
+* **Improved Model Accuracy** - eliminating noise can help true relationships stand out
+* **Lower Computational Cost** - our model becomes faster to train, and faster to make predictions
+* **Explainability** - understanding & explaining outputs for stakeholders & customers becomes much easier
+
+There are multiple ways to apply Feature Selection. These range from simple methods such as a *Correlation Matrix* showing variable relationships, to *Univariate Testing* which helps us understand statistical relationships between variables, and then to even more powerful approaches like *Recursive Feature Elimination (RFE)* which is an approach that starts with all input variables and then iteratively removes those with the weakest relationships to the output variable.
+
+For this task, I applied a variation of Recursive Feature Elimination called *Recursive Feature Elimination With Cross Validation (RFECV)* where the data is split into many "chunks" and iteratively trains & validates models on each "chunk" separately. This means that each time we assess different models with different variables included, or eliminated, the algorithm also knows how accurate each of those models was. From the suite of model scenarios that are created, the algorithm can determine which provided the best accuracy, and thus can infer the best set of input variables to use!
+
+
+```python
+
+# instantiate RFECV & the model type to be utilized
+regressor = LinearRegression()
+feature_selector = RFECV(regressor)
+
+# fit RFECV onto our training & test data
+fit = feature_selector.fit(X_train,y_train)
+
+# extract & print the optimal number of features
+optimal_feature_count = feature_selector.n_features_
+print(f"Optimal number of features: {optimal_feature_count}")
+
+# limit our training & test sets to only include the selected variables
+X_train = X_train.loc[:, feature_selector.get_support()]
+X_test = X_test.loc[:, feature_selector.get_support()]
+
+```
+
+
+The below code then produces a plot that visualizes the cross-validated accuracy with each potential number of features
+
+```python
+
+plt.style.use('seaborn-poster')
+plt.plot(range(1, len(fit.cv_results_['mean_test_score']) + 1), fit.cv_results_['mean_test_score'], marker = "o")
+plt.ylabel("Model Score")
+plt.xlabel("Number of Features")
+plt.title(f"Feature Selection using RFE \n Optimal number of features is {optimal_feature_count} (at score of {round(max(fit.cv_results_['mean_test_score']),4)})")
+plt.tight_layout()
+plt.show()
+
+```
+
+
+This creates the below plot, which shows that the highest cross-validated accuracy (0.8635) is actually when all eight of the original input variables are included. This is marginally higher than 6 included variables, and 7 included variables. We will continue on with all 8!
+
+
+
+
+
+### Model Training
+
+Instantiating and training the Linear Regression model is done using the below code
+
+```python
+
+# instantiate our model object
+regressor = LinearRegression()
+
+# fit our model using our training & test sets
+regressor.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+To assess how well the model is predicting on new data - the trained model object (here called *regressor*) is used and asked to predict the *loyalty_score* variable for the test set
+
+```python
+
+# predict on the test set
+y_pred = regressor.predict(X_test)
+
+```
+
+
+##### Calculate R-Squared
+
+R-Squared is a metric that shows the percentage of variance in our output variable *y* that is being explained by our input variable(s) *x*. It is a value that ranges between 0 and 1, with a higher value showing a higher level of explained variance. Another way of explaining this would be to say that, if we had an r-squared score of 0.8 it would suggest that 80% of the variation of the output variable is being explained by the input variables - and something else, or some other variables must account for the other 20%
+
+To calculate r-squared, I use the following code where I pass in our *predicted* outputs for the test set (y_pred) as well as the *actual* outputs for the test set (y_test)
+
+```python
+
+# calculate r-squared for our test set predictions
+r_squared = r2_score(y_test, y_pred)
+print(r_squared)
+
+```
+
+The resulting r-squared score from this is **0.78**
+
+
+##### Calculate Cross Validated R-Squared
+
+An even more powerful and reliable way to assess model performance is to utilize Cross Validation.
+
+Instead of simply dividing the data into a single training set, and a single test set, with Cross Validation, the data can be broken into a number of "chunks" and then iteratively train the model on all but one of the "chunks", test the model on the remaining "chunk" until each has had a chance to be the test set.
+
+The result of this is that a number of test set validation results is provided - and the average of these is calculated to give a much more robust & reliable view of how the model will perform on new, un-seen data!
+
+In the code below, this is put into place. First, 4 "chunks" is specified, and then we pass in the regressor object, the training set, and the test set. Also specified is the metric with which to assess, in this case, r-squared.
+
+Finally, a mean of all four test set results is calculated.
+
+```python
+
+# calculate the mean cross validated r-squared for our test set predictions
+cv = KFold(n_splits = 4, shuffle = True, random_state = 42)
+cv_scores = cross_val_score(regressor, X_train, y_train, cv = cv, scoring = "r2")
+cv_scores.mean()
+
+```
+
+The mean cross-validated r-squared score from this is **0.853**
+
+
+##### Calculate Adjusted R-Squared
+
+When applying Linear Regression with *multiple* input variables, the r-squared metric on it's own *can* end up being an overinflated view of goodness of fit. This is because each input variable will have an *additive* effect on the overall r-squared score. In other words, every input variable added to the model *increases* the r-squared value, and *never decreases* it, even if the relationship is by chance.
+
+**Adjusted R-Squared** is a metric that compensates for the addition of input variables, and only increases if the variable improves the model above what would be obtained by probability. It is best practice to use Adjusted R-Squared when assessing the results of a Linear Regression with multiple input variables, as it gives a more fair perception the fit of the data.
+
+```python
+
+# calculate adjusted r-squared for our test set predictions
+num_data_points, num_input_vars = X_test.shape
+adjusted_r_squared = 1 - (1 - r_squared) * (num_data_points - 1) / (num_data_points - num_input_vars - 1)
+print(adjusted_r_squared)
+
+```
+
+The resulting *adjusted* r-squared score from this is **0.754** which as expected, is slightly lower than the score we got for r-squared on it's own.
+
+
+### Model Summary Statistics
+
+Although the overall goal for this project is predictive accuracy, rather than an explicit understanding of the relationships of each of the input variables and the output variable, it is always interesting to look at the summary statistics for these.
+
+```python
+
+# extract model coefficients and create a DataFrame
+coefficients = pd.DataFrame(regressor.coef_)
+# Make DataFrame more useful by adding names
+input_variable_names = pd.DataFrame(X_train.columns)
+summary_stats = pd.concat([input_variable_names,coefficients], axis = 1)
+summary_stats.columns = ["input_variable", "coefficient"]
+
+# Values in the DataFrame will make up the values going into the equation for the "line of best fit" or
+# technically the "plane of best fit"
+
+# extract model intercept
+regressor.intercept_
+
+```
+
+The information from that code block can be found in the table below:
+
+
+| **input_variable** | **coefficient** |
+|---|---|
+| intercept | 0.516 |
+| distance_from_store | -0.201 |
+| credit_score | -0.028 |
+| total_sales | 0.000 |
+| total_items | 0.001 |
+| transaction_count | -0.005 |
+| product_area_count | 0.062 |
+| average_basket_value | -0.004 |
+| gender_M | -0.013 |
+
+
+The coefficient value for each of the input variables, along with that of the intercept would make up the equation for the line of best fit for this particular model (or more accurately, in this case it would be the plane of best fit, as we have multiple input variables).
+
+For each input variable, the coefficient value above tells us, with *everything else staying constant* , how many units the output variable (loyalty score) would change with a *one unit change* in this particular input variable.
+
+To provide an example of this using the table above, we can see that the *distance_from_store* input variable has a coefficient value of -0.201. This is saying that *loyalty_score* decreases by 0.201 (or 20% as loyalty score is a percentage, or at least a decimal value between 0 and 1) for *every additional mile* that a customer lives from the store. This makes intuitive sense, as customers who live a long way from this store most likely live near *another* store where they might do some of their shopping as well. Whereas, customers who live near this store, probably do a greater proportion of their shopping at "this" store and hence have a higher loyalty score!
+
+___
+
+# Decision Tree
+
+We will again utilize the scikit-learn library within Python to model our data using a Decision Tree. The code sections below are broken up into 4 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+
+
+### Data Import
+
+Since the modeling data was saved as a pickle file, it is imported. Next, the id column is removed and our data is shuffled.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+from sklearn.tree import DecisionTreeRegressor, plot_tree
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import r2_score
+from sklearn.preprocessing import OneHotEncoder
+
+# import modeling data
+data_for_model = pickle.load(open("data/customer_loyalty_modeling.p", "rb"))
+
+# drop uneccessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+```
+
+### Data Preprocessing
+
+While Linear Regression is susceptible to the effects of outliers and highly correlated input variables, Decision Trees are not so the required preprocessing here is lighter. We still; however, will put in place logic for:
+
+* Missing values in the data
+* Encoding categorical variables to numeric form
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) we will just remove those rows
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Split Out Data For Modeling
+
+In exactly the same way we did for Linear Regression, in the next code block two things are done, firstly the data is split into an **X** object which contains only the predictor variables, and a **y** object that contains only our dependent variable.
+
+Once this is complete, the data is split into training and test sets to ensure fairl validation and the accuracy of the predictions on data that was not used in training. In this case, 80% of the data is allocated for training, and the remaining 20% for validation.
+
+
+```python
+
+# split data into X and y objects for modeling
+X = data_for_model.drop(["customer_loyalty_score"], axis = 1)
+y = data_for_model["customer_loyalty_score"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)
+
+```
+
+
+##### Categorical Predictor Variables
+
+In our dataset, we have one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+Just like the Linear Regression algorithm, the Decision Tree cannot deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it; Male isn't higher or lower than Female and vice versa, One Hot Encoding is again applied to the categorical column.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse_output=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to Pandas DataFrame
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+### Model Training
+
+Instantiating and training the Decision Tree model is done using the below code. The *random_state* parameter is used to ensure reproducible results, and this helps us understand any improvements in performance with changes to model hyperparameters.
+
+```python
+
+# instantiate our model object
+regressor = DecisionTreeRegressor(random_state = 42)
+
+# fit our model using our training & test sets
+regressor.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+To assess how well our model is predicting on new data, we use the trained model object, *regressor*, and ask it to predict the *loyalty_score* variable for the test set
+
+```python
+
+# predict on the test set
+y_pred = regressor.predict(X_test)
+
+```
+
+
+##### Calculate R-Squared
+
+To calculate r-squared, the following code is used where the *predicted* outputs are passed in for the test set (y_pred), as well as the *actual* outputs for the test set (y_test)
+
+```python
+
+# calculate r-squared for our test set predictions
+r_squared = r2_score(y_test, y_pred)
+print(r_squared)
+
+```
+
+The resulting r-squared score from this is **0.898**
+
+
+##### Calculate Cross Validated R-Squared
+
+As we did when testing Linear Regression, Cross Validation is again used.
+
+Instead of simply dividing the data into a single training set and a single test set, with Cross Validation the data is broken into a number of "chunks" and then iteratively used to train the model on all but one of the "chunks". The model isn't tested on the remaining "chunk" until each has had a chance to be the test set.
+
+The result of this is that a number of test set validation results is provided and the average of these can be calculated to give a much more robust & reliable view of how our model will perform on new, un-seen data!
+
+In the code below, this into place. Again, 4 "chunks" is specified in the KFold parameter "n_splits" and then the regressor object, training set, and test set are passed in to it. Also specified is the metric with which we want to assess the model, in this case, r-squared is again used.
+
+Finally, a mean of all four test set results is calculated.
+
+```python
+
+# calculate the mean cross validated r-squared for our test set predictions
+cv = KFold(n_splits = 4, shuffle = True, random_state = 42)
+cv_scores = cross_val_score(regressor, X_train, y_train, cv = cv, scoring = "r2")
+cv_scores.mean()
+
+```
+
+The mean cross-validated r-squared score from this is **0.871** which is slighter higher than calculated for Linear Regression.
+
+
+##### Calculate Adjusted R-Squared
+
+Just like Linear Regression, the *Adjusted R-Squared* is calculated compensating for the addition of input variables, and only increaseing if the variable improves the model above what would be obtained by probability.
+
+```python
+
+# calculate adjusted r-squared for our test set predictions
+num_data_points, num_input_vars = X_test.shape
+adjusted_r_squared = 1 - (1 - r_squared) * (num_data_points - 1) / (num_data_points - num_input_vars - 1)
+print(adjusted_r_squared)
+
+```
+
+The resulting *adjusted* r-squared score from this is **0.887**, which as expected is slightly lower than the score we got for r-squared on it's own.
+
+
+### Decision Tree Regularization
+
+Decision Tree's can be prone to over-fitting, in other words, without any limits on their splitting, they will end up learning the training data perfectly. A more *generalized* set of rules is preferred as this will be more robust & reliable when making predictions on "new" data.
+
+One effective method of avoiding this over-fitting, is to apply a *max depth* to the Decision Tree, meaning it only allows it to split the data a certain number of times before it is required to stop.
+
+Unfortunately, the *best* number of splits to use for this is unknown, so below we will loop over a variety of values and assess which yields the best predictive performance!
+
+
+```python
+
+# finding the best "max_depth"
+
+# set up the range for search and an empty list to which to append the accuracy scores
+max_depth_list = list(range(1,9))
+accuracy_scores = []
+
+# loop through each possible depth, train and validate model, append test set accuracy
+for depth in max_depth_list:
+
+ regressor = DecisionTreeRegressor(max_depth = depth, random_state = 42)
+ regressor.fit(X_train,y_train)
+ y_pred = regressor.predict(X_test)
+ accuracy = r2_score(y_test,y_pred)
+ accuracy_scores.append(accuracy)
+
+# store "max accuracy" and "optimal depth"
+max_accuracy = max(accuracy_scores)
+max_accuracy_idx = accuracy_scores.index(max_accuracy)
+optimal_depth = max_depth_list[max_accuracy_idx]
+
+# plot "accuracy" by "max depth"
+plt.plot(max_depth_list,accuracy_scores)
+plt.scatter(optimal_depth, max_accuracy, marker = "x", color = "red")
+plt.title(f"Accuracy by Max Depth \n Optimal Tree Depth: {optimal_depth} (Accuracy: {round(max_accuracy,4)})")
+plt.xlabel("Max Depth of Decision Tree")
+plt.ylabel("Accuracy")
+plt.tight_layout()
+plt.show()
+
+```
+
+That code gives yields the below plot helping to visualize the results!
+
+
+
+
+
+In the plot, *maximum* classification accuracy on the test set is found when applying a *max_depth* value of 7. However, we lose very little accuracy using a value of 4 and this results in a simpler model that would generalize even better on new data. Considering how little accuracy is gained using "7", a "max_depth" value of 4 will be used to re-train our Decision Tree.
+
+
+### Visualize Our Decision Tree
+
+To see the decisions that have been made in the (re-fitted) tree, the "plot_tree" functionality that we imported from scikit-learn can be used. To do this, the below code is used:
+
+
+```python
+
+# re-fit the model using max depth of 4
+regressor = DecisionTreeRegressor(random_state = 42, max_depth = 4)
+regressor.fit(X_train, y_train)
+
+# plot the nodes of the decision tree
+plt.figure(figsize=(25,15))
+tree = plot_tree(regressor,
+ feature_names = list(X.columns),
+ filled = True,
+ rounded = True,
+ fontsize = 16)
+
+```
+
+That code gives us the below plot:
+
+
+
+
+
+This is a very powerful visual, and one that can be shown to stakeholders in the business to ensure they understand exactly what is driving the predictions.
+
+One interesting thing to note is that the *very first split* appears to be using the variable *distance from store* so it would seem that this is a very important variable when it comes to predicting loyalty!
+
+___
+
+# Random Forest
+
+Again, the scikit-learn library within Python is used to model our data using a Random Forest. The code sections below are broken up into 4 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+
+
+### Data Import
+
+Again, since the modeling data was saved as a pickle file, it will be imported. The id column is removed and the data is shuffled.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+from sklearn.ensemble import RandomForestRegressor
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import r2_score
+from sklearn.preprocessing import OneHotEncoder
+from sklearn.inspection import permutation_importance
+
+# import modeling data
+data_for_model = pickle.load(open("data/customer_loyalty_modeling.p", "rb"))
+
+# drop unnecessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+```
+
+### Data Preprocessing
+
+While Linear Regression is susceptible to the effects of outliers and highly correlated input variables, Random Forests, just like Decision Trees, are not. So, the required preprocessing here is lighter.
+However, the logic for the following is still necessary:
+
+* Missing values in the data
+* Encoding categorical variables to numeric form
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) those rows will just be removed
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Split Out Data For Modeling
+
+In exactly the same way done for Linear Regression, in the next code block two things are accomplished. Firstly, the data is split into an **X** object which contains only the predictor variables, and a **y** object that contains only our dependent variable.
+
+Once this is done, the data is split into training and test sets to ensure fair validation of the predictive accuracy on data that was not used in training. In this case, we have allocated 80% of the data for training and the remaining 20% for validation.
+
+
+```python
+
+# split data into X and y objects for modeling
+X = data_for_model.drop(["customer_loyalty_score"], axis = 1)
+y = data_for_model["customer_loyalty_score"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)
+
+```
+
+
+##### Categorical Predictor Variables
+
+In the dataset, there is one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+Just like the Linear Regression algorithm, Random Forests cannot deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it; Male isn't higher or lower than Female and vice versa, One Hot Encoding is applied to the categorical column.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse_output=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to Pandas DataFrame and drop original categorical data column
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+### Model Training
+
+Instantiating and training our Random Forest model is done using the below code. The *random_state* parameter is used to ensure reproducible results, helping to understand any improvements in performance with changes to model hyperparameters.
+
+Other parameters retain their default values, specifically meaning that the default of 100 Decision Trees will be built in this Random Forest.
+
+```python
+
+# instantiate our model object
+regressor = RandomForestRegressor(random_state = 42)
+
+# fit our model using our training & test sets
+regressor.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+To assess how well the model is predicting on new data, the trained model object, *regressor*, is used and asked to predict the *loyalty_score* variable for the test set
+
+```python
+
+# predict on the test set
+y_pred = regressor.predict(X_test)
+
+```
+
+
+##### Calculate R-Squared
+
+To calculate r-squared, the following code is used where *predicted* outputs for the test set (y_pred) is passed into it as well as the *actual* outputs for the test set (y_test)
+
+```python
+
+# calculate r-squared for our test set predictions
+r_squared = r2_score(y_test, y_pred)
+print(r_squared)
+
+```
+
+The resulting r-squared score from this is **0.957** - higher than both Linear Regression & the Decision Tree.
+
+
+##### Calculate Cross Validated R-Squared
+
+As done when testing Linear Regression & our Decision Tree, Cross Validation is utilized (for more info on how this works, please refer to the Linear Regression section above).
+
+```python
+
+# calculate the mean cross validated r-squared for our test set predictions
+cv = KFold(n_splits = 4, shuffle = True, random_state = 42)
+cv_scores = cross_val_score(regressor, X_train, y_train, cv = cv, scoring = "r2")
+cv_scores.mean()
+
+```
+
+The mean cross-validated r-squared score from this is **0.923** which agian is higher than calculated for both Linear Regression & the Decision Tree models.
+
+
+##### Calculate Adjusted R-Squared
+
+Just like we did with Linear Regression & Decision Tree, the *Adjusted R-Squared* is calculated compensating for the addition of input variables, and only increasing if the variable improves the model above what would be obtained by probability.
+
+```python
+
+# calculate adjusted r-squared for our test set predictions
+num_data_points, num_input_vars = X_test.shape
+adjusted_r_squared = 1 - (1 - r_squared) * (num_data_points - 1) / (num_data_points - num_input_vars - 1)
+print(adjusted_r_squared)
+
+```
+
+The resulting *adjusted* r-squared score from this is **0.955**, which as expected is slightly lower than the score obtained for r-squared on it's own but again higher than the other models.
+
+
+### Feature Importance
+
+To aid in understanding the relationships between input variables and our output variable "loyalty score", in the Linear Regression model, the coefficients were examined. With the Decision Tree, where data was split was analzed allowing us some insight into which input variables were having the most impact.
+
+Random Forests is an "ensemble model" made up of many Decision Trees, each of which is different due to the randomness of the data being provided and the random selection of input variables available at each potential split point.
+
+Because of this, a much more powerful and robust model is created. And, because of the random or different nature of all these Decision trees, the model gives us a unique insight into how important each of our input variables are to the overall model.
+
+As random samples of data and input variables for each Decision Tree is utilized, there are many scenarios where certain input variables are being held back enabling a way to compare how accurate the models predictions are if that variable is or isn’t present.
+
+So, at a high level in a Random Forest, we can measure *importance* by asking *How much would accuracy decrease if a specific input variable was removed or randomized?*
+
+If this decrease in performance or accuracy is large, then we’d deem that input variable to be quite important. And, if we see only a small decrease in accuracy, then we’d conclude that the variable is of less importance.
+
+At a high level, there are two common ways to tackle this. The first, often just called **Feature Importance**, is where all nodes in the Decision Trees of the forest where a particular input variable is used to split the data are found. Subsequently. they are assessed as to what the Mean Squared Error (for a Regression problem) was before the split was made and compared to the Mean Squared Error after the split was made. The *average* of these improvements is calculated across all Decision Trees in the Random Forest to obtain a score that indicates *how much better* the model functions by using that particular input variable.
+
+If this is done for *each* input variable, these scores can be compared to help understand which is adding the most value to the predictive power of the model.
+
+The other approach, often called **Permutation Importance**, cleverly uses some data that has gone *unused* when random samples are selected for each Decision Tree (this stage is called "bootstrap sampling" or "bootstrapping").
+
+These observations (data) that were not randomly selected for each Decision Tree are known as *Out of Bag* observations and can be used for "testing" the accuracy of each particular Decision Tree.
+
+For each Decision Tree, all of the *Out of Bag* observations are gathered and then passed through. Once all of these observations have been run through the Decision Tree, we obtain an accuracy score for these predictions. In the case of a regression problem this score could be Mean Squared Error or r-squared.
+
+In order to understand the *importance*, the values are *randomized* within one of the input variables. This is a process that essentially destroys any relationship that might exist between that input variable and the output variable. The updated data is run through the Decision Tree again, obtaining a second accuracy score. The difference between the original accuracy and the new accuracy gives us a view on how important that particular variable is for predicting the output.
+
+*Permutation Importance* is often preferred over *Feature Importance* which can at times inflate the importance of numerical features. Both are useful, and in most cases will give fairly similar results.
+
+Put both in place, and plot the results...
+
+
+```python
+
+# calculate feature importance
+feature_importance = pd.DataFrame(regressor.feature_importances_)
+feature_names = pd.DataFrame(X.columns)
+feature_importance_summary = pd.concat([feature_names,feature_importance], axis = 1)
+feature_importance_summary.columns = ["input_variable","feature_importance"]
+feature_importance_summary.sort_values(by = "feature_importance", inplace = True)
+
+# plot feature importance
+plt.barh(feature_importance_summary["input_variable"],feature_importance_summary["feature_importance"])
+plt.title("Feature Importance of Random Forest")
+plt.xlabel("Feature Importance")
+plt.tight_layout()
+plt.show()
+
+# calculate permutation importance
+result = permutation_importance(regressor, X_test, y_test, n_repeats = 10, random_state = 42)
+permutation_importance = pd.DataFrame(result["importances_mean"])
+feature_names = pd.DataFrame(X.columns)
+permutation_importance_summary = pd.concat([feature_names,permutation_importance], axis = 1)
+permutation_importance_summary.columns = ["input_variable","permutation_importance"]
+permutation_importance_summary.sort_values(by = "permutation_importance", inplace = True)
+
+# plot permutation importance
+plt.barh(permutation_importance_summary["input_variable"],permutation_importance_summary["permutation_importance"])
+plt.title("Permutation Importance of Random Forest")
+plt.xlabel("Permutation Importance")
+plt.tight_layout()
+plt.show()
+
+```
+
+The above code creates the "plots" below. The first is *Feature Importance* and the second is *Permutation Importance*.
+
+
+
+
+
+
+
+
+The overall story from both approaches is very similar, in that by far the most important or impactful input variable is *distance_from_store* which is the same insight derived when assessing the Linear Regression & Decision Tree models.
+
+There are slight differences in the order or "importance" for the remaining variables but overall they have provided similar findings.
+
+___
+
+# Modeling Summary
+
+The most important outcome for this project was predictive accuracy rather than explicitly understanding the drivers of prediction. Based upon this, the model that performed the best when predicted on the test set was the Random Forest model.
+
+
+**Metric 1: Adjusted R-Squared (Test Set)**
+
+* Random Forest = 0.955
+* Decision Tree = 0.886
+* Linear Regression = 0.754
+
+
+**Metric 2: R-Squared (K-Fold Cross Validation, k = 4)**
+
+* Random Forest = 0.925
+* Decision Tree = 0.871
+* Linear Regression = 0.853
+
+
+Even though the drivers of prediction were not of specific interest in this project, it was interesting to see across all three modeling approaches that the input variable with the biggest impact on the prediction was *distance_from_store* rather than variables such as *total sales*. This is potentially valuable information for the business, so discovering this was worthwhile.
+
+
+# Predicting Missing Loyalty Scores
+
+Finally in completeling our task for ABC Grocery, the Random Forest model was selected to ultimately predict the *loyalty_score* for those customers that the market research consultancy were unable to tag.
+
+As before, we cannot just pass the data for these customers into the model "as is". The data needs to be preparred to be in the "exact" the same format as what was used when training the model.
+
+Tasks to be performed in the following code:
+
+* Import the required packages for preprocessing
+* Import the data for those customers who are missing a *loyalty_score* value
+* Import our model object & any preprocessing artifacts
+* Drop columns that were not used when training the model (customer_id)
+* Drop rows with missing values
+* Apply One Hot Encoding to the gender column (using transform)
+* Make the predictions using .predict()
+
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+
+# import customers for scoring
+to_be_scored = pickle.load("data/abc_regression_scoring.p", "rb")
+
+# import model and model objects
+regressor = pickle.load(open("data/random_forest_regression_model.p", "rb"))
+one_hot_encoder = pickle.load(open("data/random_forest_regression_ohe.p", "rb"))
+
+# drop unused columns
+to_be_scored.drop(["customer_id"], axis = 1, inplace = True)
+
+# drop missing values
+to_be_scored.dropna(how = "any", inplace = True)
+
+# apply one hot encoding (transform only)
+categorical_vars = ["gender"]
+encoder_vars_array = one_hot_encoder.transform(to_be_scored[categorical_vars])
+encoder_feature_names = one_hot_encoder.get_feature_names(categorical_vars)
+encoder_vars_df = pd.DataFrame(encoder_vars_array, columns = encoder_feature_names)
+to_be_scored = pd.concat([to_be_scored.reset_index(drop=True), encoder_vars_df.reset_index(drop=True)], axis = 1)
+to_be_scored.drop(categorical_vars, axis = 1, inplace = True)
+
+# make our predictions!
+loyalty_predictions = regressor.predict(to_be_scored)
+
+```
+
+*loyalty_score* predictions have now been made for these missing customers. Considering the impressive metrics on the test set, there is reasonable confidence in these scores. This extra customer information will ensure our client can undertake more accurate and relevant customer tracking, targeting, and communications.
+
+___
+
+# Growth & Next Steps
+
+While predictive accuracy was relatively high, other modeling approaches could be tested to see if even more accuracy could be gained, especially those somewhat similar to Random Forest (i.e. XGBoost, LightGBM).
+
+Additionally, the hyperparameters of the Random Forest could also be tuned. Notably, regularization parameters such as "tree depth" as well as potentially training on a higher number of Decision Trees (n_estimators) in the Random Forest could be used.
+
+From a data point of view, further variables could be collected and further feature engineering could be undertaken ensuring as much useful information is available for predicting customer loyalty.
diff --git a/_posts/2026-07-31-enhancing-targeting-accuracy.md b/_posts/2026-07-31-enhancing-targeting-accuracy.md
new file mode 100644
index 000000000..e9df431c9
--- /dev/null
+++ b/_posts/2026-07-31-enhancing-targeting-accuracy.md
@@ -0,0 +1,1665 @@
+---
+layout: post
+title: Enhancing Targeting Accuracy Using ML
+image: "/posts/classification-title-img.png"
+tags: [Customer Targeting, Machine Learning, Classification, Python]
+---
+
+Our client, a grocery retailer, wants to utilise Machine Learning to reduce mailing costs, and improve ROI!
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results](#overview-results)
+ - [Growth/Next Steps](#overview-growth)
+- [01. Data Overview](#data-overview)
+- [02. Modelling Overview](#modelling-overview)
+- [03. Logistic Regression](#logreg-title)
+- [04. Decision Tree](#clftree-title)
+- [05. Random Forest](#rf-title)
+- [06. KNN](#knn-title)
+- [07. Modelling Summary](#modelling-summary)
+- [08. Application](#modelling-application)
+- [09. Growth & Next Steps](#growth-next-steps)
+
+___
+
+# Project Overview
+
+### Context
+
+Our client, a grocery retailer, sent out mailers in a marketing campaign for their new *delivery club*. This cost customers $100 per year for membership, and offered free grocery deliveries, rather than the normal cost of $10 per delivery.
+
+For this, they sent mailers to their entire customer base (apart from a control group) but this proved expensive. For the next batch of communications they would like to save costs by *only* mailing customers that were likely to sign up.
+
+Based upon the results of the last campaign and the customer data available, we will look to understand the *probability* of customers signing up for the *delivery club*. This would allow the client to mail a more targeted selection of customers, lowering costs, and improving ROI.
+
+Let's use Machine Learning to take on this task!
+
+
+### Actions
+
+We firstly needed to compile the necessary data from tables in the database, gathering key customer metrics that may help predict *delivery club* membership.
+
+Within our historical dataset from the last campaign, we found that 69% of customers did not sign up and 31% did. This tells us that while the data isn't perfectly balanced at 50:50, it isn't *too* imbalanced either. Even so, we make sure to not rely on classification accuracy alone when assessing results - also analysing Precision, Recall, and F1-Score.
+
+As we are predicting a binary output, we tested four classification modelling approaches, namely:
+
+* Logistic Regression
+* Decision Tree
+* Random Forest
+* K Nearest Neighbours (KNN)
+
+For each model, we will import the data in the same way but will need to pre-process the data based up the requirements of each particular algorithm. We will train & test each model, look to refine each to provide optimal performance, and then measure this predictive performance based on several metrics to give a well-rounded overview of which is best.
+
+
+
+### Results
+
+The goal for the project was to build a model that would accurately predict the customers that would sign up for the *delivery club*. This would allow for a much more targeted approach when running the next iteration of the campaign. A secondary goal was to understand what the drivers for this are, so the client can get closer to the customers that need or want this service, and enhance their messaging.
+
+Based upon these, the chosen the model is the Random Forest as it was a) the most consistently performant on the test set across classification accuracy, precision, recall, and f1-score, and b) the feature importance and permutation importance allows the client an understanding of the key drivers behind *delivery club* signups.
+
+
+**Metric 1: Classification Accuracy**
+
+* KNN = 0.936
+* Random Forest = 0.935
+* Decision Tree = 0.929
+* Logistic Regression = 0.866
+
+
+**Metric 2: Precision**
+
+* KNN = 1.00
+* Random Forest = 0.887
+* Decision Tree = 0.885
+* Logistic Regression = 0.784
+
+
+**Metric 3: Recall**
+
+* Random Forest = 0.904
+* Decision Tree = 0.885
+* KNN = 0.762
+* Logistic Regression = 0.69
+
+
+**Metric 4: F1 Score**
+
+* Random Forest = 0.895
+* Decision Tree = 0.885
+* KNN = 0.865
+* Logistic Regression = 0.734
+
+
+### Growth/Next Steps
+
+While predictive accuracy was relatively high - other modelling approaches could be tested, especially those somewhat similar to Random Forest, for example XGBoost, LightGBM to see if even more accuracy could be gained.
+
+From a data point of view, further variables could be collected, and further feature engineering could be undertaken to ensure that we have as much useful information available for predicting customer loyalty
+
+
+___
+
+# Data Overview
+
+We will be predicting the binary *signup_flag* metric from the *campaign_data* table in the client database.
+
+The key variables hypothesised to predict this will come from the client database, namely the *transactions* table, the *customer_details* table, and the *product_areas* table.
+
+We aggregated up customer data from the 3 months prior to the last campaign.
+
+After this data pre-processing in Python, we have a dataset for modelling that contains the following fields...
+
+
+
+| **Variable Name** | **Variable Type** | **Description** |
+|---|---|---|
+| signup_flag | Dependent | A binary variable showing if the customer signed up for the delivery club in the last campaign |
+| distance_from_store | Independent | The distance in miles from the customers home address, and the store |
+| gender | Independent | The gender provided by the customer |
+| credit_score | Independent | The customers most recent credit score |
+| total_sales | Independent | Total spend by the customer in ABC Grocery - 3 months pre campaign |
+| total_items | Independent | Total products purchased by the customer in ABC Grocery - 3 months pre campaign |
+| transaction_count | Independent | Total unique transactions made by the customer in ABC Grocery - 3 months pre campaign |
+| product_area_count | Independent | The number of product areas within ABC Grocery the customers has shopped into - 3 months pre campaign |
+| average_basket_value | Independent | The average spend per transaction for the customer in ABC Grocery - 3 months pre campaign |
+
+
+# Modelling Overview
+
+We will build a model that looks to accurately predict *signup_flag*, based upon the customer metrics listed above.
+
+If that can be achieved, we can use this model to predict signup & signup probability for future campaigns. This information can be used to target those more likely to sign-up, reducing marketing costs and thus increasing ROI.
+
+As we are predicting a binary output, we tested three classification modelling approaches, namely:
+
+* Logistic Regression
+* Decision Tree
+* Random Forest
+
+
+# Logistic Regression
+
+We utilise the scikit-learn library within Python to model our data using Logistic Regression. The code sections below are broken up into 5 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+* Optimal Threshold Analysis
+
+
+### Data Import
+
+Since we saved our modelling data as a pickle file, we import it. We ensure we remove the id column, and we also ensure our data is shuffled.
+
+We also investigate the class balance of our dependent variable - which is important when assessing classification accuracy.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.linear_model import LogisticRegression
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import confusion_matrix, accuracy_score, precision_score, recall_score, f1_score
+from sklearn.preprocessing import OneHotEncoder
+from sklearn.feature_selection import RFECV
+
+# import modelling data
+data_for_model = pickle.load(open("data/delivery_club_modelling.p", "rb"))
+
+# drop unnecessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+# assess class balance of dependent variable
+data_for_model["signup_flag"].value_counts(normalize = True)
+
+```
+
+From the last step in the above code, we see that **69% of customers did not sign up and 31% did**. This tells us that while the data isn't perfectly balanced at 50:50, it isn't *too* imbalanced either. Because of this, and as you will see, we make sure to not rely on classification accuracy alone when assessing results - also analysing Precision, Recall, and F1-Score.
+
+
+### Data Preprocessing
+
+For Logistic Regression we have certain data preprocessing steps that need to be addressed, including:
+
+* Missing values in the data
+* The effect of outliers
+* Encoding categorical variables to numeric form
+* Multicollinearity & Feature Selection
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) we will just remove those rows
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Outliers
+
+The ability for a Logistic Regression model to generalise well across *all* data can be hampered if there are outliers present. There is no right or wrong way to deal with outliers, but it is always something worth very careful consideration - just because a value is high or low, does not necessarily mean it should not be there!
+
+In this code section, we use **.describe()** from Pandas to investigate the spread of values for each of our predictors. The results of this can be seen in the table below.
+
+
+
+| **metric** | **distance_from_store** | **credit_score** | **total_sales** | **total_items** | **transaction_count** | **product_area_count** | **average_basket_value** |
+|---|---|---|---|---|---|---|---|
+| mean | 2.61 | 0.60 | 968.17 | 143.88 | 22.21 | 4.18 | 38.03 |
+| std | 14.40 | 0.10 | 1073.65 | 125.34 | 11.72 | 0.92 | 24.24 |
+| min | 0.00 | 0.26 | 2.09 | 1.00 | 1.00 | 1.00 | 2.09 |
+| 25% | 0.73 | 0.53 | 383.94 | 77.00 | 16.00 | 4.00 | 21.73 |
+| 50% | 1.64 | 0.59 | 691.64 | 123.00 | 23.00 | 4.00 | 31.07 |
+| 75% | 2.92 | 0.67 | 1121.53 | 170.50 | 28.00 | 5.00 | 46.43 |
+| max | 400.97 | 0.88 | 7372.06 | 910.00 | 75.00 | 5.00 | 141.05 |
+
+
+Based on this investigation, we see some *max* column values for several variables to be much higher than the *median* value.
+
+This is for columns *distance_from_store*, *total_sales*, and *total_items*
+
+For example, the median *distance_to_store* is 1.64 miles, but the maximum is over 400 miles!
+
+Because of this, we apply some outlier removal in order to facilitate generalisation across the full dataset.
+
+We do this using the "boxplot approach" where we remove any rows where the values within those columns are outside of the interquartile range multiplied by 2.
+
+
+```python
+
+outlier_investigation = data_for_model.describe()
+outlier_columns = ["distance_from_store", "total_sales", "total_items"]
+
+# boxplot approach
+for column in outlier_columns:
+
+ lower_quartile = data_for_model[column].quantile(0.25)
+ upper_quartile = data_for_model[column].quantile(0.75)
+ iqr = upper_quartile - lower_quartile
+ iqr_extended = iqr * 2
+ min_border = lower_quartile - iqr_extended
+ max_border = upper_quartile + iqr_extended
+
+ outliers = data_for_model[(data_for_model[column] < min_border) | (data_for_model[column] > max_border)].index
+ print(f"{len(outliers)} outliers detected in column {column}")
+
+ data_for_model.drop(outliers, inplace = True)
+
+```
+
+
+##### Split Out Data For Modelling
+
+In the next code block we do two things, we firstly split our data into an **X** object which contains only the predictor variables, and a **y** object that contains only our dependent variable.
+
+Once we have done this, we split our data into training and test sets to ensure we can fairly validate the accuracy of the predictions on data that was not used in training. In this case, we have allocated 80% of the data for training, and the remaining 20% for validation. We make sure to add in the *stratify* parameter to ensure that both our training and test sets have the same proportion of customers who did, and did not, sign up for the *delivery club* - meaning we can be more confident in our assessment of predictive performance.
+
+
+```python
+
+# split data into X and y objects for modelling
+X = data_for_model.drop(["signup_flag"], axis = 1)
+y = data_for_model["signup_flag"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42, stratify = y)
+
+```
+
+
+##### Categorical Predictor Variables
+
+In our dataset, we have one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+The Logistic Regression algorithm can't deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it, in other words, Male isn't higher or lower than Female and vice versa - one appropriate approach is to apply One Hot Encoding to the categorical column.
+
+One Hot Encoding can be thought of as a way to represent categorical variables as binary vectors, in other words, a set of *new* columns for each categorical value with either a 1 or a 0 saying whether that value is true or not for that observation. These new columns would go into our model as input variables, and the original column is discarded.
+
+We also drop one of the new columns using the parameter *drop = "first"*. We do this to avoid the *dummy variable trap* where our newly created encoded columns perfectly predict each other - and we run the risk of breaking the assumption that there is no multicollinearity, a requirement or at least an important consideration for some models, Linear Regression being one of them! Multicollinearity occurs when two or more input variables are *highly* correlated with each other, it is a scenario we attempt to avoid as in short, while it won't necessarily affect the predictive accuracy of our model, it can make it difficult to trust the statistics around how well the model is performing, and how much each input variable is truly having.
+
+In the code, we also make sure to apply *fit_transform* to the training set, but only *transform* to the test set. This means the One Hot Encoding logic will *learn and apply* the "rules" from the training data, but only *apply* them to the test data. This is important in order to avoid *data leakage* where the test set *learns* information about the training data, and means we can't fully trust model performance metrics!
+
+For ease, after we have applied One Hot Encoding, we turn our training and test objects back into Pandas Dataframes, with the column names applied.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to pandas dataframe
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+##### Feature Selection
+
+Feature Selection is the process used to select the input variables that are most important to your Machine Learning task. It can be a very important addition or at least, consideration, in certain scenarios. The potential benefits of Feature Selection are:
+
+* **Improved Model Accuracy** - eliminating noise can help true relationships stand out
+* **Lower Computational Cost** - our model becomes faster to train, and faster to make predictions
+* **Explainability** - understanding & explaining outputs for stakeholder & customers becomes much easier
+
+There are many, many ways to apply Feature Selection. These range from simple methods such as a *Correlation Matrix* showing variable relationships, to *Univariate Testing* which helps us understand statistical relationships between variables, and then to even more powerful approaches like *Recursive Feature Elimination (RFE)* which is an approach that starts with all input variables, and then iteratively removes those with the weakest relationships with the output variable.
+
+For our task we applied a variation of Recursive Feature Elimination called *Recursive Feature Elimination With Cross Validation (RFECV)* where we split the data into many "chunks" and iteratively trains & validates models on each "chunk" separately. This means that each time we assess different models with different variables included, or eliminated, the algorithm also knows how accurate each of those models was. From the suite of model scenarios that are created, the algorithm can determine which provided the best accuracy, and thus can infer the best set of input variables to use!
+
+
+```python
+
+# instantiate RFECV & the model type to be utilised
+clf = LogisticRegression(random_state = 42, max_iter = 1000)
+feature_selector = RFECV(clf)
+
+# fit RFECV onto our training & test data
+fit = feature_selector.fit(X_train,y_train)
+
+# extract & print the optimal number of features
+optimal_feature_count = feature_selector.n_features_
+print(f"Optimal number of features: {optimal_feature_count}")
+
+# limit our training & test sets to only include the selected variables
+X_train = X_train.loc[:, feature_selector.get_support()]
+X_test = X_test.loc[:, feature_selector.get_support()]
+
+```
+
+
+The below code then produces a plot that visualises the cross-validated classification accuracy with each potential number of features
+
+```python
+
+plt.style.use('seaborn-poster')
+plt.plot(range(1, len(fit.cv_results_['mean_test_score']) + 1), fit.cv_results_['mean_test_score'], marker = "o")
+plt.ylabel("Classification Accuracy")
+plt.xlabel("Number of Features")
+plt.title(f"Feature Selection using RFECV \n Optimal number of features is {optimal_feature_count} (at score of {round(max(fit.cv_results_['mean_test_score']),4)})")
+plt.tight_layout()
+plt.show()
+
+```
+
+
+This creates the below plot, which shows us that the highest cross-validated classification accuracy (0.904) is when we include seven of our original input variables. The variable that has been dropped is *total_sales* but from the chart we can see that the difference is negligible. However, we will continue on with the selected seven!
+
+
+
+
+
+### Model Training
+
+Instantiating and training our Logistic Regression model is done using the below code. We use the *random_state* parameter to ensure reproducible results, meaning any refinements can be compared to past results. We also specify *max_iter = 1000* to allow the solver more attempts at finding an optimal regression line, as the default value of 100 was not enough.
+
+```python
+
+# instantiate our model object
+clf = LogisticRegression(random_state = 42, max_iter = 1000)
+
+# fit our model using our training & test sets
+clf.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+To assess how well our model is predicting on new data - we use the trained model object (here called *clf*) and ask it to predict the *signup_flag* variable for the test set.
+
+In the code below we create one object to hold the binary 1/0 predictions, and another to hold the actual prediction probabilities for the positive class.
+
+```python
+
+# predict on the test set
+y_pred_class = clf.predict(X_test)
+y_pred_prob = clf.predict_proba(X_test)[:,1]
+
+```
+
+
+##### Confusion Matrix
+
+A Confusion Matrix provides us a visual way to understand how our predictions match up against the actual values for those test set observations.
+
+The below code creates the Confusion Matrix using the *confusion_matrix* functionality from within scikit-learn and then plots it using matplotlib.
+
+```python
+
+# create the confusion matrix
+conf_matrix = confusion_matrix(y_test, y_pred_class)
+
+# plot the confusion matrix
+plt.style.use("seaborn-poster")
+plt.matshow(conf_matrix, cmap = "coolwarm")
+plt.gca().xaxis.tick_bottom()
+plt.title("Confusion Matrix")
+plt.ylabel("Actual Class")
+plt.xlabel("Predicted Class")
+for (i, j), corr_value in np.ndenumerate(conf_matrix):
+ plt.text(j, i, corr_value, ha = "center", va = "center", fontsize = 20)
+plt.show()
+
+```
+
+
+
+
+
+The aim is to have a high proportion of observations falling into the top left cell (predicted non-signup and actual non-signup) and the bottom right cell (predicted signup and actual signup).
+
+Since the proportion of signups in our data was around 30:70 we will next analyse not only Classification Accuracy, but also Precision, Recall, and F1-Score which will help us assess how well our model has performed in reality.
+
+
+##### Classification Performance Metrics
+
+**Classification Accuracy**
+
+Classification Accuracy is a metric that tells us *of all predicted observations, what proportion did we correctly classify*. This is very intuitive, but when dealing with imbalanced classes, can be misleading.
+
+An example of this could be a rare disease. A model with a 98% Classification Accuracy on might appear like a fantastic result, but if our data contained 98% of patients *without* the disease, and 2% *with* the disease - then a 98% Classification Accuracy could be obtained simply by predicting that *no one* has the disease - which wouldn't be a great model in the real world. Luckily, there are other metrics which can help us!
+
+In this example of the rare disease, we could define Classification Accuracy as *of all predicted patients, what proportion did we correctly classify as either having the disease, or not having the disease*
+
+
+**Precision & Recall**
+
+Precision is a metric that tells us *of all observations that were predicted as positive, how many actually were positive*
+
+Keeping with the rare disease example, Precision would tell us *of all patients we predicted to have the disease, how many actually did*
+
+Recall is a metric that tells us *of all positive observations, how many did we predict as positive*
+
+Again, referring to the rare disease example, Recall would tell us *of all patients who actually had the disease, how many did we correctly predict*
+
+The tricky thing about Precision & Recall is that it is impossible to optimise both - it's a zero-sum game. If you try to increase Precision, Recall decreases, and vice versa. Sometimes however it will make more sense to try and elevate one of them, in spite of the other. In the case of our rare-disease prediction like we've used in our example, perhaps it would be more important to optimise for Recall as we want to classify as many positive cases as possible. In saying this however, we don't want to just classify every patient as having the disease, as that isn't a great outcome either!
+
+So - there is one more metric we will discuss & calculate, which is actually a *combination* of both...
+
+
+**F1 Score**
+
+F1-Score is a metric that essentially "combines" both Precision & Recall. Technically speaking, it is the harmonic mean of these two metrics. A good, or high, F1-Score comes when there is a balance between Precision & Recall, rather than a disparity between them.
+
+Overall, optimising your model for F1-Score means that you'll get a model that is working well for both positive & negative classifications rather than skewed towards one or the other. To return to the rare disease predictions, a high F1-Score would mean we've got a good balance between successfully predicting the disease when it's present, and not predicting cases where it's not present.
+
+Using all of these metrics in combination gives a really good overview of the performance of a classification model, and gives us an understanding of the different scenarios & considerations!
+
+
+In the code below, we utilise in-built functionality from scikit-learn to calculate these four metrics.
+
+```python
+
+# classification accuracy
+accuracy_score(y_test, y_pred_class)
+
+# precision
+precision_score(y_test, y_pred_class)
+
+# recall
+recall_score(y_test, y_pred_class)
+
+# f1-score
+f1_score(y_test, y_pred_class)
+
+```
+
+Running this code gives us:
+
+* Classification Accuracy = **0.866** meaning we correctly predicted the class of 86.6% of test set observations
+* Precision = **0.784** meaning that for our *predicted* delivery club signups, we were correct 78.4% of the time
+* Recall = **0.69** meaning that of all *actual* delivery club signups, we predicted correctly 69% of the time
+* F1-Score = **0.734**
+
+Since our data is *somewhat* imbalanced, looking at these metrics rather than just Classification Accuracy on it's own - is a good idea, and gives us a much better understanding of what our predictions mean! We will use these same metrics when applying other models for this task, and can compare how they stack up.
+
+
+### Finding The Optimal Classification Threshold
+
+By default, most pre-built classification models & algorithms will just use a 50% probability to discern between a positive class prediction (delivery club signup) and a negative class prediction (delivery club non-signup).
+
+Just because 50% is the default threshold *does not mean* it is the best one for our task.
+
+Here, we will test many potential classification thresholds, and plot the Precision, Recall & F1-Score, and find an optimal solution!
+
+
+```python
+
+# set up the list of thresholds to loop through
+thresholds = np.arange(0, 1, 0.01)
+
+# create empty lists to append the results to
+precision_scores = []
+recall_scores = []
+f1_scores = []
+
+# loop through each threshold - fit the model - append the results
+for threshold in thresholds:
+
+ pred_class = (y_pred_prob >= threshold) * 1
+
+ precision = precision_score(y_test, pred_class, zero_division = 0)
+ precision_scores.append(precision)
+
+ recall = recall_score(y_test, pred_class)
+ recall_scores.append(recall)
+
+ f1 = f1_score(y_test, pred_class)
+ f1_scores.append(f1)
+
+# extract the optimal f1-score (and it's index)
+max_f1 = max(f1_scores)
+max_f1_idx = f1_scores.index(max_f1)
+
+```
+
+
+Now we have run this, we can use the below code to plot the results!
+
+
+```python
+
+# plot the results
+plt.style.use("seaborn-poster")
+plt.plot(thresholds, precision_scores, label = "Precision", linestyle = "--")
+plt.plot(thresholds, recall_scores, label = "Recall", linestyle = "--")
+plt.plot(thresholds, f1_scores, label = "F1", linewidth = 5)
+plt.title(f"Finding the Optimal Threshold for Classification Model \n Max F1: {round(max_f1,2)} (Threshold = {round(thresholds[max_f1_idx],2)})")
+plt.xlabel("Threshold")
+plt.ylabel("Assessment Score")
+plt.legend(loc = "lower left")
+plt.tight_layout()
+plt.show()
+
+```
+
+
+
+
+Along the x-axis of the above plot we have the different classification thresholds that were testing. Along the y-axis we have the performance score for each of our three metrics. As per the legend, we have Precision as a blue dotted line, Recall as an orange dotted line, and F1-Score as a thick green line. You can see the interesting "zero-sum" relationship between Precision & Recall *and* you can see that the point where Precision & Recall meet is where F1-Score is maximised.
+
+As you can see at the top of the plot, the optimal F1-Score for this model 0.78 and this is obtained at a classification threshold of 0.44. This is higher than the F1-Score of 0.734 that we achieved at the default classification threshold of 0.50!
+
+___
+
+# Decision Tree
+
+We will again utilise the scikit-learn library within Python to model our data using a Decision Tree. The code sections below are broken up into 6 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+* Tree Visualisation
+* Decision Tree Regularisation
+
+
+### Data Import
+
+Since we saved our modelling data as a pickle file, we import it. We ensure we remove the id column, and we also ensure our data is shuffled.
+
+Just like we did for Logistic Regression - our code also investigates the class balance of our dependent variable.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.tree import DecisionTreeClassifier, plot_tree
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import confusion_matrix, accuracy_score, precision_score, recall_score, f1_score
+from sklearn.preprocessing import OneHotEncoder
+
+# import modelling data
+data_for_model = pickle.load(open("data/delivery_club_modelling.p", "rb"))
+
+# drop unnecessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+# assess class balance of dependent variable
+data_for_model["signup_flag"].value_counts(normalize = True)
+
+```
+
+### Data Preprocessing
+
+While Logistic Regression is susceptible to the effects of outliers, and highly correlated input variables - Decision Trees are not, so the required preprocessing here is lighter. We still however will put in place logic for:
+
+* Missing values in the data
+* Encoding categorical variables to numeric form
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) we will just remove those rows
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Split Out Data For Modelling
+
+In exactly the same way we did for Logistic Regression, in the next code block we do two things, we firstly split our data into an **X** object which contains only the predictor variables, and a **y** object that contains only our dependent variable.
+
+Once we have done this, we split our data into training and test sets to ensure we can fairly validate the accuracy of the predictions on data that was not used in training. In this case, we have allocated 80% of the data for training, and the remaining 20% for validation. Again, we make sure to add in the *stratify* parameter to ensure that both our training and test sets have the same proportion of customers who did, and did not, sign up for the *delivery club* - meaning we can be more confident in our assessment of predictive performance.
+
+
+
+```python
+
+# split data into X and y objects for modelling
+X = data_for_model.drop(["signup_flag"], axis = 1)
+y = data_for_model["signup_flag"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42, stratify = y)
+
+```
+
+
+##### Categorical Predictor Variables
+
+In our dataset, we have one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+Just like the Logistic Regression algorithm, the Decision Tree cannot deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it, in other words, Male isn't higher or lower than Female and vice versa - we would again apply One Hot Encoding to the categorical column.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to pandas dataframe
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+### Model Training
+
+Instantiating and training our Decision Tree model is done using the below code. We use the *random_state* parameter to ensure we get reproducible results, and this helps us understand any improvements in performance with changes to model hyperparameters.
+
+```python
+
+# instantiate our model object
+clf = DecisionTreeClassifier(random_state = 42, max_depth = 5)
+
+# fit our model using our training & test sets
+clf.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+Just like we did with Logistic Regression, to assess how well our model is predicting on new data - we use the trained model object (here called *clf*) and ask it to predict the *signup_flag* variable for the test set.
+
+In the code below we create one object to hold the binary 1/0 predictions, and another to hold the actual prediction probabilities for the positive class.
+
+```python
+
+# predict on the test set
+y_pred_class = clf.predict(X_test)
+y_pred_prob = clf.predict_proba(X_test)[:,1]
+
+```
+
+
+##### Confusion Matrix
+
+As we discussed in the above section applying Logistic Regression - a Confusion Matrix provides us a visual way to understand how our predictions match up against the actual values for those test set observations.
+
+The below code creates the Confusion Matrix using the *confusion_matrix* functionality from within scikit-learn and then plots it using matplotlib.
+
+```python
+
+# create the confusion matrix
+conf_matrix = confusion_matrix(y_test, y_pred_class)
+
+# plot the confusion matrix
+plt.style.use("seaborn-poster")
+plt.matshow(conf_matrix, cmap = "coolwarm")
+plt.gca().xaxis.tick_bottom()
+plt.title("Confusion Matrix")
+plt.ylabel("Actual Class")
+plt.xlabel("Predicted Class")
+for (i, j), corr_value in np.ndenumerate(conf_matrix):
+ plt.text(j, i, corr_value, ha = "center", va = "center", fontsize = 20)
+plt.show()
+
+```
+
+
+
+
+
+The aim is to have a high proportion of observations falling into the top left cell (predicted non-signup and actual non-signup) and the bottom right cell (predicted signup and actual signup).
+
+Since the proportion of signups in our data was around 30:70 we will again analyse not only Classification Accuracy, but also Precision, Recall, and F1-Score as they will help us assess how well our model has performed from different points of view.
+
+
+##### Classification Performance Metrics
+
+**Accuracy, Precision, Recall, F1-Score**
+
+For details on these performance metrics, please see the above section on Logistic Regression. Using all four of these metrics in combination gives a really good overview of the performance of a classification model, and gives us an understanding of the different scenarios & considerations!
+
+In the code below, we utilise in-built functionality from scikit-learn to calculate these four metrics.
+
+```python
+
+# classification accuracy
+accuracy_score(y_test, y_pred_class)
+
+# precision
+precision_score(y_test, y_pred_class)
+
+# recall
+recall_score(y_test, y_pred_class)
+
+# f1-score
+f1_score(y_test, y_pred_class)
+
+```
+
+Running this code gives us:
+
+* Classification Accuracy = **0.929** meaning we correctly predicted the class of 92.9% of test set observations
+* Precision = **0.885** meaning that for our *predicted* delivery club signups, we were correct 88.5% of the time
+* Recall = **0.885** meaning that of all *actual* delivery club signups, we predicted correctly 88.5% of the time
+* F1-Score = **0.885**
+
+These are all higher than what we saw when applying Logistic Regression, even after we had optimised the classification threshold!
+
+
+
+### Visualise Our Decision Tree
+
+To see the decisions that have been made in the tree, we can use the plot_tree functionality that we imported from scikit-learn. To do this, we use the below code:
+
+
+```python
+
+# plot the nodes of the decision tree
+plt.figure(figsize=(25,15))
+tree = plot_tree(clf,
+ feature_names = X.columns,
+ filled = True,
+ rounded = True,
+ fontsize = 16)
+
+```
+
+That code gives us the below plot:
+
+
+
+
+
+This is a very powerful visual, and one that can be shown to stakeholders in the business to ensure they understand exactly what is driving the predictions.
+
+One interesting thing to note is that the *very first split* appears to be using the variable *distance from store* so it would seem that this is a very important variable when it comes to predicting signups to the delivery club!
+
+
+### Decision Tree Regularisation
+
+Decision Tree's can be prone to over-fitting, in other words, without any limits on their splitting, they will end up learning the training data perfectly. We would much prefer our model to have a more *generalised* set of rules, as this will be more robust & reliable when making predictions on *new* data.
+
+One effective method of avoiding this over-fitting, is to apply a *max depth* to the Decision Tree, meaning we only allow it to split the data a certain number of times before it is required to stop.
+
+We initially trained our model with a placeholder depth of 5, but unfortunately, we don't necessarily know the *optimal* number for this. Below we will loop over a variety of values and assess which gives us the best predictive performance!
+
+
+```python
+
+# finding the best max_depth
+
+# set up range for search, and empty list to append accuracy scores to
+max_depth_list = list(range(1,15))
+accuracy_scores = []
+
+# loop through each possible depth, train and validate model, append test set f1-score
+for depth in max_depth_list:
+
+ clf = DecisionTreeClassifier(max_depth = depth, random_state = 42)
+ clf.fit(X_train,y_train)
+ y_pred = clf.predict(X_test)
+ accuracy = f1_score(y_test,y_pred)
+ accuracy_scores.append(accuracy)
+
+# store max accuracy, and optimal depth
+max_accuracy = max(accuracy_scores)
+max_accuracy_idx = accuracy_scores.index(max_accuracy)
+optimal_depth = max_depth_list[max_accuracy_idx]
+
+# plot accuracy by max depth
+plt.plot(max_depth_list,accuracy_scores)
+plt.scatter(optimal_depth, max_accuracy, marker = "x", color = "red")
+plt.title(f"Accuracy (F1 Score) by Max Depth \n Optimal Tree Depth: {optimal_depth} (F1 Score: {round(max_accuracy,4)})")
+plt.xlabel("Max Depth of Decision Tree")
+plt.ylabel("Accuracy (F1 Score)")
+plt.tight_layout()
+plt.show()
+
+```
+
+That code gives us the below plot - which visualises the results!
+
+
+
+
+
+In the plot we can see that the *maximum* F1-Score on the test set is found when applying a *max_depth* value of 9 which takes our F1-Score up to 0.925
+
+___
+
+# Random Forest
+
+We will again utilise the scikit-learn library within Python to model our data using a Random Forest. The code sections below are broken up into 4 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+
+
+### Data Import
+
+Again, since we saved our modelling data as a pickle file, we import it. We ensure we remove the id column, and we also ensure our data is shuffled.
+
+As this is the exact same process we ran for both Logistic Regression & the Decision Tree - our code also investigates the class balance of our dependent variable
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import confusion_matrix, accuracy_score, precision_score, recall_score, f1_score
+from sklearn.preprocessing import OneHotEncoder
+from sklearn.inspection import permutation_importance
+
+# import modelling data
+data_for_model = pickle.load(open("data/delivery_club_modelling.p", "rb"))
+
+# drop unnecessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+```
+
+### Data Preprocessing
+
+While Linear Regression is susceptible to the effects of outliers, and highly correlated input variables - Random Forests, just like Decision Trees, are not, so the required preprocessing here is lighter. We still however will put in place logic for:
+
+* Missing values in the data
+* Encoding categorical variables to numeric form
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) we will just remove those rows. Again, this is exactly the same process we ran for Logistic Regression & the Decision Tree.
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Split Out Data For Modelling
+
+In exactly the same way we did for both Logistic Regression & our Decision Tree, in the next code block we do two things, we firstly split our data into an X object which contains only the predictor variables, and a y object that contains only our dependent variable.
+
+Once we have done this, we split our data into training and test sets to ensure we can fairly validate the accuracy of the predictions on data that was not used in training. In this case, we have allocated 80% of the data for training, and the remaining 20% for validation. Again, we make sure to add in the stratify parameter to ensure that both our training and test sets have the same proportion of customers who did, and did not, sign up for the delivery club - meaning we can be more confident in our assessment of predictive performance.
+
+
+```python
+
+# split data into X and y objects for modelling
+X = data_for_model.drop(["signup_flag"], axis = 1)
+y = data_for_model["signup_flag"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42, stratify = y)
+
+```
+
+
+##### Categorical Predictor Variables
+
+In our dataset, we have one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+Just like the Logistic Regression algorithm, Random Forests cannot deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it, in other words, Male isn't higher or lower than Female and vice versa - we would again apply One Hot Encoding to the categorical column.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to pandas dataframe
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+### Model Training
+
+Instantiating and training our Random Forest model is done using the below code. We use the *random_state* parameter to ensure we get reproducible results, and this helps us understand any improvements in performance with changes to model hyperparameters.
+
+We also look to build more Decision Trees in the Random Forest (500) than would be done using the default value of 100.
+
+Lastly, since the default scikit-learn implementation of Random Forests does not limit the number of randomly selected variables offered up for splitting at each split point in each Decision Tree - we put this in place using the *max_features* parameter. This can always be refined later through testing, or through an approach such as gridsearch.
+
+```python
+
+# instantiate our model object
+clf = RandomForestClassifier(random_state = 42, n_estimators = 500, max_features = 5)
+
+# fit our model using our training & test sets
+clf.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+Just like we did with Logistic Regression & our Decision Tree, to assess how well our model is predicting on new data - we use the trained model object (here called *clf*) and ask it to predict the *signup_flag* variable for the test set.
+
+In the code below we create one object to hold the binary 1/0 predictions, and another to hold the actual prediction probabilities for the positive class.
+
+```python
+
+# predict on the test set
+y_pred_class = clf.predict(X_test)
+y_pred_prob = clf.predict_proba(X_test)[:,1]
+
+```
+
+
+##### Confusion Matrix
+
+As we discussed in the above sections - a Confusion Matrix provides us a visual way to understand how our predictions match up against the actual values for those test set observations.
+
+The below code creates the Confusion Matrix using the *confusion_matrix* functionality from within scikit-learn and then plots it using matplotlib.
+
+```python
+
+# create the confusion matrix
+conf_matrix = confusion_matrix(y_test, y_pred_class)
+
+# plot the confusion matrix
+plt.style.use("seaborn-poster")
+plt.matshow(conf_matrix, cmap = "coolwarm")
+plt.gca().xaxis.tick_bottom()
+plt.title("Confusion Matrix")
+plt.ylabel("Actual Class")
+plt.xlabel("Predicted Class")
+for (i, j), corr_value in np.ndenumerate(conf_matrix):
+ plt.text(j, i, corr_value, ha = "center", va = "center", fontsize = 20)
+plt.show()
+
+```
+
+
+
+
+
+The aim is to have a high proportion of observations falling into the top left cell (predicted non-signup and actual non-signup) and the bottom right cell (predicted signup and actual signup).
+
+Since the proportion of signups in our data was around 30:70 we will again analyse not only Classification Accuracy, but also Precision, Recall, and F1-Score as they will help us assess how well our model has performed from different points of view.
+
+
+##### Classification Performance Metrics
+
+**Accuracy, Precision, Recall, F1-Score**
+
+For details on these performance metrics, please see the above section on Logistic Regression. Using all four of these metrics in combination gives a really good overview of the performance of a classification model, and gives us an understanding of the different scenarios & considerations!
+
+In the code below, we utilise in-built functionality from scikit-learn to calculate these four metrics.
+
+```python
+
+# classification accuracy
+accuracy_score(y_test, y_pred_class)
+
+# precision
+precision_score(y_test, y_pred_class)
+
+# recall
+recall_score(y_test, y_pred_class)
+
+# f1-score
+f1_score(y_test, y_pred_class)
+
+```
+
+Running this code gives us:
+
+* Classification Accuracy = **0.935** meaning we correctly predicted the class of 93.5% of test set observations
+* Precision = **0.887** meaning that for our *predicted* delivery club signups, we were correct 88.7% of the time
+* Recall = **0.904** meaning that of all *actual* delivery club signups, we predicted correctly 90.4% of the time
+* F1-Score = **0.895**
+
+These are all higher than what we saw when applying Logistic Regression, and marginally higher than what we got from our Decision Tree. If we are after out-and-out accuracy then this would be the best model to choose. If we were happier with a simpler, easier explain model, but that had almost the same performance - then we may choose the Decision Tree instead!
+
+
+### Feature Importance
+
+Random Forests are an ensemble model, made up of many, many Decision Trees, each of which is different due to the randomness of the data being provided, and the random selection of input variables available at each potential split point.
+
+Because of this, we end up with a powerful and robust model, but because of the random or different nature of all these Decision trees - the model gives us a unique insight into how important each of our input variables are to the overall model.
+
+As we’re using random samples of data, and input variables for each Decision Tree - there are many scenarios where certain input variables are being held back and this enables us a way to compare how accurate the models predictions are if that variable is or isn’t present.
+
+So, at a high level, in a Random Forest we can measure *importance* by asking *How much would accuracy decrease if a specific input variable was removed or randomised?*
+
+If this decrease in performance, or accuracy, is large, then we’d deem that input variable to be quite important, and if we see only a small decrease in accuracy, then we’d conclude that the variable is of less importance.
+
+At a high level, there are two common ways to tackle this. The first, often just called **Feature Importance** is where we find all nodes in the Decision Trees of the forest where a particular input variable is used to split the data and assess what the gini impurity score (for a Classification problem) was before the split was made, and compare this to the gini impurity score after the split was made. We can take the *average* of these improvements across all Decision Trees in the Random Forest to get a score that tells us *how much better* we’re making the model by using that input variable.
+
+If we do this for *each* of our input variables, we can compare these scores and understand which is adding the most value to the predictive power of the model!
+
+The other approach, often called **Permutation Importance** cleverly uses some data that has gone *unused* at when random samples are selected for each Decision Tree (this stage is called "bootstrap sampling" or "bootstrapping")
+
+These observations that were not randomly selected for each Decision Tree are known as *Out of Bag* observations and these can be used for testing the accuracy of each particular Decision Tree.
+
+For each Decision Tree, all of the *Out of Bag* observations are gathered and then passed through. Once all of these observations have been run through the Decision Tree, we obtain a classification accuracy score for these predictions.
+
+In order to understand the *importance*, we *randomise* the values within one of the input variables - a process that essentially destroys any relationship that might exist between that input variable and the output variable - and run that updated data through the Decision Tree again, obtaining a second accuracy score. The difference between the original accuracy and the new accuracy gives us a view on how important that particular variable is for predicting the output.
+
+*Permutation Importance* is often preferred over *Feature Importance* which can at times inflate the importance of numerical features. Both are useful, and in most cases will give fairly similar results.
+
+Let's put them both in place, and plot the results...
+
+
+```python
+
+# calculate feature importance
+feature_importance = pd.DataFrame(clf.feature_importances_)
+feature_names = pd.DataFrame(X.columns)
+feature_importance_summary = pd.concat([feature_names,feature_importance], axis = 1)
+feature_importance_summary.columns = ["input_variable","feature_importance"]
+feature_importance_summary.sort_values(by = "feature_importance", inplace = True)
+
+# plot feature importance
+plt.barh(feature_importance_summary["input_variable"],feature_importance_summary["feature_importance"])
+plt.title("Feature Importance of Random Forest")
+plt.xlabel("Feature Importance")
+plt.tight_layout()
+plt.show()
+
+# calculate permutation importance
+result = permutation_importance(clf, X_test, y_test, n_repeats = 10, random_state = 42)
+permutation_importance = pd.DataFrame(result["importances_mean"])
+feature_names = pd.DataFrame(X.columns)
+permutation_importance_summary = pd.concat([feature_names,permutation_importance], axis = 1)
+permutation_importance_summary.columns = ["input_variable","permutation_importance"]
+permutation_importance_summary.sort_values(by = "permutation_importance", inplace = True)
+
+# plot permutation importance
+plt.barh(permutation_importance_summary["input_variable"],permutation_importance_summary["permutation_importance"])
+plt.title("Permutation Importance of Random Forest")
+plt.xlabel("Permutation Importance")
+plt.tight_layout()
+plt.show()
+
+```
+
+That code gives us the below plots - the first being for *Feature Importance* and the second for *Permutation Importance*!
+
+
+
+
+
+
+
+
+The overall story from both approaches is very similar, in that by far, the most important or impactful input variables are *distance_from_store* and *transaction_count*
+
+Surprisingly, *average_basket_size* was not as important as hypothesised.
+
+There are slight differences in the order or "importance" for the remaining variables but overall they have provided similar findings.
+
+___
+
+# K Nearest Neighbours
+
+We utilise the scikit-learn library within Python to model our data using KNN. The code sections below are broken up into 5 key sections:
+
+* Data Import
+* Data Preprocessing
+* Model Training
+* Performance Assessment
+* Optimal Value For K
+
+
+### Data Import
+
+Again, since we saved our modelling data as a pickle file, we import it. We ensure we remove the id column, and we also ensure our data is shuffled.
+
+As with the other approaches, we also investigate the class balance of our dependent variable - which is important when assessing classification accuracy.
+
+```python
+
+# import required packages
+import pandas as pd
+import pickle
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.neighbors import KNeighborsClassifier
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split, cross_val_score, KFold
+from sklearn.metrics import confusion_matrix, accuracy_score, precision_score, recall_score, f1_score
+from sklearn.preprocessing import OneHotEncoder, MinMaxScaler
+from sklearn.feature_selection import RFECV
+
+# import modelling data
+data_for_model = pickle.load(open("data/delivery_club_modelling.p", "rb"))
+
+# drop unnecessary columns
+data_for_model.drop("customer_id", axis = 1, inplace = True)
+
+# shuffle data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+# assess class balance of dependent variable
+data_for_model["signup_flag"].value_counts(normalize = True)
+
+```
+
+From the last step in the above code, we see that **69% of customers did not sign up and 31% did**. This tells us that while the data isn't perfectly balanced at 50:50, it isn't *too* imbalanced either. Because of this, and as you will see, we make sure to not rely on classification accuracy alone when assessing results - also analysing Precision, Recall, and F1-Score.
+
+
+### Data Preprocessing
+
+For KNN, as it is a distance based algorithm, we have certain data preprocessing steps that need to be addressed, including:
+
+* Missing values in the data
+* The effect of outliers
+* Encoding categorical variables to numeric form
+* Feature Scaling
+* Feature Selection
+
+
+##### Missing Values
+
+The number of missing values in the data was extremely low, so instead of applying any imputation (i.e. mean, most common value) we will just remove those rows
+
+```python
+
+# remove rows where values are missing
+data_for_model.isna().sum()
+data_for_model.dropna(how = "any", inplace = True)
+
+```
+
+
+##### Outliers
+
+As KNN is a distance based algorithm, you could argue that if a data point is a long way away, then it will simply never be selected as one of the neighbours - and this is true - but outliers can still cause us problems here. The main issue we face is when we come to scale our input variables, a very important step for a distance based algorithm.
+
+We don't want any variables to be "bunched up" due to a single outlier value, as this will make it hard to compare their values to the other input variables. We should always investigate outliers rigorously - in this case we will simply remove them.
+
+In this code section, just like we saw when applying Logistic Regression, we use **.describe()** from Pandas to investigate the spread of values for each of our predictors. The results of this can be seen in the table below.
+
+
+
+| **metric** | **distance_from_store** | **credit_score** | **total_sales** | **total_items** | **transaction_count** | **product_area_count** | **average_basket_value** |
+|---|---|---|---|---|---|---|---|
+| mean | 2.61 | 0.60 | 968.17 | 143.88 | 22.21 | 4.18 | 38.03 |
+| std | 14.40 | 0.10 | 1073.65 | 125.34 | 11.72 | 0.92 | 24.24 |
+| min | 0.00 | 0.26 | 2.09 | 1.00 | 1.00 | 1.00 | 2.09 |
+| 25% | 0.73 | 0.53 | 383.94 | 77.00 | 16.00 | 4.00 | 21.73 |
+| 50% | 1.64 | 0.59 | 691.64 | 123.00 | 23.00 | 4.00 | 31.07 |
+| 75% | 2.92 | 0.67 | 1121.53 | 170.50 | 28.00 | 5.00 | 46.43 |
+| max | 400.97 | 0.88 | 7372.06 | 910.00 | 75.00 | 5.00 | 141.05 |
+
+
+Again, based on this investigation, we see some *max* column values for several variables to be much higher than the *median* value.
+
+This is for columns *distance_from_store*, *total_sales*, and *total_items*
+
+For example, the median *distance_to_store* is 1.64 miles, but the maximum is over 400 miles!
+
+Because of this, we apply some outlier removal in order to facilitate generalisation across the full dataset.
+
+We do this using the "boxplot approach" where we remove any rows where the values within those columns are outside of the interquartile range multiplied by 2.
+
+
+```python
+
+outlier_investigation = data_for_model.describe()
+outlier_columns = ["distance_from_store", "total_sales", "total_items"]
+
+# boxplot approach
+for column in outlier_columns:
+
+ lower_quartile = data_for_model[column].quantile(0.25)
+ upper_quartile = data_for_model[column].quantile(0.75)
+ iqr = upper_quartile - lower_quartile
+ iqr_extended = iqr * 2
+ min_border = lower_quartile - iqr_extended
+ max_border = upper_quartile + iqr_extended
+
+ outliers = data_for_model[(data_for_model[column] < min_border) | (data_for_model[column] > max_border)].index
+ print(f"{len(outliers)} outliers detected in column {column}")
+
+ data_for_model.drop(outliers, inplace = True)
+
+```
+
+
+##### Split Out Data For Modelling
+
+In exactly the same way we've done for the other three models, in the next code block we do two things, we firstly split our data into an X object which contains only the predictor variables, and a y object that contains only our dependent variable.
+
+Once we have done this, we split our data into training and test sets to ensure we can fairly validate the accuracy of the predictions on data that was not used in training. In this case, we have allocated 80% of the data for training, and the remaining 20% for validation. Again, we make sure to add in the stratify parameter to ensure that both our training and test sets have the same proportion of customers who did, and did not, sign up for the delivery club - meaning we can be more confident in our assessment of predictive performance.
+
+
+```python
+
+# split data into X and y objects for modelling
+X = data_for_model.drop(["signup_flag"], axis = 1)
+y = data_for_model["signup_flag"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42, stratify = y)
+
+```
+
+
+##### Categorical Predictor Variables
+
+As we saw when applying the other algorithms, in our dataset, we have one categorical variable *gender* which has values of "M" for Male, "F" for Female, and "U" for Unknown.
+
+The KNN algorithm can't deal with data in this format as it can't assign any numerical meaning to it when looking to assess the relationship between the variable and the dependent variable.
+
+As *gender* doesn't have any explicit *order* to it, in other words, Male isn't higher or lower than Female and vice versa - one appropriate approach is to apply One Hot Encoding to the categorical column.
+
+One Hot Encoding can be thought of as a way to represent categorical variables as binary vectors, in other words, a set of *new* columns for each categorical value with either a 1 or a 0 saying whether that value is true or not for that observation. These new columns would go into our model as input variables, and the original column is discarded.
+
+We also drop one of the new columns using the parameter *drop = "first"*. We do this to avoid the *dummy variable trap* where our newly created encoded columns perfectly predict each other - and we run the risk of breaking the assumption that there is no multicollinearity, a requirement or at least an important consideration for some models, Linear Regression being one of them! Multicollinearity occurs when two or more input variables are *highly* correlated with each other, it is a scenario we attempt to avoid as in short, while it won't necessarily affect the predictive accuracy of our model, it can make it difficult to trust the statistics around how well the model is performing, and how much each input variable is truly having.
+
+In the code, we also make sure to apply *fit_transform* to the training set, but only *transform* to the test set. This means the One Hot Encoding logic will *learn and apply* the "rules" from the training data, but only *apply* them to the test data. This is important in order to avoid *data leakage* where the test set *learns* information about the training data, and means we can't fully trust model performance metrics!
+
+For ease, after we have applied One Hot Encoding, we turn our training and test objects back into Pandas Dataframes, with the column names applied.
+
+
+```python
+
+# list of categorical variables that need encoding
+categorical_vars = ["gender"]
+
+# instantiate OHE class
+one_hot_encoder = OneHotEncoder(sparse=False, drop = "first")
+
+# apply OHE
+X_train_encoded = one_hot_encoder.fit_transform(X_train[categorical_vars])
+X_test_encoded = one_hot_encoder.transform(X_test[categorical_vars])
+
+# extract feature names for encoded columns
+encoder_feature_names = one_hot_encoder.get_feature_names_out(categorical_vars)
+
+# turn objects back to pandas dataframe
+X_train_encoded = pd.DataFrame(X_train_encoded, columns = encoder_feature_names)
+X_train = pd.concat([X_train.reset_index(drop=True), X_train_encoded.reset_index(drop=True)], axis = 1)
+X_train.drop(categorical_vars, axis = 1, inplace = True)
+
+X_test_encoded = pd.DataFrame(X_test_encoded, columns = encoder_feature_names)
+X_test = pd.concat([X_test.reset_index(drop=True), X_test_encoded.reset_index(drop=True)], axis = 1)
+X_test.drop(categorical_vars, axis = 1, inplace = True)
+
+```
+
+
+##### Feature Scaling
+
+As KNN is a *distance based* algorithm, in other words it is reliant on an understanding of how similar or different data points are across different dimensions in n-dimensional space, the application of *Feature Scaling* is extremely important.
+
+Feature Scaling is where we force the values from different columns to exist on the same scale, in order to enhance the learning capabilities of the model. There are two common approaches for this, Standardisation, and Normalisation.
+
+Standardisation rescales data to have a mean of 0, and a standard deviation of 1 - meaning most datapoints will most often fall between values of around -4 and +4.
+
+Normalisation rescales datapoints so that they exist in a range between 0 and 1.
+
+The below code uses the in-built *MinMaxScaler* functionality from scikit-learn to apply Normalisation to all of our input variables. The reason we choose Normalisation over Standardisation is that our scaled data will all exist between 0 and 1, and these will then be compatible with any categorical variables that we have encoded as 1's and 0's.
+
+In the code, we also make sure to apply *fit_transform* to the training set, but only *transform* to the test set. This means the scaling logic will learn and apply the scaling "rules" from the training data, but only apply them to the test data (or any other data we predict on in the future). This is important in order to avoid data leakage where the test set learns information about the training data, and means we can’t fully trust model performance metrics!
+
+
+```python
+
+# create our scaler object
+scale_norm = MinMaxScaler()
+
+# normalise the training set (using fit_transform)
+X_train = pd.DataFrame(scale_norm.fit_transform(X_train), columns = X_train.columns)
+
+# normalise the test set (using transform only)
+X_test = pd.DataFrame(scale_norm.transform(X_test), columns = X_test.columns)
+
+```
+
+
+##### Feature Selection
+
+As we discussed when applying Logistic Regression above - Feature Selection is the process used to select the input variables that are most important to your Machine Learning task. For more information around this, please see that section above.
+
+When applying KNN, Feature Selection is an interesting topic. The algorithm is measuring the distance between data-points across all dimensions, where each dimension is one of our input variables. The algorithm treats each input variable as equally important, there isn't really a concept of "feature importance" so the spread of data within an unimportant variable could have an effect on judging other data points as either "close" or "far". If we had a lot of "unimportant" variables in our data, this *could* create a lot of noise for the algorithm to deal with, and we'd just see poor classification accuracy without really knowing why.
+
+Having a high number of input variables also means the algorithm has to process a lot more information when processing distances between all of the data-points, so any way to reduce dimensionality is important from a computational perspective as well.
+
+For our task here we are again going to apply *Recursive Feature Elimination With Cross Validation (RFECV)* which is an approach that starts with all input variables, and then iteratively removes those with the weakest relationships with the output variable. RFECV does this using Cross Validation, so splits the data into many "chunks" and iteratively trains & validates models on each "chunk" separately. This means that each time we assess different models with different variables included, or eliminated, the algorithm also knows how accurate each of those models was. From the suite of model scenarios that are created, the algorithm can determine which provided the best accuracy, and thus can infer the best set of input variables to use!
+
+
+```python
+
+# instantiate RFECV & the model type to be utilised
+from sklearn.ensemble import RandomForestClassifier
+clf = RandomForestClassifier(random_state = 42)
+feature_selector = RFECV(clf)
+
+# fit RFECV onto our training & test data
+fit = feature_selector.fit(X_train,y_train)
+
+# extract & print the optimal number of features
+optimal_feature_count = feature_selector.n_features_
+print(f"Optimal number of features: {optimal_feature_count}")
+
+# limit our training & test sets to only include the selected variables
+X_train = X_train.loc[:, feature_selector.get_support()]
+X_test = X_test.loc[:, feature_selector.get_support()]
+
+```
+
+
+The below code then produces a plot that visualises the cross-validated classification accuracy with each potential number of features
+
+```python
+
+plt.style.use('seaborn-poster')
+plt.plot(range(1, len(fit.cv_results_['mean_test_score']) + 1), fit.cv_results_['mean_test_score'], marker = "o")
+plt.ylabel("Classification Accuracy")
+plt.xlabel("Number of Features")
+plt.title(f"Feature Selection using RFECV \n Optimal number of features is {optimal_feature_count} (at score of {round(max(fit.cv_results_['mean_test_score']),4)})")
+plt.tight_layout()
+plt.show()
+
+```
+
+
+This creates the below plot, which shows us that the highest cross-validated classification accuracy (0.9472) is when we include six of our original input variables - although there isn't much difference in predictive performance between using three variables through to eight variables - and this syncs with what we saw in the Random Forest section above where only three of the input variables scored highly when assessing Feature Importance & Permutation Importance.
+
+The variables that have been dropped are *total_items* and *credit score* - we will continue on with the remaining six!
+
+
+
+
+
+### Model Training
+
+Instantiating and training our KNN model is done using the below code. At this stage we will just use the default parameters, meaning that the algorithm:
+
+* Will use a value for k of 5, or in other words it will base classifications based upon the 5 nearest neighbours
+* Will use *uniform* weighting, or in other words an equal weighting to all 5 neighbours regardless of distance
+
+```python
+
+# instantiate our model object
+clf = KNeighborsClassifier()
+
+# fit our model using our training & test sets
+clf.fit(X_train, y_train)
+
+```
+
+
+### Model Performance Assessment
+
+##### Predict On The Test Set
+
+To assess how well our model is predicting on new data - we use the trained model object (here called *clf*) and ask it to predict the *signup_flag* variable for the test set.
+
+In the code below we create one object to hold the binary 1/0 predictions, and another to hold the actual prediction probabilities for the positive class (which is based upon the majority class within the k nearest neighbours)
+
+```python
+
+# predict on the test set
+y_pred_class = clf.predict(X_test)
+y_pred_prob = clf.predict_proba(X_test)[:,1]
+
+```
+
+
+##### Confusion Matrix
+
+As we've seen with all models so far, our Confusion Matrix provides us a visual way to understand how our predictions match up against the actual values for those test set observations.
+
+The below code creates the Confusion Matrix using the *confusion_matrix* functionality from within scikit-learn and then plots it using matplotlib.
+
+```python
+
+# create the confusion matrix
+conf_matrix = confusion_matrix(y_test, y_pred_class)
+
+# plot the confusion matrix
+plt.style.use("seaborn-poster")
+plt.matshow(conf_matrix, cmap = "coolwarm")
+plt.gca().xaxis.tick_bottom()
+plt.title("Confusion Matrix")
+plt.ylabel("Actual Class")
+plt.xlabel("Predicted Class")
+for (i, j), corr_value in np.ndenumerate(conf_matrix):
+ plt.text(j, i, corr_value, ha = "center", va = "center", fontsize = 20)
+plt.show()
+
+```
+
+
+
+
+
+The aim is to have a high proportion of observations falling into the top left cell (predicted non-signup and actual non-signup) and the bottom right cell (predicted signup and actual signup).
+
+The results here are interesting - all of the errors are where the model incorrectly classified *delivery club* signups as non-signups - the model made no errors when classifying non-signups non-signups.
+
+Since the proportion of signups in our data was around 30:70 we will next analyse not only Classification Accuracy, but also Precision, Recall, and F1-Score which will help us assess how well our model has performed in reality.
+
+
+##### Classification Performance Metrics
+
+**Accuracy, Precision, Recall, F1-Score**
+
+For details on these performance metrics, please see the above section on Logistic Regression. Using all four of these metrics in combination gives a really good overview of the performance of a classification model, and gives us an understanding of the different scenarios & considerations!
+
+In the code below, we utilise in-built functionality from scikit-learn to calculate these four metrics.
+
+```python
+
+# classification accuracy
+accuracy_score(y_test, y_pred_class)
+
+# precision
+precision_score(y_test, y_pred_class)
+
+# recall
+recall_score(y_test, y_pred_class)
+
+# f1-score
+f1_score(y_test, y_pred_class)
+
+```
+
+Running this code gives us:
+
+* Classification Accuracy = **0.936** meaning we correctly predicted the class of 93.6% of test set observations
+* Precision = **1.00** meaning that for our *predicted* delivery club signups, we were correct 100% of the time
+* Recall = **0.762** meaning that of all *actual* delivery club signups, we predicted correctly 76.2% of the time
+* F1-Score = **0.865**
+
+These are interesting. The KNN has obtained the highest overall Classification Accuracy & Precision, but the lower Recall score has penalised the F1-Score meaning that is actually lower than what was seen for both the Decision Tree & the Random Forest!
+
+
+### Finding The Optimal Value For k
+
+By default, the KNN algorithm within scikit-learn will use k = 5 meaning that classifications are based upon the five nearest neighbouring data-points in n-dimensional space.
+
+Just because this is the default threshold *does not mean* it is the best one for our task.
+
+Here, we will test many potential values for k, and plot the Precision, Recall & F1-Score, and find an optimal solution!
+
+
+```python
+
+# set up range for search, and empty list to append accuracy scores to
+k_list = list(range(2,25))
+accuracy_scores = []
+
+# loop through each possible value of k, train and validate model, append test set f1-score
+for k in k_list:
+
+ clf = KNeighborsClassifier(n_neighbors = k)
+ clf.fit(X_train,y_train)
+ y_pred = clf.predict(X_test)
+ accuracy = f1_score(y_test,y_pred)
+ accuracy_scores.append(accuracy)
+
+# store max accuracy, and optimal k value
+max_accuracy = max(accuracy_scores)
+max_accuracy_idx = accuracy_scores.index(max_accuracy)
+optimal_k_value = k_list[max_accuracy_idx]
+
+# plot accuracy by max depth
+plt.plot(k_list,accuracy_scores)
+plt.scatter(optimal_k_value, max_accuracy, marker = "x", color = "red")
+plt.title(f"Accuracy (F1 Score) by k \n Optimal Value for k: {optimal_k_value} (Accuracy: {round(max_accuracy,4)})")
+plt.xlabel("k")
+plt.ylabel("Accuracy (F1 Score)")
+plt.tight_layout()
+plt.show()
+
+```
+
+That code gives us the below plot - which visualises the results!
+
+
+
+
+
+In the plot we can see that the *maximum* F1-Score on the test set is found when applying a k value of 5 - which is exactly what we started with, so nothing needs to change!
+
+___
+
+# Modelling Summary
+
+The goal for the project was to build a model that would accurately predict the customers that would sign up for the *delivery club*. This would allow for a much more targeted approach when running the next iteration of the campaign. A secondary goal was to understand what the drivers for this are, so the client can get closer to the customers that need or want this service, and enhance their messaging.
+
+Based upon these, the chosen the model is the Random Forest as it was a) the most consistently performant on the test set across classification accuracy, precision, recall, and f1-score, and b) the feature importance and permutation importance allows the client an understanding of the key drivers behind *delivery club* signups.
+
+
+**Metric 1: Classification Accuracy**
+
+* KNN = 0.936
+* Random Forest = 0.935
+* Decision Tree = 0.929
+* Logistic Regression = 0.866
+
+
+**Metric 2: Precision**
+
+* KNN = 1.00
+* Random Forest = 0.887
+* Decision Tree = 0.885
+* Logistic Regression = 0.784
+
+
+**Metric 3: Recall**
+
+* Random Forest = 0.904
+* Decision Tree = 0.885
+* KNN = 0.762
+* Logistic Regression = 0.69
+
+
+**Metric 4: F1 Score**
+
+* Random Forest = 0.895
+* Decision Tree = 0.885
+* KNN = 0.865
+* Logistic Regression = 0.734
+
+___
+
+# Application
+
+We now have a model object, and a the required pre-processing steps to use this model for the next *delivery club* campaign. When this is ready to launch we can aggregate the necessary customer information and pass it through, obtaining predicted probabilities for each customer signing up.
+
+Based upon this, we can work with the client to discuss where their budget can stretch to, and contact only the customers with a high propensity to join. This will drastically reduce marketing costs, and result in a much improved ROI.
+
+___
+
+# Growth & Next Steps
+
+While predictive accuracy was relatively high - other modelling approaches could be tested, especially those somewhat similar to Random Forest, for example XGBoost, LightGBM to see if even more accuracy could be gained.
+
+We could even look to tune the hyperparameters of the Random Forest, notably regularisation parameters such as tree depth, as well as potentially training on a higher number of Decision Trees in the Random Forest.
+
+From a data point of view, further variables could be collected, and further feature engineering could be undertaken to ensure that we have as much useful information available for predicting customer loyalty
diff --git a/_posts/2026-08-01-customer-segmentation.md b/_posts/2026-08-01-customer-segmentation.md
new file mode 100644
index 000000000..fa5a2ee79
--- /dev/null
+++ b/_posts/2026-08-01-customer-segmentation.md
@@ -0,0 +1,369 @@
+---
+layout: post
+title: The "You Are What You Eat" Customer Segmentation
+image: "/posts/clustering-title-img.png"
+tags: [Customer Segmentation, Machine Learning, Clustering, Python]
+---
+
+In this project we use k-means clustering to segment up the customer base in order to increase business understanding, and to enhance the relevancy of targeted messaging & customer communications.
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results](#overview-results)
+ - [Growth/Next Steps](#overview-growth)
+- [01. Data Overview](#data-overview)
+- [02. K-Means](#kmeans-title)
+ - [Concept Overview](#kmeans-overview)
+ - [Data Preprocessing](#kmeans-preprocessing)
+ - [Finding A Good Value For K](#kmeans-k-value)
+ - [Model Fitting](#kmeans-model-fitting)
+ - [Appending Clusters To Customers](#kmeans-append-clusters)
+ - [Segment Profiling](#kmeans-cluster-profiling)
+- [03. Application](#kmeans-application)
+- [04. Growth & Next Steps](#growth-next-steps)
+
+___
+
+# Project Overview
+
+### Context
+
+The Senior Management team from our client, a supermarket chain, are disagreeing about how customers are shopping, and how lifestyle choices may affect which food areas customers are shopping into, or more interestingly, not shopping into.
+
+They have asked us to use data, and Machine Learning to help segment up their customers based upon their engagement with each of the major food categories - aiding business understanding of the customer base, and to enhance the relevancy of targeted messaging & customer communications.
+
+
+
+### Actions
+
+We firstly needed to compile the necessary data from several tables in the database, namely the *transactions* table and the *product_areas* table. We joined together the relevant information using Pandas, and then aggregated the transactional data across product areas, from the most recent six month to a customer level. The final data for clustering is, for each customer, the percentage of sales allocated to each product area.
+
+As a starting point, we test & apply k-means clustering for this task. We need to apply some data pre-processing, most importantly feature scaling to ensure all variables exist on the same scale - a very important consideration for distance based algorithms such as k-means.
+
+As k-means is an *unsupervised learning* approach, in other words there are no labels - we use a process known as *Within Cluster Sum of Squares (WCSS)* to understand what a "good" number of clusters or segments is.
+
+Based upon this, we apply the k-means algorithm onto the product area data, append the clusters to our customer base, and then profile the resulting customer segments to understand what the differentiating factors were!
+
+
+
+### Results
+
+Based upon iterative testing using WCSS we settled on a customer segmentation with 3 clusters. These clusters ranged in size, with Cluster 0 accounting for 73.6% of the customer base, Cluster 2 accounting for 14.6%, and Cluster 1 accounting for 11.8%.
+
+There were some extremely interesting findings from profiling the clusters.
+
+For *Cluster 0* we saw a significant portion of spend being allocated to each of the product areas - showing customers without any particular dietary preference.
+
+For *Cluster 1* we saw quite high proportions of spend being allocated to Fruit & Vegetables, but very little to the Dairy & Meat product areas. It could be hypothesised that these customers are following a vegan diet.
+
+Finally customers in *Cluster 2* spent significant portions within Dairy, Fruit & Vegetables, but very little in the Meat product area - so similarly, we would make an early hypothesis that these customers are more along the lines of those following a vegetarian diet.
+
+To help embed this segmentation into the business, we have proposed to call this the "You Are What You Eat" segmentation.
+
+
+
+### Growth/Next Steps
+
+It would be interesting to run this clustering/segmentation at a lower level of product areas, so rather than just the four areas of Meat, Dairy, Fruit, Vegetables - clustering spend across the sub-categories *below* those categories. This would mean we could create more specific clusters, and get an even more granular understanding of dietary preferences within the customer base.
+
+Here we've just focused on variables that are linked directly to sales - it could be interesting to also include customer metrics such as distance to store, gender etc to give a even more well-rounded customer segmentation.
+
+It would be useful to test other clustering approaches such as hierarchical clustering or DBSCAN to compare the results.
+
+
+
+___
+
+# Data Overview
+
+We are primarily looking to discover segments of customers based upon their transactions within *food* based product areas so we will need to only select those.
+
+In the code below, we:
+
+* Import the required python packages & libraries
+* Import the tables from the database
+* Merge the tables to tag on *product_area_name* which only exists in the *product_areas* table
+* Drop the non-food categories
+* Aggregate the sales data for each product area, at customer level
+* Pivot the data to get it into the right format for clustering
+* Change the values from raw dollars, into a percentage of spend for each customer (to ensure each customer is comparable)
+
+
+```python
+
+# import required Python packages
+from sklearn.cluster import KMeans
+from sklearn.preprocessing import MinMaxScaler
+import pandas as pd
+import matplotlib.pyplot as plt
+
+# import tables from database
+transactions = ...
+product_areas = ...
+
+# merge product_area_name on
+transactions = pd.merge(transactions, product_areas, how = "inner", on = "product_area_id")
+
+# drop the non-food category
+transactions.drop(transactions[transactions["product_area_name"] == "Non-Food"].index, inplace = True)
+
+# aggregate sales at customer level (by product area)
+transaction_summary = transactions.groupby(["customer_id", "product_area_name"])["sales_cost"].sum().reset_index()
+
+# pivot data to place product areas as columns
+transaction_summary_pivot = transactions.pivot_table(index = "customer_id",
+ columns = "product_area_name",
+ values = "sales_cost",
+ aggfunc = "sum",
+ fill_value = 0,
+ margins = True,
+ margins_name = "Total").rename_axis(None,axis = 1)
+
+# transform sales into % sales
+transaction_summary_pivot = transaction_summary_pivot.div(transaction_summary_pivot["Total"], axis = 0)
+
+# drop the "total" column as we don't need that for clustering
+data_for_clustering = transaction_summary_pivot.drop(["Total"], axis = 1)
+
+```
+
+
+After the data pre-processing using Pandas, we have a dataset for clustering that looks like the below sample:
+
+
+
+| **customer_id** | **dairy** | **fruit** | **meat** | **vegetables** |
+|---|---|---|---|---|
+| 2 | 0.246 | 0.198 | 0.394 | 0.162 |
+| 3 | 0.142 | 0.233 | 0.528 | 0.097 |
+| 4 | 0.341 | 0.245 | 0.272 | 0.142 |
+| 5 | 0.213 | 0.250 | 0.430 | 0.107 |
+| 6 | 0.180 | 0.178 | 0.546 | 0.095 |
+| 7 | 0.000 | 0.517 | 0.000 | 0.483 |
+
+
+The data is at customer level, and we have a column for each of the highest level food product areas. Within each of those we have the *percentage* of sales that each customer allocated to that product area over the past six months.
+
+___
+
+# K-Means
+
+
+### Concept Overview
+
+K-Means is an *unsupervised learning* algorithm, meaning that it does not look to predict known labels or values, but instead looks to isolate patterns within unlabelled data.
+
+The algorithm works in a way where it partitions data-points into distinct groups (clusters) based upon their *similarity* to each other.
+
+This similarity is most often the Euclidean (straight-line) distance between data-points in n-dimensional space. Each variable that is included lies on one of the dimensions in space.
+
+The number of distinct groups (clusters) is determined by the value that is set for "k".
+
+The algorithm does this by iterating over four key steps, namely:
+
+1. It selects "k" random points in space (these points are known as centroids)
+2. It then assigns each of the data points to the nearest centroid (based upon Euclidean distance)
+3. It then repositions the centroids to the *mean* dimension values of it's cluster
+4. It then reassigns each data-point to the nearest centroid
+
+Steps 3 & 4 continue to iterate until no data-points are reassigned to a closer centroid.
+
+
+### Data Preprocessing
+
+There are three vital preprocessing steps for k-means, namely:
+
+* Missing values in the data
+* The effect of outliers
+* Feature Scaling
+
+
+##### Missing Values
+
+Missing values can cause issues for k-means, as the algorithm won't know where to plot those data-points along the dimension where the value is not present. If we have observations with missing values, the most common options are to either remove the observations, or to use an imputer to fill-in or to estimate what those value might be.
+
+As we aggregated our data for each customer, we actually don't suffer from missing values so we don't need to deal with that here.
+
+
+##### Outliers
+
+As k-means is a distance based algorithm, outliers can cause problems. The main issue we face is when we come to scale our input variables, a very important step for a distance based algorithm.
+
+We don’t want any variables to be “bunched up” due to a single outlier value, as this will make it hard to compare their values to the other input variables. We should always investigate outliers rigorously - however in our case where we're dealing with percentages, we thankfully don't face this issue!
+
+
+##### Feature Scaling
+
+Again, as k-means is a distance based algorithm, in other words it is reliant on an understanding of how similar or different data points are across different dimensions in n-dimensional space, the application of Feature Scaling is extremely important.
+
+Feature Scaling is where we force the values from different columns to exist on the same scale, in order to enhance the learning capabilities of the model. There are two common approaches for this, Standardisation, and Normalisation.
+
+Standardisation rescales data to have a mean of 0, and a standard deviation of 1 - meaning most datapoints will most often fall between values of around -4 and +4.
+
+Normalisation rescales datapoints so that they exist in a range between 0 and 1.
+
+For k-means clustering, either approach is going to be *far better* than using no scaling at all. Here, we will look to apply normalisation as this will ensure all variables will end up having the same range, fixed between 0 and 1, and therefore the k-means algorithm can judge each variable in the same context. Standardisation *can* result in different ranges, variable to variable, and this is not so useful (although this isn't explicitly true in all scenarios).
+
+Another reason for choosing Normalisation over Standardisation is that our scaled data will *all* exist between 0 and 1, and these will then be compatible with any categorical variables that we have encoded as 1’s and 0’s (although we don't have any variables of this type in our task here).
+
+In our specific task here, we are using percentages, so our values are _already_ spread between 0 and 1. We will still apply normalisation for the following reason. One of the product areas might commonly make up a large proportion of customer sales, and this may end up dominating the clustering space. If we normalise all of our variables, even product areas that make up smaller volumes, will be spread proportionately between 0 and 1!
+
+The below code uses the in-built MinMaxScaler functionality from scikit-learn to apply Normalisation to all of our variables. The reason we create a new object (here called data_for_clustering_scaled) is that we want to use the scaled data for clustering, but when profiling the clusters later on, we may want to use the actual percentages as this may make more intuitive business sense, so it's good to have both options available!
+
+```python
+
+# create our scaler object
+scale_norm = MinMaxScaler()
+
+# normalise the data
+data_for_clustering_scaled = pd.DataFrame(scale_norm.fit_transform(data_for_clustering), columns = data_for_clustering.columns)
+
+```
+
+
+### Finding A Good Value For k
+
+At this point here, our data is ready to be fed into the k-means clustering algorithm. Before that however, we want to understand what number of clusters we want the data split into.
+
+In the world of unsupervised learning, there is no *right or wrong* value for this - it really depends on the data you are dealing with, as well as the unique scenario you're utilising the algorithm for. From our client, having a very high number of clusters might not be appropriate as it would be too hard for the business to understand the nuance of each in a way where they can apply the right strategies.
+
+Finding the "right" value for k, can feel more like art than science, but there are some data driven approaches that can help us!
+
+The approach we will utilise here is known as *Within Cluster Sum of Squares (WCSS)* which measures the sum of the squared euclidean distances that data points lie from their closest centroid. WCSS can help us understand the point where adding *more clusters* provides little extra benefit in terms of separating our data.
+
+By default, the k-means algorithm within scikit-learn will use k = 8 meaning that it will look to split the data into eight distinct clusters. We want to find a better value that fits our data, and our task!
+
+In the code below we will test multiple values for k, and plot how this WCSS metric changes. As we increase the value for k (in other words, as we increase the number or centroids or clusters) the WCSS value will always decrease. However, these decreases will get smaller and smaller each time we add another centroid and we are looking for a point where this decrease is quite prominent *before* this point of diminishing returns.
+
+```python
+
+# set up range for search, and empty list to append wcss scores to
+k_values = list(range(1,10))
+wcss_list = []
+
+# loop through each possible value of k, fit to the data, append the wcss score
+for k in k_values:
+ kmeans = KMeans(n_clusters = k, random_state = 42)
+ kmeans.fit(data_for_clustering_scaled)
+ wcss_list.append(kmeans.inertia_)
+
+# plot wcss by k
+plt.plot(k_values, wcss_list)
+plt.title("Within Cluster Sum of Squares - by k")
+plt.xlabel("k")
+plt.ylabel("WCSS Score")
+plt.tight_layout()
+plt.show()
+
+```
+
+That code gives us the below plot - which visualises our results!
+
+
+
+
+
+Based upon the shape of the above plot - there does appear to be an elbow at k = 3. Prior to that we see a significant drop in the WCSS score, but following the decreases are much smaller, meaning this could be a point that suggests adding *more clusters* will provide little extra benefit in terms of separating our data. A small number of clusters can be beneficial when considering how easy it is for the business to focus on, and understand, each - so we will continue on, and fit our k-means clustering solution with k = 3.
+
+
+### Model Fitting
+
+The below code will instantiate our k-means object using a value for k equal to 3. We then fit this object to our scaled dataset to separate our data into three distinct segments or clusters.
+
+```python
+
+# instantiate our k-means object
+kmeans = KMeans(n_clusters = 3, random_state = 42)
+
+# fit to our data
+kmeans.fit(data_for_clustering_scaled)
+
+```
+
+
+### Append Clusters To Customers
+
+With the k-means algorithm fitted to our data, we can now append those clusters to our original dataset, meaning that each customer will be tagged with the cluster number that they most closely fit into based upon their sales data over each product area.
+
+In the code below we tag this cluster number onto our original dataframe.
+
+```python
+
+# add cluster labels to our original data
+data_for_clustering["cluster"] = kmeans.labels_
+
+```
+
+
+### Cluster Profiling
+
+Once we have our data separated into distinct clusters, our client needs to understand *what is is* that is driving the separation. This means the business can understand the customers within each, and the behaviours that make them unique.
+
+
+##### Cluster Sizes
+
+In the below code we firstly assess the number of customers that fall into each cluster.
+
+
+```python
+
+# check cluster sizes
+data_for_clustering["cluster"].value_counts(normalize=True)
+
+```
+
+
+Running that code shows us that the three clusters are different in size, with the following proportions:
+
+* Cluster 0: **73.6%** of customers
+* Cluster 2: **14.6%** of customers
+* Cluster 1: **11.8%** of customers
+
+Based on these results, it does appear we do have a skew toward Cluster 0 with Cluster 1 & Cluster 2 being proportionally smaller. This isn't right or wrong, it is simply showing up pockets of the customer base that are exhibiting different behaviours - and this is *exactly* what we want.
+
+
+##### Cluster Attributes
+
+To understand what these different behaviours or characteristics are, we can look to analyse the attributes of each cluster, in terms of the variables we fed into the k-means algorithm.
+
+
+```python
+
+# profile clusters (mean % sales for each product area)
+cluster_summary = data_for_clustering.groupby("cluster")[["Dairy","Fruit","Meat","Vegetables"]].mean().reset_index()
+
+```
+
+That code results in the following table...
+
+| **Cluster** | **Dairy** | **Fruit** | **Meat** | **Vegetables** |
+|---|---|---|---|---|
+| 0 | 22.1% | 26.5% | 37.7% | 13.8% |
+| 1 | 0.2% | 63.8% | 0.4% | 35.6% |
+| 2 | 36.4% | 39.4% | 2.9% | 21.3% |
+
+
+For *Cluster 0* we see a reasonably significant portion of spend being allocated to each of the product areas. For *Cluster 1* we see quite high proportions of spend being allocated to Fruit & Vegetables, but very little to the Dairy & Meat product areas. It could be hypothesised that these customers are following a vegan diet. Finally customers in *Cluster 2* spend, on average, significant portions within Dairy, Fruit & Vegetables, but very little in the Meat product area - so similarly, we would make an early hypothesis that these customers are more along the lines of those following a vegetarian diet - very interesting!
+
+___
+
+# Application
+
+Even though this is a simple solution, based upon high level product areas it will help leaders in the business, and category managers gain a clearer understanding of the customer base.
+
+Tracking these clusters over time would allow the client to more quickly react to dietary trends, and adjust their messaging and inventory accordingly.
+
+Based upon these clusters, the client will be able to target customers more accurately - promoting products & discounts to customers that are truly relevant to them - overall enabling a more customer focused communication strategy.
+
+___
+
+# Growth & Next Steps
+
+It would be interesting to run this clustering/segmentation at a lower level of product areas, so rather than just the four areas of Meat, Dairy, Fruit, Vegetables - clustering spend across the sub-categories *below* those categories. This would mean we could create more specific clusters, and get an even more granular understanding of dietary preferences within the customer base.
+
+Here we've just focused on variables that are linked directly to sales - it could be interesting to also include customer metrics such as distance to store, gender etc to give an even more well-rounded customer segmentation.
+
+It would be useful to test other clustering approaches such as hierarchical clustering or DBSCAN to compare the results.
diff --git a/_posts/2026-08-05-pca.md b/_posts/2026-08-05-pca.md
new file mode 100644
index 000000000..a804aebf6
--- /dev/null
+++ b/_posts/2026-08-05-pca.md
@@ -0,0 +1,367 @@
+---
+layout: post
+title: Compressing Feature Space For Classification Using PCA
+image: "/posts/pca-title-img.png"
+tags: [PCA, Machine Learning, Python]
+---
+
+In this project we use Principal Component Analysis (PCA) to compress 100 unlabelled, sparse features into a more manageable number for classifying buyers of Ed Sheeran's latest album.
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results](#overview-results)
+ - [Growth/Next Steps](#overview-growth)
+- [01. Data Overview](#data-overview)
+- [02. PCA Overview](#pca-overview)
+- [03. Data Preparation](#pca-data-prep)
+- [04. Fitting PCA](#pca-fit)
+- [05. Analysis Of Explained Variance](#pca-variance)
+- [06. Applying our PCA solution](#pca-application)
+- [07. Classification Model](#pca-classification)
+- [08. Growth & Next Steps](#growth-next-steps)
+
+___
+
+# Project Overview
+
+### Context
+
+Our client is looking to promote Ed Sheeran's new album - and want to be both targeted with their customer communications, and as efficient as possible with their marketing budget.
+
+As a proof-of-concept they would like us to build a classification model for customers who purchased Ed's *last* album based upon a small sample of listening data they have acquired for some of their customers at that time.
+
+If we can do this successfully, they will look to purchase up-to-date listening data, apply the model, and use the predicted probabilities to promote to customers who are most likely to purchase.
+
+The sample data is short but wide. It contains only 356 customers, but for each, columns that represent the percentage of historical listening time allocated to each of 100 artists. On top of these, the 100 columns do not contain the artist in question, instead being labelled *artist1, artist2* etc.
+
+We will need to compress this data into something more manageable for classification!
+
+
+
+### Actions
+
+We firstly needed to bring in the required data, both the historical listening sample, and the flag showing which customers purchased Ed Sheeran's last album. We ensure we split our data into training set & a test set, for classification purposes. For PCA, we ensure that we scale the data so that all features exist on the same scale.
+
+We then apply PCA without any specified number of components - which allows us to examine & plot the percentage of explained variance for every number of components. Based upon this we make a call to limit our dataset to the number of components that make up 75% of the variance of the initial feature set (rather than limiting to a specific number of components). We apply this rule to both our training set (using fit_transform) and our test set (using transform only)
+
+With this new, compressed dataset, we apply a Random Forest Classifier to predict the sales of the album, and we assess the predictive performance!
+
+
+
+
+### Results
+
+Based upon an analysis of variance vs. components - we made a call to keep 75% of the variance of the initial feature set, which meant we dropped the number of features from 100 down to 24.
+
+Using these 24 components, we trained a Random Forest Classifier which was able to predict customers that would purchase Ed Sheeran's last album with a Classification Accuracy of 93%
+
+
+
+### Growth/Next Steps
+
+We only tested one type of classifier here (Random Forest) - it would be worthwhile testing others. We also only used the default classifier hyperparameters - we would want to optimise these.
+
+Here, we selected 24 components based upon the fact this accounted for 75% of the variance of the initial feature set. We would instead look to search for the optimal number of components to use based upon classification accuracy.
+
+
+
+
+___
+
+# Data Overview
+
+Our dataset contains only 356 customers, but 102 columns.
+
+In the code below, we:
+
+* Import the required python packages & libraries
+* Import the data from the database
+* Drop the ID column for each customer
+* Shuffle the dataset
+* Analyse the class balance between album buyers, and non-album buyers
+
+
+```python
+
+# import required Python packages
+import pandas as pd
+import matplotlib.pyplot as plt
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.utils import shuffle
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import accuracy_score
+from sklearn.preprocessing import StandardScaler
+from sklearn.decomposition import PCA
+
+# import data
+data_for_model = ...
+
+# drop the id column
+data_for_model.drop("user_id", axis = 1, inplace = True)
+
+# shuffle the data
+data_for_model = shuffle(data_for_model, random_state = 42)
+
+# analyse the class balance
+data_for_model["purchased_album"].value_counts(normalize = True)
+
+```
+
+
+From the last step in the above code, we see that 53% of customers in our sample did purchase Ed's last album, and 47% did not. Since this is evenly balanced, we can most likely rely solely on *Classification Accuracy* when assessing the performance of the classification model later on.
+
+After these steps, we have a dataset that looks like the below sample (not all columns shown):
+
+
+
+| **purchased_album** | **artist1** | **artist2** | **artist3** | **artist4** | **artist5** | **artist6** | **artist7** | **…** |
+|---|---|---|---|---|---|---|---|---|
+| 1 | 0.0278 | 0 | 0 | 0 | 0 | 0.0036 | 0.0002 | … |
+| 1 | 0 | 0 | 0.0367 | 0.0053 | 0 | 0 | 0.0367 | … |
+| 1 | 0.0184 | 0 | 0 | 0 | 0 | 0 | 0 | … |
+| 0 | 0.0017 | 0.0226 | 0 | 0 | 0 | 0 | 0 | … |
+| 1 | 0.0002 | 0 | 0 | 0 | 0 | 0 | 0 | … |
+| 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | … |
+| 1 | 0.0042 | 0 | 0 | 0 | 0 | 0 | 0 | … |
+| 0 | 0 | 0 | 0.0002 | 0 | 0 | 0 | 0 | … |
+| 1 | 0 | 0 | 0 | 0 | 0.1759 | 0 | 0 | … |
+| 1 | 0.0001 | 0 | 0.0001 | 0 | 0 | 0 | 0 | … |
+| 1 | 0 | 0 | 0 | 0.0555 | 0 | 0.0003 | 0 | … |
+| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | … |
+| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | … |
+
+
+The data is at customer level. We have a binary column showing whether the customer purchased the prior album or not, and following that 100 columns containing the percentage of historical listening time allocated to each artist. We do not know the names of these artists.
+
+From the above sample, we can also see the sparsity of the data, customers do not listen to all artists and therefore many of the values are 0.
+
+___
+
+# PCA Overview
+
+Principal Component Analysis (PCA) is often used as a *Dimensionality Reduction* technique that can reduce a large set of variables down to a smaller set, that still contains most of the original information.
+
+In other words, PCA takes a high number of dimensions, or variables and boils them down into a much smaller number of new variables - each of which is called a *principal component*. These new *components* are somewhat abstract - they are a blend of some of the original features where the PCA algorithm found they were correlated. By blending the original variables rather than just removing them, the hope is that we still keep much of the key information that was held in the original feature set.
+
+Dimensionality Reduction techniques like PCA are mainly used to simplify the space in which we're operating. Attempting to apply the k-means clustering algorithm (for example) across hundreds or thousands of features can be computationally expensive, PCA reduces this vastly while maintaining much of the key information contained in the data. But PCA doesn’t have applications just within the realms of unsupervised learning, it could just as easily be applied to a set of input variables in a supervised learning approach - exactly like we will do here!
+
+In supervised learning, we often focus on *Feature Selection* where we look to remove variables that are not deemed to be important in predicting our output. PCA is often used in a similar way, although in this case we aren't explicitly *removing* variables - we are simply creating a smaller number of *new* ones that contain much of the information contained in the original set.
+
+Business consideration of PCA: It is much more difficult to interpret the outputs of a predictive model (for example) that is based upon component values versus the original variables.
+
+___
+
+# Data Preparation
+
+
+##### Split Out Data For Modelling
+
+In the next code block we do two things, we firstly split our data into an X object which contains only the predictor variables, and a y object that contains only our dependent variable.
+
+Once we have done this, we split our data into training and test sets to ensure we can fairly validate the accuracy of the predictions on data that was not used in training. In this case, we have allocated 80% of the data for training, and the remaining 20% for validation. We make sure to add in the stratify parameter to ensure that both our training and test sets have the same proportion of customers who did, and did not, purchase the album - meaning we can be more confident in our assessment of predictive performance.
+
+```python
+
+# split data into X and y objects for modelling
+X = data_for_model.drop(["purchased_album"], axis = 1)
+y = data_for_model["purchased_album"]
+
+# split out training & test sets
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42, stratify = y)
+
+```
+
+
+##### Feature Scaling
+
+Feature Scaling is extremely important when applying PCA - it means that the algorithm can successfully "judge" the correlations between the variables and effectively create the principal components for us. The general consensus is to apply Standardisation rather than Normalisation.
+
+The below code uses the in-built StandardScaler functionality from scikit-learn to apply Standardisation to all of our variables.
+
+In the code, we use *fit_transform* for the training set, but only *transform* to the test set. This means the standardisation logic will learn and apply the “rules” from the training data, but only apply them to the test data. This is important in order to avoid data leakage where the test set learns information about the training data, and means we can’t fully trust model performance metrics!
+
+```python
+
+# create our scaler object
+scale_standard = StandardScaler()
+
+# standardise the data
+X_train = scale_standard.fit_transform(X_train)
+X_test = scale_standard.transform(X_test)
+
+```
+
+___
+
+# Fitting PCA
+
+We firstly apply PCA to our training set without limiting the algorithm to any particular number of components, in other words we're not explicitly reducing the feature space at this point.
+
+Allowing all components to be created here allows us to examine & plot the percentage of explained variance for each, and assess which solution might work best for our task.
+
+In the code below we instantiate our PCA object, and then fit it to our training set.
+
+```python
+
+# instantiate our PCA object (no limit on components)
+pca = PCA(n_components = None, random_state = 42)
+
+# fit to our training data
+pca.fit(X_train)
+
+```
+
+___
+
+# Analysis Of Explained Variance
+
+There is no right or wrong number of components to use - this is something that we need to decide based upon the scenario we're working in. We know we want to reduce the number of features, but we need to trade this off with the amount of information we lose.
+
+In the following code, we extract this information from the prior step where we fit the PCA object to our training data. We extract the variance for each component, and we do the same again, but for the *cumulative* variance. Will will assess & plot both of these in the next step.
+
+```python
+
+# explained variance across components
+explained_variance = pca.explained_variance_ratio_
+
+# explained variance across components (cumulative)
+explained_variance_cumulative = pca.explained_variance_ratio_.cumsum()
+
+```
+
+
+In the following code, we create two plots - one for the variance of each principal component, and one for the cumulative variance.
+
+```python
+
+num_vars_list = list(range(1,101))
+plt.figure(figsize=(16,9))
+
+# plot the variance explained by each component
+plt.subplot(2,1,1)
+plt.bar(num_vars_list,explained_variance)
+plt.title("Variance across Principal Components")
+plt.xlabel("Number of Components")
+plt.ylabel("% Variance")
+plt.tight_layout()
+
+# plot the cumulative variance
+plt.subplot(2,1,2)
+plt.plot(num_vars_list,explained_variance_cumulative)
+plt.title("Cumulative Variance across Principal Components")
+plt.xlabel("Number of Components")
+plt.ylabel("Cumulative % Variance")
+plt.tight_layout()
+plt.show()
+
+```
+
+As we can see in the top plot, PCA works in a way where the first component holds the most variance, and each subsequent component holds less and less.
+
+The second plot shows this as a cumulative measure - and we can see how many components we would need remain in order to keep any amount of variance from the original feature set.
+
+
+
+
+
+Based upon the cumulative plot above, we can see that we could keep 75% of the variance from the original feature set with only around 25 components, in other words with only a quarter of the number of features we can still hold onto around three-quarters of the information.
+
+___
+
+# Applying our PCA solution
+
+Now we've run our analysis of variance by component - we can apply our PCA solution.
+
+In the code below - we *re-instantiate* our PCA object, this time specifying that we want the number of components that will keep 75% of the initial variance.
+
+We then apply this solution to both our training set (using fit_transform) and our test set (using transform only).
+
+Finally - based on this 75% threshold, we confirm the number of components that this leaves us with.
+
+```python
+
+# re-instantiate our PCA object (keeping 75% of variance)
+pca = PCA(n_components = 0.75, random_state = 42)
+
+# fit to our data
+X_train = pca.fit_transform(X_train)
+X_test = pca.transform(X_test)
+
+# check the number of components
+print(pca.n_components_)
+
+```
+
+
+Turns out we were almost correct from looking at our chart - we will retain 75% of the information from our initial feature set, with only 24 principal components.
+
+Our X_train and X_test objects now contain 24 columns, each representing one of the principal components - we can see a sample of X_train below:
+
+| **0** | **1** | **2** | **3** | **4** | **5** | **6** | **…** |
+|---|---|---|---|---|---|---|---|
+| -0.402194 | -0.756999 | 0.219247 | -0.0995449 | 0.0527621 | 0.0968236 | -0.0500932 | … |
+| -0.360072 | -1.13108 | 0.403249 | -0.573797 | -0.18079 | -0.305604 | -1.33653 | … |
+| 10.6929 | -0.866574 | 0.711987 | 0.168807 | -0.333284 | 0.558677 | 0.861932 | … |
+| -0.47788 | -0.688505 | 0.0876652 | -0.0656084 | -0.0842425 | 1.06402 | 0.309337 | … |
+| -0.258285 | -0.738503 | 0.158456 | -0.0864722 | -0.0696632 | 1.79555 | 0.583046 | … |
+| -0.440366 | -0.564226 | 0.0734247 | -0.0372701 | -0.0331369 | 0.204862 | 0.188869 | … |
+| -0.56328 | -1.22408 | 1.05047 | -0.931397 | -0.353803 | -0.565929 | -2.4482 | … |
+| -0.282545 | -0.379863 | 0.302378 | -0.0382711 | 0.133327 | 0.135512 | 0.131 | … |
+| -0.460647 | -0.610939 | 0.085221 | -0.0560837 | 0.00254932 | 0.534791 | 0.251593 | … |
+| … | … | … | … | … | … | … | … |
+
+
+Here, column "0" represents the first component, column "1" represents the second component, and so on. These are the input variables we will feed into our classification model to predict which customers purchased Ed Sheeran's last album!
+
+___
+
+# Classification Model
+
+##### Training The Classifier
+
+To start with, we will simply apply a Random Forest Classifier to see if it is possible to predict based upon our set of 24 components.
+
+In the code below we instantiate the Random Forest using the default parameters, and then we fit this to our data.
+
+```python
+
+# instantiate our model object
+clf = RandomForestClassifier(random_state = 42)
+
+# fit our model using our training & test sets
+clf.fit(X_train, y_train)
+
+```
+
+##### Classification Performance
+
+In the code below we use the trained classifier to predict on the test set - and we run a simple analysis for the classification accuracy for the predictions vs. actuals.
+
+```python
+
+# predict on the test set
+y_pred_class = clf.predict(X_test)
+
+# assess the classification accuracy
+accuracy_score(y_test, y_pred_class)
+
+```
+
+The result of this is a **93%** classification accuracy, in other words, using a classifier trained on 24 principal components we were able to accurately predict which test set customers purchased Ed Sheeran's last album, with an accuracy of 93%.
+
+___
+
+# Application
+
+Based upon this proof-of-concept, we could go back to the client and recommend that they purchase some up to date listening data. We could apply PCA to this, create the components, and predict which customers are likely to buy Ed's *next* album.
+
+___
+
+# Growth & Next Steps
+
+We only tested one type of classifier here (Random Forest) - it would be worthwhile testing others. We also only used the default classifier hyperparameters - we would want to optimise these.
+
+Here, we selected 24 components based upon the fact this accounted for 75% of the variance of the initial feature set. We would instead look to search for the optimal number of components to use based upon classification accuracy.
diff --git a/_posts/2026-08-06-association-rule-learning.md b/_posts/2026-08-06-association-rule-learning.md
new file mode 100644
index 000000000..ea4d2158f
--- /dev/null
+++ b/_posts/2026-08-06-association-rule-learning.md
@@ -0,0 +1,406 @@
+---
+layout: post
+title: Understanding Alcohol Product Relationships Using Association Rule Learning
+image: "/posts/association-rules-title-img.png"
+tags: [Association Rule Learning, Python]
+---
+
+In this project we use Association Rule Learning to analyse the transactional relationships & dependencies between products in the alcohol section of a grocery store.
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results](#overview-results)
+ - [Growth/Next Steps](#overview-growth)
+- [01. Data Overview](#data-overview)
+- [02. Apriori Overview](#apriori-overview)
+- [03. Data Preparation](#apriori-data-prep)
+- [04. Applying The Apriori Algorithm](#apriori-fit)
+- [05. Interpreting The Results](#apriori-results)
+- [06. Growth & Next Steps](#growth-next-steps)
+
+___
+
+# Project Overview
+
+### Context
+
+Our client is looking to re-jig the alcohol section within their store. Customers are often complaining that they can't find the products they want, and are also wanting recommendations about which other products to try. On top of this, their marketing team would like to start running "bundled" promotions as this has worked well in other areas of the store - but need guidance with selecting which products to put together.
+
+They have provided us a sample of 3,500 alcohol transactions - our task is fairly open - to see if we can find solutions or insights that might help the business address the aforementioned problems!
+
+
+
+### Actions
+
+Based upon the tasks at hand - we apply Association Rule Learning, specifically *Apriori* to examine & analyse the strength of relationship between different products within the transactional data.
+
+We firstly installed the apyori package, which contains all of the required functionality for this task.
+
+We then needed to bring in the sample data, and get it into the right format for the Apriori algorithm to deal with.
+
+From there we apply the Apriori algorithm to provide us with several different relationship metrics, namely:
+
+* Support
+* Confidence
+* Expected Confidence
+* Lift
+
+These metrics examine product relationships in different ways, so we utilise each to put forward ideas that address each of the tasks at hand. You can read more about these metrics, and the Apriori algorithm in the relevant section below.
+
+
+
+
+### Results
+
+Interestingly, the strongest relationship existed between two products labelled as "gifts" - this is useful information for the category managers as they may want to ensure that gift products are available in one section of the aisle, rather than existing in their respective product types.
+
+We also saw some strong relationships between French wines, and other French wines - which again is extremely useful for category managers who are thinking about the best way to lay out the products - having sections by country rather than necessarily by type might make it easier for customers to find what they are after.
+
+Another interesting association is between products labelled "small". At this point, we don't know exactly what that means - but it is certainly something to take back to the client as they may be able to make more sense of it, and turn it into an actionable insight!
+
+We propose to also build a "search engine" for category managers where they can look-up products by keyword in the product association table.
+
+As an example - we search for any products that associate strongly with "New Zealand" products. There appeared to be *some* relationship between New Zealand wines and other New Zealand wines, but what was also interesting was that New Zealand wines seemed to be more associated with French & South American wines than they were with Australian Wines.
+
+New Zealand & Australia are often grouped together, but in terms of wine this wouldn't make sense - perhaps because of the difference climates the wines are very different and thus it wouldn't make sense to group wines by geographical proximity, but by preference instead. This is only a hypothesis for now - we will need to take this back to the client and get their category experts to help us interpret it!
+
+
+
+### Growth/Next Steps
+
+As this is first & foremost an exploratory project, we will take back the results to the client Category Managers & discuss the results, our views on how these insights can be actioned best, and any considerations that need to be taken into account when interpreting.
+
+From there we will recommend applying this same logic to all other categories, as well as potentially across the full-product range.
+
+We will also propose the build of the "Keyword Search Engine" which will help Category Managers extract and utilise the insights held within the data.
+
+
+
+
+___
+
+# Data Overview
+
+Our initial dataset contains 3,500 transactions, each of which shows the alcohol products that were present in that transaction.
+
+In the code below, we import Pandas, as well as the apriori algorithm from the apyori library, and we bring the raw data into Python.
+
+```python
+
+# import required Python packages
+import pandas as pd
+from apyori import apriori
+
+# import the sample data
+alcohol_transactions = pd.read_csv("data/sample_data_apriori.csv")
+
+```
+
+
+A sample of this data (the first 10 transactions) can be seen below:
+
+
+
+| **transaction_id** | **product1** | **product2** | **product3** | **product4** | **product5** | **…** |
+|---|---|---|---|---|---|---|
+| 1 | Premium Lager | Iberia | … | | | ... |
+| 2 | Sparkling | Premium Lager | Premium Cider | Own Label | Italy White | … |
+| 3 | Small Sizes White | Small Sizes Red | Sherry Spanish | No/Low Alc Cider | Cooking Wine | … |
+| 4 | White Uk | Sherry Spanish | Port | Italian White | Italian Red | … |
+| 5 | Premium Lager | Over-Ice Cider | French White South | French Rose | Cocktails/Liqueurs | … |
+| 6 | Kosher Red | … | | | | ... |
+| 7 | Own Label | Italy White | Australian Red | … | | ... |
+| 8 | Brandy/Cognac | … | | | | ... |
+| 9 | Small Sizes White | Bottled Ale | … | | | ... |
+| 10 | White Uk | Spirits Mixers | Sparkling | German | Australian Red | … |
+| … | … | … | … | … | … | … |
+
+
+To explain this data, *Transaction 1* (the first row) contained two products, Premium Lager, and Iberia. As there were only two products in this transaction, the remaining columns are blank.
+
+Transaction 2 (the second row) contained nine products (not all shown in the snippet). The first nine columns for this row are therefore populated, followed by blank values.
+
+For our sample data, the maximum number of unique products was 45, meaning the table of data had a total of 46 columns (45 for products + transaction_id).
+
+The *apyori* library that we are using does not want the data in this format, it instead wants it passed in as a *list of lists* so we will need to modify it. The code and logic for this can be found in the Data Preparation section below.
+
+___
+
+# Apriori Overview
+
+Association Rule Learning is an approach that discovers the strength of relationships between different data-points. It is commonly utilised to understand which products are frequently (or infrequently) purchased together.
+
+In a business sense this can provide some really interesting, and useful information that can help optimise:
+
+* Product Arrangement/Placement (making the customer journey more efficient)
+* Product Recommendations (customers who purchased product A also purchased product B)
+* Bundled Discounts (which products should/should not be put together)
+
+One powerful, intuitive, and commonly used algorithm for Association Rule Learning is **Apriori**.
+
+In Apriori there are four key metrics, namely:
+
+* Support
+* Confidence
+* Expected Confidence
+* Lift
+
+Each of these metrics help us understand items, and their relationship with other items in their own way.
+
+
+##### Support
+
+Support is extremely intuitive, it simply tells us the percentage of all transactions that contain *both* Item A and Item B. To calculate this we’d just count up the transactions that include both items, and divide this by the total number of transactions.
+
+You can think of Support as a baseline metric that helps us understand how common or popular this particular *pair* of items is.
+
+
+##### Confidence
+
+Confidence takes us a little bit further than Support, and looks more explicitly at the *relationship* between the two items.
+
+It asks "of all transactions that *included item A*, what proportion also included item B?"
+
+In other words, here we are counting up the number of transactions that contained *both items A and B* and then rather than dividing by *all transactions* like we did for Support, we instead divide this by the *total number of transactions that contained item A*.
+
+A high score for Confidence can mean a strong product relationship - but not always! When one of the items is very popular we can get an inflated score. To help us regulate this, we can look at two further metrics, Expected Confidence and Lift!
+
+
+##### Expected Confidence
+
+Expected Confidence is quite simple, it is the percentage of *all transactions* that *contained item B*.
+
+This is important as it provides indication of what the Confidence *would be* if there were no relationship between the items. We can use Expected Confidence, along with Confidence to calculate our final (and most powerful) metric - Lift!
+
+
+##### Lift
+
+Lift is the factor by which the Confidence, exceeds the Expected Confidence. In other words, Lift tells us how likely item B is purchased *when item A is purchased*, while *controlling* for how popular item B is.
+
+We calculate Lift by dividing Confidence by Expected Confidence.
+
+A Lift score *greater than 1* indicates that items A & B appear together *more often* than expected, and conversely a Lift score *less than 1* indicates that items A & B appear together *less often* than expected.
+
+
+##### In Practice
+
+While above we're just discussing two products (Item A & Item B) - in reality this score would be calculated between *all* pairs of products, and we could then sort these by Lift score (for example) and see exactly what the strongest or weakest relationships were - and this information would guide our decisions regarding product layout, recommendations for customers, or promotions.
+
+
+##### An Important Consideration
+
+Something to consider when assessing the results of Apriori is that, Item/Product relationships that have a *high Lift score* but also have a *low Support score* should be interpreted with caution!
+
+In other words, if we sorted all Item relationships by descending Lift score, the one that comes out on top might initially seem very impressive and it may appear that there is a very strong relationship between the two items. Always take into account the Support metric - it could be that this relationship is only taking place by chance due to the rarity of the item set.
+
+___
+
+# Data Preparation
+
+As mentioned in the Data Overview section above, the *apyori* library that we are using does not want the data in table format, it instead wants it passed in as a *list of lists* so we will need to modify it here.
+
+In the code below, we:
+
+* Remove the ID column as it is not required
+* Iterate over the DataFrame, appending each transaction to a list, and appending those to a master list
+* Print out the first 10 lists from the master list
+
+
+```python
+
+# drop ID column
+alcohol_transactions.drop("transaction_id", axis = 1, inplace = True)
+
+# modify data for apriori algorithm
+transactions_list = []
+for index, row in alcohol_transactions.iterrows():
+ transaction = list(row.dropna())
+ transactions_list.append(transaction)
+
+# print out first 10 lists from master list
+print(transactions_list[:10])
+
+[['Premium Lager', 'Iberia'],
+ ['Sparkling', 'Premium Lager', 'Premium Cider', 'Own Label', 'Italy White', 'Italian White', 'Italian Red', 'French Red', 'Bottled Ale'],
+ ['Small Sizes White', 'Small Sizes Red', 'Sherry Spanish', 'No/Low Alc Cider', 'Cooking Wine', 'Cocktails/Liqueurs', 'Bottled Ale'],
+ ['White Uk', 'Sherry Spanish', 'Port', 'Italian White', 'Italian Red'],
+ ['Premium Lager', 'Over-Ice Cider', 'French White South', 'French Rose', 'Cocktails/Liqueurs', 'Bottled Ale'],
+ ['Kosher Red'],
+ ['Own Label', 'Italy White', 'Australian Red'],
+ ['Brandy/Cognac'],
+ ['Small Sizes White', 'Bottled Ale'],
+ ['White Uk', 'Spirits Mixers', 'Sparkling', 'German', 'Australian Red', 'American Red']]
+
+```
+
+
+As you can see from the print statement, each transaction (row) from the initial DataFrame is now contained within a list, all making up the master list.
+
+___
+
+# Applying The Apriori Algorithm
+
+In the code below we apply the apriori algorithm from the apyori library.
+
+This algorithm allows us to specify the association rules that we want. We set:
+
+* A minimum *Support* of 0.003 to eliminate very rare product sets
+* A minimum *Confidence* of 0.2
+* A minimum *Lift* of 3 to ensure we're only focusing on product sets with strong relationships
+* A minimum & maximum length of 2 meaning we're only focusing on product *pairs* rather than larger sets
+
+```python
+
+# apply the apriori algorithm and specify required parameters
+apriori_rules = apriori(transactions_list,
+ min_support = 0.003,
+ min_confidence = 0.2,
+ min_lift = 3,
+ min_length = 2,
+ max_length = 2)
+
+# convert the output to a list
+apriori_rules = list(apriori_rules)
+
+# print out the first element
+apriori_rules[0]
+
+RelationRecord(items=frozenset({'America White', 'American Rose'}), support=0.020745724698626296, ordered_statistics=[OrderedStatistic(items_base=frozenset({'American Rose'}), items_add=frozenset({'America White'}), confidence=0.5323741007194245, lift=3.997849299507762)])
+
+```
+
+The output from the algorithm is in the form of a generator. We convert this to a list as this is easier to manipulate & analyse.
+
+Based upon the parameters we set when applying the algorithm, we get 132 product pairs. We print out the first element from the list to see what the output looks like, and while this contains all the key information we need - to make it easier to analyse (and more accessible & useable for stakeholders) - in the next code snippet, we extract the key elements and use list comprehension to re-work this data to exist as a Pandas DataFrame.
+
+```python
+
+# extract each piece of information
+product1 = [list(rule[2][0][0])[0] for rule in apriori_rules]
+product2 = [list(rule[2][0][1])[0] for rule in apriori_rules]
+support = [rule[1] for rule in apriori_rules]
+confidence = [rule[2][0][2] for rule in apriori_rules]
+lift = [rule[2][0][3] for rule in apriori_rules]
+
+# compile into a single dataframe
+apriori_rules_df = pd.DataFrame({"product1" : product1,
+ "product2" : product2,
+ "support" : support,
+ "confidence": confidence,
+ "lift" : lift})
+
+```
+
+A sample of this data (the first 5 product pairs - not in any order) can be seen below:
+
+
+
+| **product1** | **product2** | **support** | **confidence** | **lift** |
+|---|---|---|---|---|
+| American Rose | America White | 0.021 | 0.532 | 3.998 |
+| America White | American White | 0.054 | 0.408 | 3.597 |
+| Australian Rose | America White | 0.005 | 0.486 | 3.653 |
+| Low Alcohol A.C | America White | 0.003 | 0.462 | 3.466 |
+| American Rose | American Red | 0.016 | 0.403 | 3.575 |
+| … | … | … | … | … |
+
+
+In the DataFrame we have the two products in the pair, and then the three key metrics; Support, Confidence, and Lift.
+
+___
+
+# Interpreting The Results
+
+
+#### Associated Products
+
+Now we have our data in a useable format - let's look at the product pairs with the *strongest* relationships - we can do this by sorting our Lift column, in descending order.
+
+```python
+
+# sort pairs by descending Lift
+apriori_rules_df.sort_values(by = "lift", ascending = False, inplace = True)
+
+```
+
+
+In the table below, we can see the ten highest product relationships, based upon Lift
+
+
+
+| **product1** | **product2** | **support** | **confidence** | **lift** |
+|---|---|---|---|---|
+| Wine Gifts | Beer/Lager Gifts | 0.004 | 0.314 | 10.173 |
+| Beer/Lager Gifts | Spirits & Fortified | 0.013 | 0.427 | 9.897 |
+| Wine Gifts | Spirits & Fortified | 0.006 | 0.412 | 9.537 |
+| Red Wine Bxes & 25Cl | White Boxes | 0.015 | 0.474 | 9.344 |
+| French White Rhone | French Red | 0.003 | 0.480 | 8.691 |
+| Small Sizeswhite Oth | Small Sizes White | 0.005 | 0.559 | 8.340 |
+| Small Sizes Red | Small Sizes White | 0.025 | 0.486 | 7.258 |
+| French White Loire | French White South | 0.004 | 0.349 | 6.763 |
+| French White Rhone | French White 2 | 0.005 | 0.760 | 6.661 |
+| Small Sizeswhite Oth | Small Sizes Red | 0.003 | 0.324 | 6.306 |
+| Small Sizes Wht Othr | Small Sizes White | 0.003 | 0.414 | 6.176 |
+
+
+Interestingly, the strongest relationship exists between two products labelled as "gifts" - this is useful information for the category managers as they may want to ensure that gift products are available in one section of the aisle, rather than existing in their respective product types.
+
+We also see some strong relationships between French wines, and other French wines - which again is extremely useful for category managers who are thinking about the best way to lay out the products - having sections by country rather than necessarily by type might make it easier for customers to find what they are after.
+
+Another interesting association is between products labelled "small". At this point, we don't know exactly what that means - but it is certainly something to take back to the client as they may be able to make more sense of it, and turn it into an actionable insight!
+
+
+#### Search Tool For Category Managers
+
+With the data now stored as a DataFrame, we will also go back to the client with a proposal to build a simple "search" tool for Category Managers to use.
+
+An example of how this might work would be to test a hypothesis around New Zealand wines.
+
+The code below uses a string function to pull back all rows in the DataFrame where *product1* contains the words "New Zealand"
+
+```python
+
+# search based upon text
+apriori_rules_df[apriori_rules_df["product1"].str.contains("New Zealand")]
+
+```
+
+The results of this search, in order of descending Lift are as follows:
+
+
+
+| **product1** | **product2** | **support** | **confidence** | **lift** |
+|---|---|---|---|---|
+| New Zealand Red | Malt Whisky | 0.005326605 | 0.271428571 | 5.628986711 |
+| New Zealand Red | Iberia White | 0.007289038 | 0.371428571 | 4.616326531 |
+| New Zealand Red | New Zealand White | 0.012615643 | 0.642857143 | 4.613825812 |
+| New Zealand Red | French White South | 0.004485562 | 0.228571429 | 4.431055901 |
+| New Zealand Red | French White 2 | 0.009531819 | 0.485714286 | 4.256862057 |
+| New Zealand Red | French Red | 0.004205214 | 0.214285714 | 3.879985497 |
+| New Zealand Red | French Red South | 0.006447996 | 0.328571429 | 3.868033946 |
+| New Zealand Red | South America | 0.010933558 | 0.557142857 | 3.799863425 |
+| New Zealand Red | Other Red | 0.004485562 | 0.228571429 | 3.591692889 |
+| New Zealand Red | Iberia | 0.012054948 | 0.614285714 | 3.528433402 |
+| New Zealand Red | Champagne | 0.008690777 | 0.442857143 | 3.526052296 |
+| New Zealand White | South America White | 0.049341183 | 0.354124748 | 3.423205902 |
+| New Zealand Red | French Red 2 | 0.010092515 | 0.514285714 | 3.359811617 |
+| New Zealand Red | South America White | 0.006728343 | 0.342857143 | 3.314285714 |
+| New Zealand Red | Australia White | 0.007289038 | 0.371428571 | 3.215742025 |
+
+
+There appears to be *some* relationship between New Zealand wines and other New Zealand wines, but what is also interesting is that New Zealand wines seem to be more associated with French & South American wines than they are with Australian Wines.
+
+New Zealand & Australia are often grouped together, but in terms of wine this wouldn't make sense - perhaps because of the different climates the wines are very different and thus it wouldn't make sense to group wines by geographical proximity, but by preference instead. This is only a hypothesis for now - we will need to take this back to the client and get their category experts to help us interpret it!
+
+___
+
+# Growth & Next Steps
+
+As this was first & foremost an exploratory project, we will take back the results to the client Category Managers & discuss the results, our views on how these insights can be actioned best, and any considerations that need to be taken into account when interpreting.
+
+From there we will recommend applying this same logic to all other categories, as well as potentially across the full-product range.
+
+We will also propose the build of the "Keyword Search Engine" which will help Category Managers extract and utilise the insights held within the data.
diff --git a/_posts/2026-08-07-causal-impact.md b/_posts/2026-08-07-causal-impact.md
new file mode 100644
index 000000000..ee764276f
--- /dev/null
+++ b/_posts/2026-08-07-causal-impact.md
@@ -0,0 +1,341 @@
+---
+layout: post
+title: Quantifying Sales Uplift With Causal Impact Analysis
+image: "/posts/causal-impact-title-img.png"
+tags: [Causal Impact Analysis, Python]
+---
+
+In this project we use Causal Impact Analysis to analyse & understand the sales uplift of customers that joined the new "Delivery Club" campaign.
+
+# Table of contents
+
+- [00. Project Overview](#overview-main)
+ - [Context](#overview-context)
+ - [Actions](#overview-actions)
+ - [Results](#overview-results)
+ - [Growth/Next Steps](#overview-growth)
+- [01. Causal Impact Analysis Overview](#causal-impact-overview)
+- [02. Data Overview & Preparation](#causal-impact-data-prep)
+- [03. Applying Causal Impact Analysis](#causal-impact-fit)
+- [04. Analysing The Results](#causal-impact-results)
+- [05. Growth & Next Steps](#growth-next-steps)
+
+___
+
+# Project Overview
+
+### Context
+
+Earlier in the year, our client, a grocery retailer, ran a campaign to promote their new "Delivery Club" - an initiative that costs a customer $100 per year for membership, but offers free grocery deliveries rather than the normal cost of $10 per delivery.
+
+They want to understand if customers who did join the club have increased their spend in the three months following. The hypothesis is that, if customers are not paying for deliveries, they will be tempted to shop more frequently, and hopefully purchase more each time.
+
+The aim of this work is to understand and quantify the uplift in sales for customers that joined the club, over and above what they *would* have spent had the club not come into existence!
+
+
+
+### Actions
+
+We applied Causal Impact Analysis (see full details below) using the *pycausalimpact* library.
+
+In the client database, we have a *campaign_data* table which shows us which customers received each type of "Delivery Club" mailer, which customers were in the control group, and which customers joined the club as a result.
+
+Since Delivery Club membership was open to *all customers* - the control group we have in the *campaign_data* table would help us measure the impact of *contacting* customers but here, we are actually looking to measure the overall impact on sales from the Delivery Club itself. Because of this, we instead used customers who did not sign up as the control. The hypothesis was that customers who did not sign up should continue their normal shopping habits after the club went live, and this will help us create the counter-factual for the customers that did sign-up.
+
+Sales data was from the *transactions* table and was aggregated from a customer/transaction/product area level to customer/date level as per the requirements of the algorithm.
+
+We used a three-month pre-period for the algorithm to model, and a three-month post-period for the counterfactual.
+
+
+
+
+### Results
+
+We saw a 41.1% uplift in sales for those customers that joined the Delivery Club, over and above what we believe they would have spent, had the club not been in existence. This was across the three month post-period, and the uplift was deemed to be significantly significant (@ 95%).
+
+
+
+### Growth/Next Steps
+
+It would be interesting to look at this pool of customers (both those who did and did not join the Delivery club) and investigate if there were any differences in sales in these time periods *last year* - this would help us understand if any of the uplift we are seeing here is actually the result of seasonality.
+
+It would be interesting to track this uplift over time and see if:
+
+* It continues to grow
+* It flattens or returns to normal
+* We see any form of uplift pull-forward
+
+It would also be interesting to analyse what it is that is making up this uplift. Are customers increasing their spend across the same categories - or are they buying into new categories
+
+
+
+
+___
+
+# Causal Impact Analysis Overview
+
+
+#### Context
+
+One of the most common tasks we undertake in Data Science & Data Analysis is *understanding and quantifying a change in a key business metric after some event has taken place*.
+
+Depending on the industry - this could be the uplift in sales after a promotion or a product release, the additional clicks, conversions, or signups generated by an online ad campaign, the change in share price after a market event, or even the change in the value of the US dollar after the president opens his mouth.
+
+Whatever the scenario, the task is *essentially* the same - we want to understand how big this change was.
+
+But to understand this *robustly & reliably* we really need to understand what *would have happened* had the event not taken place.
+
+In most cases the trends *preceding the event in question* isn’t tame, it is filled with lumps and bumps and ups and downs. When some key event does take place, understanding what *would have happened had the event not taken place* so we can estimate the true impact can be difficult!
+
+In many cases, the event that we’re analysing is part of a randomised & controlled experiment, and this means understanding the difference between the group that was affected by the event can be compared to a control group, that was purposely held back from the effect of the event.
+
+But there are a lot of cases where we just can’t run a randomised experiment, either because it’s expensive, or potentially it’s just impossible. As an example, in the case of measuring the change in a share price after an event, we don’t really have a direct control group to lean on for comparison purposes.
+
+An approach that works really well in both scenarios, is Causal Impact Analysis.
+
+
+#### How It Works
+
+Causal Impact is a time-series technique, originally developed by Google.
+
+It estimates what *would have happened* (known as a "counterfactual") by applying a model to *comparable data* in a pre-period and projecting this model onto that data in a post-period. The difference between the actual data and the counterfactual in the post-period, is the estimated impact of the event.
+
+The *comparable data* that we pass in can be a control group, another set of related data, or even multiple sets of related data - but for this approach to work robustly & reliably, this additional data must adhere to several rules:
+
+It must not be affected by the event that we’re measuring, but it must be predictive of our output, or have some relationship with our initial time-series data.
+
+So, in the case of randomised experiment, we could use the control group as our additional set of data.
+
+In the case where we don't have a control group, we need to find other sets of data that meet the aforementioned rules. These must not be affected by the event, but they should have some relationship or correlation with the time-series data we’re measuring. If we were measuring stock prices, perhaps we could use other stocks that are in a similar industry to us. If we were measuring the sales of a certain section of the grocery store, say health and beauty products, perhaps our second time-series could be the sales of another non-food category in the store.
+
+Either way, this additional data provides the algorithm insights into the trends of the data over time.
+
+The algorithm uses these insights to model the relationship between the two (or more) time-series in the pre-period. In other words, it finds a set of rules that best predict the time-series of interest, based on the movements and fluctuations of the other time-series that we provided it.
+
+Once the algorithm has modelled this relationship, it then looks to apply the learnings from this model in the post-period, the result of which is an estimation for the counterfactual, or what the model *believes would have happened* to our time series if our event never took place!
+
+Once we have this counterfactual, we can proceed to calculate the estimation for the causal effect, or in other words, the effect caused by our event!
+
+
+#### Application
+
+Here we will utilise a Python package called **pycausalimpact** to apply this algorithm to our data. This will model the relationships, and provide very useful plots and summarises to help us understand the results.
+
+___
+
+# Data Overview & Preparation
+
+In the client database, we have a *campaign_data* table which shows us which customers received each type of "Delivery Club" mailer, which customers were in the control group, and which customers joined the club as a result.
+
+Since Delivery Club membership was open to *all customers* - the control group we have in the *campaign_data* table would help us measure the impact of *contacting* customers but here, we are actually look to measure the overall impact on sales from the Delivery Club itself. Because of this, we will instead just use customers who did not sign up as the control. The customers who did not sign up should continue their normal shopping habits after the club went live, and this will help us create the counter-factual for the customers that did sign-up.
+
+In the code below, we:
+
+* Load in the Python libraries we require
+* Import the required data from the *transactions* and *campaign_data* tables (3 months prior, 3 months post campaign)
+* Aggregate the transactions table from customer/transaction/product area level to customer/date level
+* Merge on the signup flag from the *campaign_data* table
+* Pivot & aggregate to give us aggregated daily sales by signed-up/did not sign-up groups
+* Manoeuvre the data specifically for the pycausalimpact algorithm
+* Give our groups some meaningful names, to help with interpretation
+
+
+```python
+
+# install the required python libraries
+from causalimpact import CausalImpact
+import pandas as pd
+
+# import data tables
+transactions = ...
+campaign_data = ...
+
+# aggregate transaction data to customer, date level
+customer_daily_sales = transactions.groupby(["customer_id", "transaction_date"])["sales_cost"].sum().reset_index()
+
+# merge on the signup flag
+customer_daily_sales = pd.merge(customer_daily_sales, campaign_data, how = "inner", on = "customer_id")
+
+# pivot the data to aggregate daily sales by signup group
+causal_impact_df = customer_daily_sales.pivot_table(index = "transaction_date",
+ columns = "signup_flag",
+ values = "sales_cost",
+ aggfunc = "mean")
+
+# provide a frequency for our DateTimeIndex (avoids a warning message)
+causal_impact_df.index.freq = "D"
+
+# ensure the impacted group is in the first column (the library expects this)
+causal_impact_df = causal_impact_df[[1,0]]
+
+# rename columns to something clear & meaningful
+causal_impact_df.columns = ["member", "non_member"]
+
+```
+
+A sample of this data (the first 5 days of data) can be seen below:
+
+
+
+| **transaction_date** | **member** | **non_member** |
+|---|---|---|
+| 01/04/2020 | 194.49 | 74.46 |
+| 02/04/2020 | 185.16 | 75.56 |
+| 03/04/2020 | 118.12 | 74.39 |
+| 04/04/2020 | 198.53 | 63.00 |
+| 05/04/2020 | 145.46 | 72.44 |
+
+
+In the DataFrame we have the transaction data, and then a column showing the average daily sales for those who signed up (member) and those who did not (non_member). This is the required format for applying the algorithm.
+
+___
+
+# Applying The Causal Impact Algorithm
+
+In the code below, we specify the start and end dates of the "pre-period" and the start and end dates of the "post-period". We then apply the algorithm by passing in the DataFrame and the specified pre and post period time windows.
+
+The algorithm will model the relationship between members & non-members in the pre-period - and it will use this to create the counterfactual, in other words what it believes would happen to the average daily spend for members in the post-period if no event was to have taken place!
+
+The difference between this counterfactual and the actual data in the post-period will be our "causal impact"
+
+```python
+
+# specify the pre & post periods
+pre_period = ["2020-04-01","2020-06-30"]
+post_period = ["2020-07-01","2020-09-30"]
+
+# apply the algorithm
+ci = CausalImpact(causal_impact_df, pre_period, post_period)
+
+```
+
+We can use the created object (called ci above) to examine & plot the results.
+
+___
+
+# Analysing The Results
+
+
+#### Plotting The Results
+
+The *pycausalimpact* library makes plotting the results extremely easy - all done with the single line of code below:
+
+```python
+
+# plot the results
+ci.plot()
+
+```
+
+The resulting plot(s) can be seen below.
+
+
+
+
+
+To explain what we have in the above image...
+
+The vertical dotted line down the middle of each plot is the date that the Delivery Club membership started. Everything to the left of this dotted line is the pre-period, and everything to the right of the dotted line is the post-period.
+
+
+**Chart 1: Actual vs. Counterfactual**
+
+The top chart shows the actual data for the impacted group as a black line, in other words the *actual* average daily sales for customers who did go on to sign up to the Delivery Club. You can also see the counterfactual, which is shown with the blue dotted line. The purple area around the blue dotted line represent the confidence intervals around the counterfactual - in other words, the range in which the algorithm believes the prediction should fall in. A wider confidence interval suggests that the model is less sure about it's counterfactual prediction - and this is all taken into account when we look to quantify the actual uplift.
+
+Just eyeing this first chart, it does indeed look like there is some increase in daily average spend for customers who joined the club, over-and-above what the model suggests they would have done, if the club was never in existence. We will look at the actual numbers for this very soon.
+
+
+**Chart 2: Pointwise Effects**
+
+This second chart shows us, for each day (or data point in general) in our time-series, the *raw differences* between the actual values and the values for the counterfactual. It is plotting the *differences* from Chart 1. As an example, if on Day 1 the actual and the counterfactual were the same, this chart would show a value of 0. If the actual is higher than the counterfactual then we would see a positive value on this chart, and vice versa. It is essentially showing how far above or below the counterfactual, the actual values are.
+
+What is interesting here is that for the pre-period we see a difference surrounding zero, but in the post period we see mostly positive values mirroring what we saw in Chart 1 where the actual average spend was greater than the counterfactual.
+
+
+**Chart 3: Cumulative Effects**
+
+The bottom chart shows the cumulative uplift over time. In other words this chart is effectively adding up the Pointwise contributions from the second chart over time. This is very useful as it helps the viewer get a feel for what the total uplift or difference is at any point in time.
+
+As we would expect based on the other two charts, there does appear to be a cumulative uplift over time.
+
+
+#### Interpreting The Numbers
+
+The *pycausalimpact* library also makes interpreting the numbers very easy. We can get a clean results summary with the following line of code:
+
+```python
+
+# results summary
+print(ci.summary())
+
+Posterior Inference {Causal Impact}
+ Average Cumulative
+Actual 171.33 15762.67
+Prediction (s.d.) 121.42 (4.33) 11170.19 (398.51)
+95% CI [112.79, 129.77] [10376.65, 11938.77]
+
+Absolute effect (s.d.) 49.92 (4.33) 4592.48 (398.51)
+95% CI [41.56, 58.54] [3823.9, 5386.02]
+
+Relative effect (s.d.) 41.11% (3.57%) 41.11% (3.57%)
+95% CI [34.23%, 48.22%] [34.23%, 48.22%]
+
+Posterior tail-area probability p: 0.0
+Posterior prob. of a causal effect: 100.0%
+
+```
+
+At the top of the results summary (above) we see that in the post-period the average actual daily sales per customer over the post-period was $171, higher than that of the counterfactual, which was $121. This counterfactual prediction had 95% confidence intervals of $113 and $130.
+
+Below that we can see the *absolute effect* which is the difference between actual and counterfactual (so the difference between $171 and $121) - and this figure is essentially showing us the average daily *uplift* in sales over the post-period. We also get the confidence intervals surrounding that effect, and since these do not pass through zero, we can confidently say that there *was* an uplift driven by the Delivery Club.
+
+Below that, we get these same numbers - as percentages.
+
+In the columns on the right of the summary, we see the *cumulative* values for these across the entire post-period, rather than the average per day.
+
+What is amazing about the *pycausalimpact* library is that, with an extra parameter, we can actually get all of this information provided as a written output.
+
+If we put:
+
+```python
+
+# results summary - report
+print(ci.summary(output = "report"))
+
+Analysis report {CausalImpact}
+
+During the post-intervention period, the response variable had an average value of approx. 171.33. By contrast, in the absence of an intervention, we would have expected an average response of 121.42.
+
+The 95% interval of this counterfactual prediction is [112.79, 129.77].
+
+Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 49.92 with a 95% interval of [41.56, 58.54]. For a discussion of the significance of this effect, see below.
+
+Summing up the individual data points during the post-intervention period (which can only sometimes be meaningfully interpreted), the response variable had an overall value of 15762.67. By contrast, had the intervention not taken place, we would have expected a sum of 11170.19. The 95% interval of this prediction is [10376.65, 11938.77].
+
+The above results are given in terms of absolute numbers. In relative terms, the response variable showed an increase of +41.11%. The 95% interval of this percentage is [34.23%, 48.22%].
+
+This means that the positive effect observed during the intervention period is statistically significant and unlikely to be due to random fluctuations. It should be noted, however, that the question of whether this increase also bears substantive significance can only be answered by comparing the absolute effect (49.92) to the original goal
+of the underlying intervention.
+
+The probability of obtaining this effect by chance is very small (Bayesian one-sided tail-area probability p = 0.0). This means the causal effect can be considered statistically
+significant.
+
+```
+
+So, this is the same information as we saw above, but put into a written report which can go straight to the client.
+
+The high level story of this that, yes, we did see an uplift in sales for those customers that joined the Delivery Club, over and above what we believe they would have spent, had the club not been in existence. This uplift was deemed to be significantly significant (@ 95%)
+
+___
+
+# Growth & Next Steps
+
+It would be interesting to look at this pool of customers (both those who did and did not join the Delivery club) and investigate if there were any differences in sales in these time periods *last year* - this would help us understand if any of the uplift we are seeing here is actually the result of seasonality.
+
+It would be interesting to track this uplift over time and see if:
+
+* It continues to grow
+* It flattens or returns to normal
+* We see any form of uplift pull-forward
+
+It would also be interesting to analyse what it is that is making up this uplift. Are customers increasing their spend across the same categories - or are they buying into new categories
diff --git a/img/CAF9BD2A-23C8-4235-BE2F-89315A9439E7_1_105_c.jpeg b/img/CAF9BD2A-23C8-4235-BE2F-89315A9439E7_1_105_c.jpeg
new file mode 100644
index 000000000..2f8ae9c07
Binary files /dev/null and b/img/CAF9BD2A-23C8-4235-BE2F-89315A9439E7_1_105_c.jpeg differ
diff --git a/img/posts/ab-testing-title-img.png b/img/posts/ab-testing-title-img.png
new file mode 100644
index 000000000..840c09964
Binary files /dev/null and b/img/posts/ab-testing-title-img.png differ
diff --git a/img/posts/association-rules-title-img.png b/img/posts/association-rules-title-img.png
new file mode 100644
index 000000000..d753dbef2
Binary files /dev/null and b/img/posts/association-rules-title-img.png differ
diff --git a/img/posts/causal-impact-results-plot.png b/img/posts/causal-impact-results-plot.png
new file mode 100644
index 000000000..c93a72b5a
Binary files /dev/null and b/img/posts/causal-impact-results-plot.png differ
diff --git a/img/posts/causal-impact-title-img.png b/img/posts/causal-impact-title-img.png
new file mode 100644
index 000000000..045186e22
Binary files /dev/null and b/img/posts/causal-impact-title-img.png differ
diff --git a/img/posts/classification-title-img.png b/img/posts/classification-title-img.png
new file mode 100644
index 000000000..222396c2a
Binary files /dev/null and b/img/posts/classification-title-img.png differ
diff --git a/img/posts/clf-tree-confusion-matrix.png b/img/posts/clf-tree-confusion-matrix.png
new file mode 100644
index 000000000..54549c731
Binary files /dev/null and b/img/posts/clf-tree-confusion-matrix.png differ
diff --git a/img/posts/clf-tree-max-depth-plot.png b/img/posts/clf-tree-max-depth-plot.png
new file mode 100644
index 000000000..534f51797
Binary files /dev/null and b/img/posts/clf-tree-max-depth-plot.png differ
diff --git a/img/posts/clf-tree-nodes-plot.png b/img/posts/clf-tree-nodes-plot.png
new file mode 100644
index 000000000..93e4a15fc
Binary files /dev/null and b/img/posts/clf-tree-nodes-plot.png differ
diff --git a/img/posts/clustering-title-img.png b/img/posts/clustering-title-img.png
new file mode 100644
index 000000000..76ac65f49
Binary files /dev/null and b/img/posts/clustering-title-img.png differ
diff --git a/img/posts/kmeans-optimal-k-value-plot.png b/img/posts/kmeans-optimal-k-value-plot.png
new file mode 100644
index 000000000..106997e05
Binary files /dev/null and b/img/posts/kmeans-optimal-k-value-plot.png differ
diff --git a/img/posts/knn-confusion-matrix.png b/img/posts/knn-confusion-matrix.png
new file mode 100644
index 000000000..bcef47ec7
Binary files /dev/null and b/img/posts/knn-confusion-matrix.png differ
diff --git a/img/posts/knn-feature-selection-plot.png b/img/posts/knn-feature-selection-plot.png
new file mode 100644
index 000000000..5dab7d122
Binary files /dev/null and b/img/posts/knn-feature-selection-plot.png differ
diff --git a/img/posts/knn-optimal-k-value-plot.png b/img/posts/knn-optimal-k-value-plot.png
new file mode 100644
index 000000000..6a2ab4e5f
Binary files /dev/null and b/img/posts/knn-optimal-k-value-plot.png differ
diff --git a/img/posts/lin-reg-feature-selection-plot.png b/img/posts/lin-reg-feature-selection-plot.png
new file mode 100644
index 000000000..3bced3729
Binary files /dev/null and b/img/posts/lin-reg-feature-selection-plot.png differ
diff --git a/img/posts/log-reg-confusion-matrix.png b/img/posts/log-reg-confusion-matrix.png
new file mode 100644
index 000000000..0e6b3ef6a
Binary files /dev/null and b/img/posts/log-reg-confusion-matrix.png differ
diff --git a/img/posts/log-reg-feature-selection-plot.png b/img/posts/log-reg-feature-selection-plot.png
new file mode 100644
index 000000000..43dacc7f7
Binary files /dev/null and b/img/posts/log-reg-feature-selection-plot.png differ
diff --git a/img/posts/log-reg-optimal-threshold-plot.png b/img/posts/log-reg-optimal-threshold-plot.png
new file mode 100644
index 000000000..dd6c43469
Binary files /dev/null and b/img/posts/log-reg-optimal-threshold-plot.png differ
diff --git a/img/posts/pca-title-img.png b/img/posts/pca-title-img.png
new file mode 100644
index 000000000..4f2273007
Binary files /dev/null and b/img/posts/pca-title-img.png differ
diff --git a/img/posts/pca-variance-plots.png b/img/posts/pca-variance-plots.png
new file mode 100644
index 000000000..e62313e60
Binary files /dev/null and b/img/posts/pca-variance-plots.png differ
diff --git a/img/posts/regression-title-img.png b/img/posts/regression-title-img.png
new file mode 100644
index 000000000..ddecd7ddc
Binary files /dev/null and b/img/posts/regression-title-img.png differ
diff --git a/img/posts/regression-tree-max-depth-plot.png b/img/posts/regression-tree-max-depth-plot.png
new file mode 100644
index 000000000..5cbcc6ac5
Binary files /dev/null and b/img/posts/regression-tree-max-depth-plot.png differ
diff --git a/img/posts/regression-tree-nodes-plot.png b/img/posts/regression-tree-nodes-plot.png
new file mode 100644
index 000000000..6b90d580e
Binary files /dev/null and b/img/posts/regression-tree-nodes-plot.png differ
diff --git a/img/posts/rf-classification-feature-importance.png b/img/posts/rf-classification-feature-importance.png
new file mode 100644
index 000000000..493d41c87
Binary files /dev/null and b/img/posts/rf-classification-feature-importance.png differ
diff --git a/img/posts/rf-classification-permutation-importance.png b/img/posts/rf-classification-permutation-importance.png
new file mode 100644
index 000000000..bce09fbea
Binary files /dev/null and b/img/posts/rf-classification-permutation-importance.png differ
diff --git a/img/posts/rf-confusion-matrix.png b/img/posts/rf-confusion-matrix.png
new file mode 100644
index 000000000..2ad580ac1
Binary files /dev/null and b/img/posts/rf-confusion-matrix.png differ
diff --git a/img/posts/rf-regression-feature-importance.png b/img/posts/rf-regression-feature-importance.png
new file mode 100644
index 000000000..83b6a940f
Binary files /dev/null and b/img/posts/rf-regression-feature-importance.png differ
diff --git a/img/posts/rf-regression-permutation-importance.png b/img/posts/rf-regression-permutation-importance.png
new file mode 100644
index 000000000..bdfd2fca2
Binary files /dev/null and b/img/posts/rf-regression-permutation-importance.png differ