Cluster Analysis - Practical

1 Generate the Dataset

Objective: To access and prepare the data.

1.1 Tasks

Download the dataset here and create a dataframe called ‘data’.

https://www.dropbox.com/scl/fi/jw7fo5tdz4jauaoj5izmq/cluster_analysis_prac_data.csv?rlkey=f5nw3aexuacjgqe4oi5nyqyut&dl=1

Now, rename the feature ‘X’ to ‘ID’.
Then, modify data$ID to a factor and check conversion.

1.2 Solutions

Code

rm(list=ls())

# Load the dataset

data <- read.csv("https://www.dropbox.com/scl/fi/jw7fo5tdz4jauaoj5izmq/cluster_analysis_prac_data.csv?rlkey=f5nw3aexuacjgqe4oi5nyqyut&dl=1")

data$ID <- data$X
data$X <- NULL

# Then, modify data$ID to a factor and check conversion

data$ID <- as.factor(data$ID)
str(data)

2 Data Preparation

Objective: To prepare data for clustering.

2.1 Tasks

Create a subset of the data that contains only the variables you wish to scale.

What would you include in this subset, and why?

Scale the dataset to ensure all features are on the same scale.

Why? Clustering methods are sensitive to differences in scale.

Merge the dataframes together. Make sure it’s clear which features are scaled and which are the originals.
Now, create a further dataframe that includes only the ID and the scaled features.

2.2 Solutions

Code

athlete_data <- subset(data, select = -ID)

# Scale the data
scaled_data <- scale(athlete_data)
scaled_data <- as.data.frame(scaled_data)

rm(athlete_data)

# View scaled data
head(scaled_data)

# Rename the columns of scaled_data to include the "scale_" prefix
names(scaled_data) <- paste0("scale_", names(scaled_data))

# Merge the scaled data with the original data
merged_data <- cbind(data, scaled_data)

final_data <- subset(merged_data, select = c(ID, scale_Speed, scale_Height, scale_Agility, scale_Weight))

# View the resulting dataset
head(final_data)

rm(data, merged_data, scaled_data)

3 Partitioning Clustering (k-means)

Objective: To perform k-means clustering to partition the athletes into 3 clusters based on their features.

3.1 Tasks

Run the k-means algorithm.
Then, visualise which cluster each athlete belongs to.
Explore other ways of visualising or reviewing clusters. For example, use a boxplot to review the means of each cluster for the four features, or a scatterplot to interpret the clusters.

Note: this is why scaling first is so important!

3.2 Solutions

To run the kmeans clustering:

Code

# Apply k-means clustering
set.seed(123)  # For reproducibility
kmeans_result <- kmeans(final_data, centers = 3, nstart = 25)

# Add cluster membership to merged_data
final_data$KCluster <- factor(kmeans_result$cluster)

# View the first few rows of the updated dataset
head(final_data)

To visualise and explore the results:

Code

library(ggplot2)

# Strip plot for cluster assignments
ggplot(final_data, aes(x = KCluster, y = as.numeric(row.names(final_data)), color = KCluster)) +
  geom_jitter(width = 0.1, size = 2) +
  labs(title = "Cluster Assignments for Participants", x = "Cluster", y = "Participant") +
  theme_minimal()

Code

# Aggregate feature means by cluster
cluster_summary <- aggregate(. ~ KCluster, data = final_data, mean)

# View the cluster summary
print(cluster_summary)

# Boxplot for one feature (e.g., Height) by cluster
ggplot(final_data, aes(x = KCluster, y = scale_Height, fill = KCluster)) +
  geom_boxplot() +
  labs(title = "Height Distribution by Cluster", x = "Cluster", y = "Height") +
  theme_minimal()

Code

# Scatter plot of two variables
ggplot(final_data, aes(x = scale_Height, y = scale_Weight, color = KCluster)) +
  geom_point(size = 3) +
  labs(title = "Participants by Cluster", x = "Scaled Height", y = "Scaled Weight") +
  theme_minimal()

4 Formal Partitioning Model Evaluation

Objective: To evaluate and explore the model further.

4.1 Tasks

Review cluster centers by examining the means of each cluster for each feature.
Evaluate clustering quality using the silhouette score.

4.2 Solutions

Code

# Load required package

library(cluster)

# Cluster centers
kmeans_result$centers

# Evaluate clustering quality using the silhouette score.



# Calculate silhouette scores
sil <- silhouette(kmeans_result$cluster, dist(final_data))

# Plot silhouette scores
plot(sil, main = "Silhouette Plot (k-means)")

Finally, evaluate what your cluster analysis has told you about your data. Is this useful? Why?

5 Hierarchical Clustering

Objective: To explore hierarchical relationships between athletes.

5.1 Tasks

Perform hierarchical clustering.

Subset the data frame to contain only those variables that make sense to include in the cluster analysis.
Compute a distance matrix and apply hierarchical clustering:
Plot an initial dendrogram
Cut the dendrogram into 3 clusters and assign cluster labels.
Then view how many observations have been assigned to each cluster.
Finally, allocate cluster membership to each observation, based on the hierarchical cluster analysis.

5.2 Solutions

Code

subset_data <- subset(final_data, select = c(scale_Height, scale_Speed, scale_Agility, scale_Weight))

# Compute distance matrix

dist_matrix <- dist(subset_data)

# Perform hierarchical clustering

hclust_result <- hclust(dist_matrix, method = "ward.D2")

# Plot an initial dendrogram

plot(hclust_result, main = "Dendrogram of Athletes", xlab = "", sub = "")

Code

# Cut the dendrogram into 3 clusters and assign cluster labels. 

hclust_clusters <- cutree(hclust_result, k = 3)

# View cluster assignments

table(hclust_clusters)

# Add cluster membership to merged_data

# Assign each observation to a cluster
HCluster <- cutree(hclust_result, k = 3)

# Add HCluster to the dataset
final_data$HCluster <- factor(HCluster)

# View the first few rows of the updated dataset
head(final_data)

6 Investigate Cluster Membership

Objective: To look at how the two approaches have modelled the clusters within the data.

6.1 Tasks

Compare k-means and hierarchical clustering memberships. You could use a heatmap or other visualisaton to do this.
Visualise the clusters. Explore how the two approaches have reflected Speed and Weight in their clustering.

6.2 Solutions

Code

library(ggplot2)

# Create a contingency table
cluster_table <- table(final_data$KCluster, final_data$HCluster)

# Visualise table as a heatmap

# Convert the table to a data frame for ggplot2
cluster_df <- as.data.frame(as.table(cluster_table))

# Plot heatmap
ggplot(cluster_df, aes(x = Var2, y = Var1, fill = Freq)) +
  geom_tile(color = "white") +
  geom_text(aes(label = Freq), color = "black", size = 5) +
  scale_fill_gradient(low = "lightblue", high = "blue") +
  labs(
    title = "Heatmap of KMeans vs Hierarchical Clusters",
    x = "HCluster",
    y = "KCluster",
    fill = "Count"
  ) +
  theme_minimal()

Code

# Visualise clusters in 2D using PCA for dimensionality reduction 

library(lattice)

# Lattice plot for KMeans Clusters
xyplot(
  scale_Weight ~ scale_Speed | factor(KCluster),
  data = final_data,
  groups = KCluster,
  auto.key = TRUE,
  main = "KMeans Clusters: Speed vs Weight",
  xlab = "Speed (km/h)",
  ylab = "Weight (kg)",
  pch = 16,
  col = c("blue", "red", "green", "purple")
)

Code

# Lattice plot for KMeans Clusters
xyplot(
  scale_Weight ~ scale_Speed | factor(HCluster),
  data = final_data,
  groups = HCluster,
  auto.key = TRUE,
  main = "Hierarchical Clusters: Speed vs Weight",
  xlab = "Speed (km/h)",
  ylab = "Weight (kg)",
  pch = 16,
  col = c("blue", "red", "green", "purple")
)