Supervised Learning Techniques
Types of learning
Machine learning in sport
Types
Ridge Regression - Example
Lasso Regression
Logistic Regression
ROC Curve displays model’s true positive rate vs false positive rate.
The Area Under Curve (AUC) measures overall predictive accuracy.
Higher AUC indicates better performance in distinguishing outcomes.
Poisson Regression
Plots actual versus predicted count values (e.g., goals scored).
Shows the quality of model predictions in real scenarios.
Ideally, points lie along the diagonal, indicating accurate predictions.
Gamma Regression - Example
Code
set.seed(123)
df <- data.frame(x = rnorm(100))
# Generate gamma-distributed response, depending on x
df$y <- rgamma(100, shape = 2, rate = 1/(exp(0.5 + 0.2 * df$x)))
# Fit a Gamma GLM
gamma_model <- glm(y ~ x, family = Gamma(link = "log"), data = df)
df$pred <- predict(gamma_model, type = "response")
# Plot actual vs. predicted
plot(df$x, df$y, pch = 19, col = "blue",
xlab = "x", ylab = "y", main = "Gamma Regression: Actual vs Predicted")
points(df$x, df$pred, col = "red", pch = 19)
legend("topleft", legend = c("Actual", "Predicted"),
pch = 19, col = c("blue", "red"))
SVM plot
Polynomial Kernel
Radial Basis Function Kernel
Simple Decision Tree - Example
Feature Importance in Random Forests
Out-of-bag (OOB) error
Decision boundaries in random forests
Code
library(ggplot2)
library(randomForest)
# Generate a 2-feature dataset
set.seed(123)
df <- data.frame(
Feature1 = rnorm(200),
Feature2 = rnorm(200),
Class = factor(sample(c("A", "B"), 200, replace=TRUE))
)
# Train a Random Forest with only 2 features
rf2 <- randomForest(Class ~ Feature1 + Feature2, data=df, ntree=100, mtry=1)
# Create a grid of points to predict across feature space
grid <- expand.grid(
Feature1 = seq(min(df$Feature1), max(df$Feature1), length=100),
Feature2 = seq(min(df$Feature2), max(df$Feature2), length=100)
)
# Predict class labels for the grid
grid$Prediction <- predict(rf2, newdata=grid)
# Plot decision boundary and original points
ggplot(df, aes(x=Feature1, y=Feature2)) +
geom_tile(data=grid, aes(fill=Prediction), alpha=0.3) +
geom_point(aes(color=Class), size=2) +
labs(title="Random Forest Decision Boundary") +
theme_minimal()
Partial Dependence Plot
XGBoost
Code
library(xgboost)
set.seed(123)
data(agaricus.train, package='xgboost')
bst <- xgboost(
data = agaricus.train$data,
label = agaricus.train$label,
nrounds = 20,
objective = "binary:logistic",
verbose = 0
)
importance_matrix <- xgb.importance(model = bst)
xgb.plot.importance(importance_matrix, main = "XGBoost Feature Importance")
XGBoost Example
Demonstrates how a trained XGBoost model’s predictions change as one feature varies (partial dependence).
What it shows:
How model’s predicted probability varies when
feature1changes, keeping other features average.Highlights which values of feature1 increase (or decrease) the chance of a positive class.
ROC Curves and AUC
Code
Area under the curve: 0.498
SVM ROC Curve
Code
library(ROCR)
# Obtain predicted probabilities for the "positive" class (assuming 2-class problem)
svm_probs <- attr(predict(svm_model, test_data, probability = TRUE), "probabilities")
# If positive class is second level in factor, e.g. 'Win'
predicted_probs <- svm_probs[,2]
# Create prediction
pred_obj <- prediction(predicted_probs, test_data$result)
# Performance object for TPR (true positive rate) vs FPR (false positive rate)
perf <- performance(pred_obj, "tpr", "fpr")
# Plot ROC
plot(perf, main = "SVM ROC Curve")Code
AUC: 0.8479345 
SVM: Feature Importance (Partial Dependence)
This shows how predictions change, on average, as each feature changes (while other features are held at typical values).
Code
Code
