Chapter 1.8: Tree-Based Models: Decision Trees, Random Forests and GBMs

Free companion material for this chapter:

Datasets and R setup for this chapter

Install the packages used in this chapter (free, from CRAN):

install.packages(c("tidyverse", "ExamPAData", "caret", "rpart", "randomForest", "gbm", "pdp"))

health_insurance: ExamPAData package: MIT license (CRAN). Package docs describe it as practice-exam health claims data. A similar public 1,338-row ‘Medical Cost Personal Datasets’ file is on Kaggle under ODbL / Database Contents License. Source: https://cran.r-project.org/package=ExamPAData

install.packages("ExamPAData")
library(ExamPAData)
data(health_insurance)

soa_mortality: License NOT confirmed: this dataset is not in the CRAN release of ExamPAData. Treat it as unavailable until the author confirms the source.

# Not in the CRAN release of ExamPAData (v0.5.0).
# Source to be confirmed by the author before publishing.

insurance.csv: Kaggle ‘Medical Cost Personal Datasets’: Open Database License (ODbL), contents under Database Contents License. Source: https://www.kaggle.com/datasets/mirichoi0218/insurance

# Python example: download insurance.csv from Kaggle (free account required)
# https://www.kaggle.com/datasets/mirichoi0218/insurance
import pandas as pd
data = pd.read_csv("insurance.csv")

Sources and further reading

• rpart vignette: An Introduction to Recursive Partitioning
• gbm vignette (Ridgeway): Generalized Boosted Models
• randomForest on CRAN
• pdp: partial dependence plots in R
• scikit-learn: Prediction intervals for gradient boosting regression
• Interpretable Machine Learning (Christoph Molnar)