README.md

The flashlight package

CRAN version

The goal of this package is shed light on black box machine learning models.

The main props of flashlight:

  1. It is simple, yet flexible.

  2. It offers model agnostic tools like model performance, variable importance, global surrogate models, ICE profiles, partial dependence, ALE, and further effects plots, scatter plots, interaction strength, and variable contribution breakdown/SHAP for single observations.

  3. It allows to assess multiple models in parallel.

  4. It supports "group by" operations.

  5. It works with case weights.

Currently, models with numeric or binary response are supported.

Installation

From CRAN:

install.packages("flashlight")

Latest version from github:

library(devtools)
install_github("mayer79/flashlight", subdir = "release/flashlight")

Example Code

library(ggplot2)
library(MetricsWeighted)
library(flashlight)

# Fit model
fit <- lm(Sepal.Length ~ ., data = iris)

# Make flashlight
fl <- flashlight(
  model = fit, 
  data = iris, 
  y = "Sepal.Length", 
  label = "ols",               
  metrics = list(rmse = rmse, `R-squared` = r_squared)
)

Performance (overall and grouped by Species)

plot(light_performance(fl), fill = "darkred") +
  ggtitle("Overall")
plot(light_performance(fl, by = "Species"), fill = "darkred") +
  ggtitle("Grouped by Species")

Performance Grouped

Permutation importance

imp <- light_importance(fl, m_repetitions = 4)
plot(imp, fill = "darkred")

ICE curves for Petal.Width

plot(light_ice(fl, v = "Petal.Width"))

Partial dependence plot for Petal.Width

plot(light_profile(fl, v = "Petal.Width")) +
  ggtitle("Overall")
plot(light_profile(fl, v = "Petal.Width", by = "Species")) +
  ggtitle("Grouped by Species")

Partial Dependence Partial Dependence (grouped)

2D partial dependence

plot(light_profile2d(fl, v = c("Petal.Width", "Petal.Length")))

Accumulated local effects (ALE) profiles for Petal.Width

plot(light_profile(fl, v = "Petal.Width", type = "ale"))

Prediction, response and residual profiles, e.g.

plot(light_profile(fl, v = "Petal.Width", type = "residual", 
                   stats = "quartile"))

Different profile plots in one...

plot(light_effects(fl, v = "Petal.Width"), use = "all")

Variable contribution breakdown for single observation

plot(light_breakdown(fl, new_obs = iris[2, ]))

Global surrogate

plot(light_global_surrogate(fl))

Check out the vignette to see the full capabilities of the package.



mayer79/flashlight documentation built on May 8, 2021, 12:47 p.m.