Reports
In mlr3fairness: Fairness Auditing and Debiasing for 'mlr3'

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

library(mlr3)
library(mlr3fairness)
td = "../docs/articles"
if (!dir.exists(td)) dir.create(td, recursive = TRUE)
report_dirs = c("datasheet", "modelcard", "fairness")
unlink(paste0(td, "/", report_dirs), recursive = TRUE)

mlr3fairness contains several templates that allow for creating reports based on RMarkdown files. The report_* functions instantiate a new .Rmd file that can be further adapted by the user.

The following reports are currently available in mlr3fairness.

| Report | Description | Reference | | ------------------ | ----------------------- | --------------------- | | report_modelcard | Modelcard for ML models | Mitchell et al., 2018 | | report_datasheet | Datasheet for data sets | Gebru et al., 2018 | | report_fairness | Fairness Report | – |

Usage:

Templates contain a set of pre-defined questions which can be used for reporting as well as initial graphics. The created .Rmd file can then be extended by the user. It can later be converted into a html report using rmarkdown::render().

library(mlr3fairness)
rmdfile = report_datasheet()
rmarkdown::render(rmdfile)

rmdfile = report_modelcard(paste0(td, "/modelcard"))
rmarkdown::render(rmdfile)

Examples

Example: Model Card

rmdfile = report_datasheet(paste0(td, "/datasheet"))
rmarkdown::render(rmdfile)

Example: Data Sheet

task = tsk("adult_train")$filter(1:700)$select(c("age", "education", "marital_status", "sex", "race"))
learner = lrn("classif.rpart", predict_type = "prob")
rr = resample(task, learner, rsmp("cv", folds = 5))
rmdfile = report_fairness(paste0(td, "/fairness"), list(task = task, resample_result = rr))
rmarkdown::render(rmdfile)