performance_and_fairness: Performance and fairness

In ModelOriented/FairModels: Flexible Tool for Bias Detection, Visualization, and Mitigation

Performance and fairness

Description

Measure performance in both fairness metric and

Usage

performance_and_fairness(x, fairness_metric = NULL, performance_metric = NULL)

Arguments

x	object of class fairness_object
fairness_metric	fairness metric, one of metrics in fairness_objects parity_loss_metric_data (ACC, TPR, PPV, ...) Full list in fairness_check documentation.
performance_metric	performance metric, one of

Details

Creates perfomance_and_fairness object. Measure model performance and model fairness metric at the same time. Choose best model according to both metrics. When plotted y axis is inversed to accentuate that models in top right corner are the best according to both metrics.

Value

performance_and_fairness object. It is list containing:

• paf_data - performance and fairness data.frame containing fairness and performance metric scores for each model

• fairness_metric - chosen fairness metric name

• performance_metric - chosen performance_metric name

• label - model labels

Examples

data("german")

y_numeric <- as.numeric(german$Risk) - 1

lm_model <- glm(Risk ~ .,
  data = german,
  family = binomial(link = "logit")
)


explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)

fobject <- fairness_check(explainer_lm,
  protected = german$Sex,
  privileged = "male"
)

paf <- performance_and_fairness(fobject)
plot(paf)


rf_model <- ranger::ranger(Risk ~ .,
  data = german,
  probability = TRUE,
  num.trees = 200
)

explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)

fobject <- fairness_check(explainer_rf, fobject)

# same explainers with different cutoffs for female
fobject <- fairness_check(explainer_lm, explainer_rf, fobject,
  protected = german$Sex,
  privileged = "male",
  cutoff = list(female = 0.4),
  label = c("lm_2", "rf_2")
)

paf <- performance_and_fairness(fobject)

plot(paf)

ModelOriented/FairModels documentation built on Aug. 30, 2022, 5:48 p.m.