group_metric: Group metric

In fairmodels: Flexible Tool for Bias Detection, Visualization, and Mitigation

Group metric

Description

Group metric enables to extract data from metrics generated for each subgroup (values in protected variable) The closer metric values are to each other, the less bias particular model has. If parity_loss parameter is set to TRUE, distance between privileged and unprivileged subgroups will be measured. When plotted shows both fairness metric and chosen performance metric.

Usage

group_metric(
  x,
  fairness_metric = NULL,
  performance_metric = NULL,
  parity_loss = FALSE,
  verbose = TRUE
)

Arguments

x	object of class fairness_object
fairness_metric	character, fairness metric name, if NULL the default metric will be used which is TPR.
performance_metric	character, performance metric name
parity_loss	logical, if TRUE parity loss will supersede basic metric
verbose	logical, whether to print information about metrics on console or not. Default TRUE

Details

Available metrics:

Fairness metrics (Full names explained in fairness_check documentation):

• TPR: True Positive Rate

• TNR: True Negative Rate

• PPV: Positive Predictive Value

• NPV: Negative Predictive Value

• FNR: False Negative Rate

• FPR: False Positive Rate

• FDR: False Discovery Rate

• FOR: False Omission Rate

• TS: Threat Score

• ACC: Accuracy

• STP: Statistical Parity

• F1: F1 Score

Performance metrics:

• recall: Recall

• precision: Precision

• accuracy: Accuracy

• f1: F1 Score

• auc: Area Under the Curve

Value

group_metric object. It is a list with following items:

group_metric_data	- data.frame containing fairness metric scores for each model
performance_data	- data.frame containing performance metric scores for each model
fairness_metric	- name of fairness metric
performance_metric	- name of performance metric

Examples

data("german")

y_numeric <- as.numeric(german$Risk) - 1

lm_model <- glm(Risk ~ .,
  data = german,
  family = binomial(link = "logit")
)


explainer_lm <- DALEX::explain(lm_model, data = german[, -1], y = y_numeric)

fobject <- fairness_check(explainer_lm,
  protected = german$Sex,
  privileged = "male"
)

gm <- group_metric(fobject, "TPR", "f1", parity_loss = TRUE)
plot(gm)


rf_model <- ranger::ranger(Risk ~ .,
  data = german,
  probability = TRUE,
  num.trees = 200
)

explainer_rf <- DALEX::explain(rf_model, data = german[, -1], y = y_numeric)

fobject <- fairness_check(explainer_rf, fobject)

gm <- group_metric(fobject, "TPR", "f1", parity_loss = TRUE)

plot(gm)

fairmodels documentation built on Dec. 1, 2025, 1:08 a.m.