Flexible tool for bias detection, visualization, and mitigation. Use models explained with DALEX and calculate fairness classification metrics based on confusion matrices using `fairness_check()`

or try newly developed module for regression models using `fairness_check_regression()`

. R package fairmodels allows to compare and gain information about various machine learning models. Mitigate bias with various pre-processing and post-processing techniques. *Make sure your models are classifying protected groups similarly*.

Install it from CRAN:

```
install.packages("fairmodels")
```

or developer version from GitHub:

```
devtools::install_github("ModelOriented/fairmodels")
```

Checking fairness is easy!

```
library(fairmodels)
library(ranger)
library(DALEX)
data("german")
# ------------ step 1 - create model(s) -----------------
lm_model <- glm(Risk~.,
data = german,
family=binomial(link="logit"))
rf_model <- ranger(Risk ~.,
data = german,
probability = TRUE,
num.trees = 200)
# ------------ step 2 - create explainer(s) ------------
# numeric y for explain function
y_numeric <- as.numeric(german$Risk) -1
explainer_lm <- explain(lm_model, data = german[,-1], y = y_numeric)
explainer_rf <- explain(rf_model, data = german[,-1], y = y_numeric)
# ------------ step 3 - fairness check -----------------
fobject <- fairness_check(explainer_lm, explainer_rf,
protected = german$Sex,
privileged = "male")
print(fobject)
plot(fobject)
```

Compas recidivism data use case: Basic tutorial Bias mitigation techniques on Adult data: Advanced tutorial

`fairness_check`

parameters are
* x, ... - explainers and fairness_objects (products of fairness_check).
* protected - factor with different subgroups as levels. Usually specific race, sex etc...

Models might be trained on different data, even without protected variable. May have different cutoffs which gives different values of metrics.
`fairness_check()`

is place where `explainers`

and `fairness_objects`

are checked for compatibility and then glued together.
So it is possible to to something like this:

```
fairness_object <- fairness_check(explainer1, explainer2, ...)
fairness_object <- fairness_check(explainer3, explainer4, fairness_object, ...)
```

even with more `fairness_objects`

!

If one is even more keen to know how `fairmodels`

works and what are relations between objects, please look at this diagram class diagram

There are 12 metrics based on confusion matrix :

| Metric | Formula | Full name | fairness names while checking among subgroups | |--------|---------|-----------|-------------| | TPR | | true positive rate | equal opportunity | TNR | | true negative rate | | PPV | | positive predictive value | predictive parity | NPV | | negative predictive value | | FNR | | false negative rate | | FPR | | false positive rate | predictive equality | FDR | | false discovery rate | FOR | | false omission rate | | TS | | threat score | | STP | | statistical parity | statistical parity | ACC | | accuracy | Overall accuracy equality | F1 | | F1 score |

*and their parity loss.*
How is *parity loss* calculated?

Where `i`

denotes the membership to unique subgroup from protected variable. Unprivileged subgroups are represented by small letters and privileged by simply "privileged".

some fairness metrics like *Equalized odds* are satisfied if parity loss in both *TPR* and *FPR* is low

It is relatively easy! Check it out here

R package fairmodels has support for regression models. Check fairness using `fairness_check_regression()`

to approximate classification fairness metrics in regression setting. Plot object with `plot()`

to visualize *fairness check* or with `plot_density()`

to see model's output.

Zafar, Valera, Rodriguez, Gummadi (2017) https://arxiv.org/pdf/1610.08452.pdf

Barocas, Hardt, Narayanan (2019) https://fairmlbook.org/

Steinberg, Daniel & Reid, Alistair & O'Callaghan, Simon. (2020). Fairness Measures for Regression via Probabilistic Classification. - https://arxiv.org/pdf/2001.06089.pdf

**Any scripts or data that you put into this service are public.**

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.