FairnessTest: Fairness testing of an ML model

In RifatMehreen/moccf: Counterfactual Fairness

Description

Fairness testing of an ML model

Fairness testing of an ML model

Public fields

predictor

(Predictor)
The object (created with iml::Predictor$new()) holding the machine learning model and the data.

df

(data_frame)
The dataset or dataframe for the prediction

sensitive_attribute

(string)
The name of the sensitive attribute.

n_generations

(integerish(1))
The number of generations. Default is 175L.

cfs

(data_frame)
A datframe containing counterfactuals (plausible)

org_cfs

(data_frame)
A dataframe containing all counterfactuals

pred_diff

(data_frame)
A dataframe containing the prediction differences

Methods

Public methods

• FairnessTest$new()

• FairnessTest$generate_counterfactuals()

• FairnessTest$get_prediction_difference()

• FairnessTest$plot_tSNE()

• FairnessTest$get_prediction()

• FairnessTest$prediction_percentages()

• FairnessTest$get_mpd()

• FairnessTest$get_cfactuals_count()

• FairnessTest$clone()

---

Method new()

Create a new FairnessTest object.

Usage

FairnessTest$new(
  predictor = NULL,
  df = NULL,
  sensitive_attribute = NULL,
  n_generations = 175L
)

Arguments

predictor

(Predictor)
The object (created with iml::Predictor$new()) holding the machine learning model and the data.

df

(data_frame)
The dataset or dataframe

sensitive_attribute

(string)
The name of the protected attribute.

n_generations

(integerish(1))
The number of generations. Default is 175L.

---

Method generate_counterfactuals()

Creates a dataframe with the plausible counterfactuals

Usage

FairnessTest$generate_counterfactuals(
  x_interest,
  desired_level,
  desired_prob,
  fixed_features = NULL
)

Arguments

x_interest

(data.table(1) | data.frame(1))
A single row with the observation of interest.

desired_level

(character(1) | NULL)
The desired level of the protected attribute. If NULL (default) then predictor_protected$class is taken.

desired_prob

(numeric(1) | numeric(2))
The desired predicted probability of the desired_level. It can be a numeric scalar or a vector with two numeric values that specify a probability range. For hard classification tasks this can be set to 0 or 1, respectively.

fixed_features

(character() | NULL)
Names of features that are not allowed to change. NULL (default) allows to change all features.

Returns

A dataframe with the plausible counterfactuals

---

Method get_prediction_difference()

Creates a dataframe with the differences of predictions

Usage

FairnessTest$get_prediction_difference(x_interest)

Arguments

x_interest

(data.table(1) | data.frame(1))
A single row with the observation of interest.

Returns

A dataframe with the differences of predictions of the original instance and the counterfactuals

---

Method plot_tSNE()

Creates a plot of the distribution

Usage

FairnessTest$plot_tSNE(x_interest, factor_variable = NULL)

Arguments

x_interest

(data.table(1) | data.frame(1))
A single row with the observation of interest.

factor_variable

(string)
name of another factor variable that we want to see in the plot

---

Method get_prediction()

prints the prediction of the plausible counterfactuals

Usage

FairnessTest$get_prediction()

---

Method prediction_percentages()

calculates the percentage for the predictions

Usage

FairnessTest$prediction_percentages(x_interest)

Arguments

x_interest

(data.table(1) | data.frame(1))
A single row with the observation of interest.

Returns

the percentage for the predictions of generated counterfactuals

---

Method get_mpd()

returns mean of the prediction differences

Usage

FairnessTest$get_mpd()

Returns

mean of the prediction diferences

---

Method get_cfactuals_count()

Gives the total count of counterfactuals

Usage

FairnessTest$get_cfactuals_count()

---

Method clone()

The objects of this class are cloneable with this method.

Usage

FairnessTest$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

if (require("randomForest")) {
  # data pre-processing 
  compas <- fairml::compas
  compas <- compas %>% drop_na()
  compas <- compas %>% distinct()
  # Train a model
  set.seed(142)
  rf = randomForest(two_year_recid ~ ., data = compas[-17L, ])
  # Create a predictor object
  predictor = iml::Predictor$new(rf, type = "prob")
  # Create a FairnessTest object
  fairness_obj = FairnessTest$new(predictor, df = compas, sensitive_attribute = "race", n_generations = 175)
  # Generate plausible counterfactuals
  cfatuals = fairness_obj$generate_counterfactuals(x_interest, desired_level = "Caucasian", desired_prob = c(0.5,1))
  # Find differences of the prediction of counterfactuals and original instance 
  pred_diff = fairness_obj$get_prediction_difference(x_interest)
  # Print the results
  print(pred_diff)
}

RifatMehreen/moccf documentation built on Feb. 18, 2022, 1:35 a.m.