classification_metric: Classification Metric

In aif360: Help Detect and Mitigate Bias in Machine Learning Models

Description

Class for computing metrics based on two BinaryLabelDatasets. The first dataset is the original one and the second is the output of the classification transformer (or similar)

Usage

classification_metric(dataset, classified_dataset, unprivileged_groups, privileged_groups)

Arguments

dataset	(BinaryLabelDataset) Dataset containing ground-truth labels
classified_dataset	(BinaryLabelDataset) Dataset containing predictions
unprivileged_groups	Unprivileged groups. List containing unprivileged protected attribute name and value of the unprivileged protected attribute.
privileged_groups	Privileged groups. List containing privileged protected attribute name and value of the privileged protected attribute.

See Also

Explore available classification metrics explanations here

Available metrics:

• accuracy

• average_abs_odds_difference

• average_odds_difference

• between_all_groups_coefficient_of_variation

• between_all_groups_generalized_entropy_index

• between_all_groups_theil_index

• between_group_coefficient_of_variation

• between_group_generalized_entropy_index

• between_group_theil_index

• binary_confusion_matrix

• coefficient_of_variation

• disparate_impact

• equal_opportunity_difference

• error_rate

• error_rate_difference

• error_rate_ratio

• false_discovery_rate

• false_discovery_rate_difference

• false_discovery_rate_ratio

• false_negative_rate

• false_negative_rate_difference

• false_negative_rate_ratio

• false_omission_rate

• false_omission_rate_difference

• false_omission_rate_ratio

• false_positive_rate

• false_positive_rate_difference

• false_positive_rate_ratio

• generalized_binary_confusion_matrix

• generalized_entropy_index

• generalized_false_negative_rate

• generalized_false_positive_rate

• generalized_true_negative_rate

• generalized_true_positive_rate

• negative_predictive_value

• num_false_negatives

• num_false_positives

• num_generalized_false_negatives

• num_generalized_false_positives

• num_generalized_true_negatives

• num_generalized_true_positives

• num_pred_negatives

• num_pred_positives

• num_true_negatives

• num_true_positives

• performance_measures

• positive_predictive_value

• power

• precision

• recall

• selection_rate

• sensitivity

• specificity

• statistical_parity_difference

• theil_index

• true_negative_rate

• true_positive_rate

• true_positive_rate_difference

Examples

load_aif360_lib()
# Input dataset
data <- data.frame("feat" = c(0,0,1,1,1,1,0,1,1,0), "label" = c(1,0,0,1,0,0,1,0,1,1))
# Create aif compatible input dataset
act <- aif360::aif_dataset(data_path = data,  favor_label=0, unfavor_label=1,
                            unprivileged_protected_attribute=0,
                            privileged_protected_attribute=1,
                            target_column="label", protected_attribute="feat")
# Classified dataset
pred_data <- data.frame("feat" = c(0,0,1,1,1,1,0,1,1,0), "label" = c(1,0,1,1,1,0,1,0,0,1))
# Create aif compatible classified dataset
pred <- aif360::aif_dataset(data_path = pred_data,  favor_label=0, unfavor_label=1,
                             unprivileged_protected_attribute=0,
                             privileged_protected_attribute=1,
                             target_column="label", protected_attribute="feat")
# Create an instance of classification metric
cm <- classification_metric(act, pred, list('feat', 1), list('feat', 0))
# Access metric functions
cm$accuracy()

aif360 documentation built on July 1, 2020, 5:34 p.m.