demographic_parity | R Documentation |
Demographic parity is satisfied when a model's predictions have the
same predicted positive rate across groups. A value of 0 indicates parity
across groups. Note that this definition does not depend on the true
outcome; the truth
argument is included in outputted metrics
for consistency.
demographic_parity()
is calculated as the difference between the largest
and smallest value of detection_prevalence()
across groups.
Demographic parity is sometimes referred to as group fairness, disparate impact, or statistical parity.
See the "Measuring Disparity" section for details on implementation.
demographic_parity(by)
by |
The column identifier for the sensitive feature. This should be an unquoted column name referring to a column in the un-preprocessed data. |
This function outputs a yardstick fairness metric function. Given a
grouping variable by
, demographic_parity()
will return a yardstick metric
function that is associated with the data-variable grouping by
and a
post-processor. The outputted function will first generate a set
of detection_prevalence metric values by group before summarizing across
groups using the post-processing function.
The outputted function only has a data frame method and is intended to be used as part of a metric set.
By default, this function takes the difference in range of detection_prevalence
.estimate
s across groups. That is, the maximum pair-wise disparity between
groups is the return value of demographic_parity()
's .estimate
.
For finer control of group treatment, construct a context-aware fairness
metric with the new_groupwise_metric()
function by passing a custom aggregate
function:
# the actual default `aggregate` is: diff_range <- function(x, ...) {diff(range(x$.estimate))} demographic_parity_2 <- new_groupwise_metric( fn = detection_prevalence, name = "demographic_parity_2", aggregate = diff_range )
In aggregate()
, x
is the metric_set()
output with detection_prevalence values
for each group, and ...
gives additional arguments (such as a grouping
level to refer to as the "baseline") to pass to the function outputted
by demographic_parity_2()
for context.
Agarwal, A., Beygelzimer, A., Dudik, M., Langford, J., & Wallach, H. (2018). "A Reductions Approach to Fair Classification." Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research. 80:60-69.
Verma, S., & Rubin, J. (2018). "Fairness definitions explained". In Proceedings of the international workshop on software fairness (pp. 1-7).
Bird, S., DudÃk, M., Edgar, R., Horn, B., Lutz, R., Milan, V., ... & Walker, K. (2020). "Fairlearn: A toolkit for assessing and improving fairness in AI". Microsoft, Tech. Rep. MSR-TR-2020-32.
Other fairness metrics:
equal_opportunity()
,
equalized_odds()
library(dplyr)
data(hpc_cv)
head(hpc_cv)
# evaluate `demographic_parity()` by Resample
m_set <- metric_set(demographic_parity(Resample))
# use output like any other metric set
hpc_cv %>%
m_set(truth = obs, estimate = pred)
# can mix fairness metrics and regular metrics
m_set_2 <- metric_set(sens, demographic_parity(Resample))
hpc_cv %>%
m_set_2(truth = obs, estimate = pred)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.