DistributionWAgg: Aggregation Method: DistributionWAgg

In aggreCAT: Mathematically Aggregating Expert Judgments

Aggregation Method: DistributionWAgg

Description

Calculate the arithmetic mean of distributions created with expert judgements. The aggregate is the median of the average distribution fitted on the individual estimates.

Usage

DistributionWAgg(
  expert_judgements,
  type = "DistribArMean",
  name = NULL,
  placeholder = FALSE,
  percent_toggle = FALSE,
  round_2_filter = TRUE
)

Arguments

expert_judgements	A dataframe in the format of data_ratings.
type	One of "DistribArMean" or "TriDistribArMean".
name	Name for aggregation method. Defaults to type unless specified.
placeholder	Toggle the output of the aggregation method to impute placeholder data.
percent_toggle	Change the values to probabilities. Default is FALSE.
round_2_filter	Note that the IDEA protocol results in both a Round 1 and Round 2 set of probabilities for each claim. Unless otherwise specified, we will assume that the final Round 2 responses (after discussion) are being referred to.

Details

\loadmathjax

This method assumes that the elicited probabilities and bounds can be considered to represent participants' subjective distributions associated with relative frequencies (rather than unique events). That is to say that we considered that the lower bound of the individual per claim corresponds to the 5th percentile of their subjective distribution on the probability of replication, denoted \mjeqnq_5,iascii, the best estimate corresponds to the median, \mjeqnq_50,iascii, and the upper bound corresponds to the 95th percentile, \mjeqnq_95,iascii. With these three percentiles, we can fit parametric or non-parametric distributions and aggregate them rather than the (point) best estimates.

type may be one of the following:

DistribArMean: Applies a non-parametric distribution evenly across upper, lower and best estimates.

Using the three percentiles we can build the minimally informative non-parametric distribution that spreads the mass uniformly between the three percentiles.

\mjdeqn

F_i(x) = \begincases \displaystyle 0, \text for x<0
\displaystyle \frac0.05q_5,i\cdot x, \text for 0 \leq x< q_5,i
\displaystyle \frac0.45q_50,i-q_5,i\cdot(x-q_5,i)+0.05, \text for q_5,i\leq x< q_50,i
\displaystyle \frac0.45q_95,i-q_50,i\cdot(x-q_50,i)+0.5, \text for q_50,i\leq x< q_95,i
\displaystyle \frac0.051 - q_95,i\cdot(x-q_95,i)+0.95, \text for q_95,i\leq x< 1
\displaystyle 1, \text for x\geq 1. \endcasesascii

Then take the average of all constructed distributions of participants for each claim:

\mjdeqn

AvDistribution = \frac1N\sum_i=1^N F_i(x),ascii

and the aggregation is the median of the average distribution:

\mjdeqn\hat

p_c\left( DistribArMean \right) = AvDistribution^-1(0.5).ascii

TriDistribArMean: Applies a triangular distribution to the upper, lower and best estimates.

A more restrictive fit with different assumptions about the elicited best estimates, upper and lower bounds. We can assume that the lower and upper bounds form the support of the distribution, and the best estimate corresponds to the mode.

\mjdeqn

F_i(x)= \begincases \displaystyle 0, \text for x < L_i
\displaystyle \frac\left( x-L_i\right)^2\left( U_i-L_i\right)\left( B_i-L_i \right), \text for L_i \leq x < B_i
\displaystyle 1 - \frac\left( U_i-x\right)^2\left( U_i-L_i\right)\left ( U_i-B_i\right), \text for B_i < x < U_i
\displaystyle 1, \text for x \geq U_i. \endcasesascii

Then take the average of all constructed distributions of participants for each claim:

\mjdeqn

AvDistribution = \frac1N\sum_i=1^N F_i(x),ascii

and the aggregation is the median of the average distribution:

\mjdeqn \hat

p_c\left(TriDistribArMean\right) = AvDistribution^-1(0.5).ascii

Value

A tibble of confidence scores cs for each paper_id.

Examples

DistributionWAgg(data_ratings)

aggreCAT documentation built on June 8, 2025, 11:06 a.m.