IntervalWAgg: Aggregation Method: IntervalWAgg
In aggreCAT: Mathematically Aggregating Expert Judgments
IntervalWAgg R Documentation
Aggregation Method: IntervalWAgg
Description
Calculate one of several types of linear-weighted best estimates where the weights
are dependent on the lower and upper bounds of three-point elicitation (interval widths).
Usage
IntervalWAgg(
expert_judgements,
type = "IntWAgg",
name = NULL,
placeholder = FALSE,
percent_toggle = FALSE,
round_2_filter = TRUE
)
Arguments
expert_judgements
A dataframe in the format of data_ratings.
type
One of "IntWAgg", "IndIntWAgg", "AsymWAgg", "IndIntAsymWAgg", "VarIndIntWAgg",
"KitchSinkWAgg".
name
Name for aggregation method. Defaults to type unless specified.
placeholder
Toggle the output of the aggregation method to impute placeholder data.
percent_toggle
Change the values to probabilities. Default is FALSE.
round_2_filter
Note that the IDEA protocol results in both a Round 1
and Round 2 set of probabilities for each claim. Unless otherwise specified,
we will assume that the final Round 2 responses (after discussion) are being
referred to.
Details
The width of the interval provided by individuals may be an indicator of certainty,
and arguably of accuracy of the best estimate contained between the bounds of the interval.
type may be one of the following:
IntWAgg: Weighted according to the interval width across individuals for that claim, rewarding narrow interval widths.
\loadmathjax
\mjdeqnw\_Interval_i,c= \frac1U_i,c - L_i,cascii
\mjdeqn\hatp_c( IntWAgg) = \sum_i=1^N \tildew\_Interval_i,cB_i,cascii
where \mjeqnU_i,d - L_i,dascii are individual \mjeqniascii's judgements for claim \mjeqndascii. Then
IndIntWAgg: Weighted by the rescaled interval width (interval width relative to largest
interval width provided by that individual)
Because of the variability in interval widths between individuals across claims,
it may be beneficial to account for this individual variability by rescaling interval
widths across all claims per individual. This results in a re-scaled interval width weight,
for individual \mjeqniascii for claim \mjeqncascii, relative to the widest interval provided by
that individual across all claims \mjeqnCascii:
\mjdeqnw\_nIndivInterval_i,c= \frac1\fracU_i,c - L_i,cmax( (U_i,d
- L_i,d): d = 1,..., C)ascii
\mjdeqn\hatp_c\left( IndIntWAgg \right) = \sum_i=1^N \tildew\_nIndivInterval_i,cB_i,cascii
AsymWAgg: Weighted by the asymmetry of individuals' intervals, rewarding increasing asymmetry.
We use the asymmetry of an interval relative to the corresponding best estimate
to define the following weights:
\mjdeqnw\_asym_i,c= \begincases
1 - 2 \cdot \fracU_i,c-B_i,cU_i,c-L_i,c, \textfor\ B_i,c \geq
\fracU_i,c-L_i,c2+L_i,c
1 - 2 \cdot \fracB_i,c-L_i,cU_i,c-L_i,c, \textotherwise
\endcasesascii
then,
\mjdeqn\hatp_c(AsymWAgg) = \sum_i=1^N \tildew\_asym_i,cB_i,c.ascii
IndIntAsymWAgg: Weighted by individuals’ interval widths and asymmetry
This rewards both asymmetric and narrow intervals. We simply multiply the weights calculated
in the "AsymWAgg" and "IndIntWAgg" methods.
\mjdeqnw\_nIndivInterval\_asym_i,c = \tildew\_nIndivInterval_i,c \cdot \tildew\_asym_i,cascii
\mjdeqn\hatp_c( IndIntAsymWAgg) = \sum_i=1^N \tildew\_nIndivInterval\_asym_i,cB_i,cascii
VarIndIntWAgg: Weighted by the variation in individuals’ interval widths
A higher variance in individuals' interval width across claims may indicate a higher responsiveness
to the supporting evidence of different claims. Such responsiveness might be predictive of more
accurate assessors. We define:
\mjdeqnw\_varIndivInterval_i= var(U_i,c - L_i,c): c = 1,..., C,ascii
where the variance (\mjeqnvarascii) is calculated across all claims for individual \mjeqniascii.
Then,
\mjdeqn\hatp_c(VarIndIntWAgg) = \sum_i=1^N \tildew\_varIndivInterval_iB_i,cascii
KitchSinkWAgg: Weighted by everything but the kitchen sink
This method is informed by the intuition that we want to reward narrow and asymmetric intervals,
as well as the variability of individuals' interval widths (across their estimates). Again, we multiply
the weights calculated in the "AsymWAgg", "IndIntWAgg" and "VarIndIntWAgg" methods above.
\mjdeqnw\_kitchSink_i,c = \tildew\_nIndivInterval_i,c \cdot \tildew\_asym_i,c \cdot \tildew\_varIndivInterval_iascii
\mjdeqn\hatp_c(KitchSinkWAgg) = \sum_i=1^N \tildew\_kitchSink_i,cB_i,cascii
Value
A tibble of confidence scores cs for each paper_id.
Examples
IntervalWAgg(data_ratings)
aggreCAT documentation built on June 8, 2025, 11:06 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| IntervalWAgg | R Documentation |
Aggregation Method: IntervalWAgg
Description
Calculate one of several types of linear-weighted best estimates where the weights are dependent on the lower and upper bounds of three-point elicitation (interval widths).
Usage
IntervalWAgg(
expert_judgements,
type = "IntWAgg",
name = NULL,
placeholder = FALSE,
percent_toggle = FALSE,
round_2_filter = TRUE
)
Arguments
expert_judgements |
A dataframe in the format of data_ratings. |
type |
One of |
name |
Name for aggregation method. Defaults to |
placeholder |
Toggle the output of the aggregation method to impute placeholder data. |
percent_toggle |
Change the values to probabilities. Default is |
round_2_filter |
Note that the IDEA protocol results in both a Round 1 and Round 2 set of probabilities for each claim. Unless otherwise specified, we will assume that the final Round 2 responses (after discussion) are being referred to. |
Details
The width of the interval provided by individuals may be an indicator of certainty, and arguably of accuracy of the best estimate contained between the bounds of the interval.
type may be one of the following:
IntWAgg: Weighted according to the interval width across individuals for that claim, rewarding narrow interval widths. \loadmathjax \mjdeqnw\_Interval_i,c= \frac1U_i,c - L_i,cascii \mjdeqn\hatp_c( IntWAgg) = \sum_i=1^N \tildew\_Interval_i,cB_i,cascii
where \mjeqnU_i,d - L_i,dascii are individual \mjeqniascii's judgements for claim \mjeqndascii. Then
IndIntWAgg: Weighted by the rescaled interval width (interval width relative to largest interval width provided by that individual)
Because of the variability in interval widths between individuals across claims, it may be beneficial to account for this individual variability by rescaling interval widths across all claims per individual. This results in a re-scaled interval width weight, for individual \mjeqniascii for claim \mjeqncascii, relative to the widest interval provided by that individual across all claims \mjeqnCascii:
\mjdeqnw\_nIndivInterval_i,c= \frac1\fracU_i,c - L_i,cmax( (U_i,d - L_i,d): d = 1,..., C)ascii
\mjdeqn\hatp_c\left( IndIntWAgg \right) = \sum_i=1^N \tildew\_nIndivInterval_i,cB_i,cascii
AsymWAgg: Weighted by the asymmetry of individuals' intervals, rewarding increasing asymmetry.
We use the asymmetry of an interval relative to the corresponding best estimate to define the following weights:
\mjdeqnw\_asym_i,c= \begincases
1 - 2 \cdot \fracU_i,c-B_i,cU_i,c-L_i,c, \textfor\ B_i,c \geq
\fracU_i,c-L_i,c2+L_i,c
1 - 2 \cdot \fracB_i,c-L_i,cU_i,c-L_i,c, \textotherwise
\endcasesascii
then,
\mjdeqn\hatp_c(AsymWAgg) = \sum_i=1^N \tildew\_asym_i,cB_i,c.ascii
IndIntAsymWAgg: Weighted by individuals’ interval widths and asymmetry
This rewards both asymmetric and narrow intervals. We simply multiply the weights calculated
in the "AsymWAgg" and "IndIntWAgg" methods.
w\_nIndivInterval\_asym_i,c = \tildew\_nIndivInterval_i,c \cdot \tildew\_asym_i,cascii
\mjdeqn\hatp_c( IndIntAsymWAgg) = \sum_i=1^N \tildew\_nIndivInterval\_asym_i,cB_i,cascii
VarIndIntWAgg: Weighted by the variation in individuals’ interval widths
A higher variance in individuals' interval width across claims may indicate a higher responsiveness to the supporting evidence of different claims. Such responsiveness might be predictive of more accurate assessors. We define:
\mjdeqnw\_varIndivInterval_i= var(U_i,c - L_i,c): c = 1,..., C,ascii
where the variance (\mjeqnvarascii) is calculated across all claims for individual \mjeqniascii. Then,
\mjdeqn\hatp_c(VarIndIntWAgg) = \sum_i=1^N \tildew\_varIndivInterval_iB_i,cascii
KitchSinkWAgg: Weighted by everything but the kitchen sink
This method is informed by the intuition that we want to reward narrow and asymmetric intervals,
as well as the variability of individuals' interval widths (across their estimates). Again, we multiply
the weights calculated in the "AsymWAgg", "IndIntWAgg" and "VarIndIntWAgg" methods above.
w\_kitchSink_i,c = \tildew\_nIndivInterval_i,c \cdot \tildew\_asym_i,c \cdot \tildew\_varIndivInterval_iascii
\mjdeqn\hatp_c(KitchSinkWAgg) = \sum_i=1^N \tildew\_kitchSink_i,cB_i,cascii
Value
A tibble of confidence scores cs for each paper_id.
Examples
IntervalWAgg(data_ratings)
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.