add.evidence: Add evidence classes to "umbrella" objects
In metaumbrella: Umbrella Review Package for R

add.evidence

R Documentation

Add evidence classes to “umbrella” objects

Description

Add evidence classes to the factors included in an umbrella review.

Usage

add.evidence(
  x,
  criteria = "Ioannidis",
  class_I = c(n_studies = NA, total_n = NA, n_cases = NA, p_value = NA, I2 = NA,
    imprecision = NA, rob = NA, amstar = NA, egger_p = NA, esb_p = NA, JK_p = NA, pi =
    NA, largest_CI = NA),
  class_II = c(n_studies = NA, total_n = NA, n_cases = NA, p_value = NA, I2 = NA,
    imprecision = NA, rob = NA, amstar = NA, egger_p = NA, esb_p = NA, JK_p = NA, pi =
    NA, largest_CI = NA),
  class_III = c(n_studies = NA, total_n = NA, n_cases = NA, p_value = NA, I2 = NA,
    imprecision = NA, rob = NA, amstar = NA, egger_p = NA, esb_p = NA, JK_p = NA, pi =
    NA, largest_CI = NA),
  class_IV = c(n_studies = NA, total_n = NA, n_cases = NA, p_value = NA, I2 = NA,
    imprecision = NA, rob = NA, amstar = NA, egger_p = NA, esb_p = NA, JK_p = NA, pi =
    NA, largest_CI = NA),
  verbose = TRUE
)

Arguments

`x`	an object of class “umbrella”.
`criteria`	the evidence criteria. It must be "GRADE", "Ioannidis" or "Personalized".
`class_I`	a vector or list of threshold values required for reaching Class I in the Personalized criteria (see details below).
`class_II`	a vector or list of threshold values required for reaching Class II in the Personalized criteria (see details below).
`class_III`	a vector or list of threshold values required for reaching Class III in the Personalized criteria (see details below).
`class_IV`	a vector or list of threshold values required for reaching Class IV in the Personalized criteria (see details below).
`verbose`	logical variable indicating whether text outputs and messages should be generated. We recommend turning this option to FALSE only after having carefully read all the generated messages.

Details

The add.evidence() function performs a stratification of evidence according to three criteria.

"Ioannidis" classification

This classification allows to stratify evidence according to the criteria described in Fusar-Poli & Radua (2018). This classification proposes to stratify evidence in five ordinal classes: "Class I", "Class II", "Class III", "Class IV", "Class ns". The criteria for each class are the following:

Class I: number of cases > 1000, p-value of the meta-analysis < 10^-6, I^2 < 0.5, 95% prediction interval excluding the null, p-value of the Egger test > .05 and p-value of the excess of statistical significance test > .05.
Class II: number of cases > 1000, p-value of the meta-analysis < 10^-6, largest study with a statistically significant effect and class I criteria not met.
Class III: number of cases > 1000, p-value of the meta-analysis < 10^-3 and class I-II criteria not met.
Class IV: p-value of the meta-analysis < 0.05 and class I-III criteria not met.
Class ns: p-value of the meta-analysis >= 0.05. To apply this classification with R and Z effect size measures, you should indicate both the 'n_sample' AND the 'n_cases'.

"GRADE" classification

This classification allows to stratify evidence according to four ordinal classes: "High", "Moderate", "Low", "Very low". Importantly, this classification should not be taken as an equivalent to the subjective approach underlying the standard GRADE classification However, in line with the standard GRADE approach, this classification uses a downgrading procedure in which all factors start with a "High" evidence class that could then be downgraded according to the following criteria:

Imprecision: a total number of participants included in the meta-analysis giving a lower power than 0.8 to detect a SMD = 0.20 leads to a downgrading of 1 class. A number of participants giving a lower power than 0.8 to detect a SMD = 0.50, leads to a downgrading of 2 classes.
Limitations: a proportion of participants included in studies at low risk of bias inferior to 75% leads to a downgrading of 1 class. A proportion inferior to 50% leads to a downgrading of 2 classes.
Publication bias: a p-value of an Egger's test < .10 leads to a downgrading of 1 class.
Inconsistency: an I^2 value >= 0.5 leads to a downgrading of 1 class. This classification is not available for R and Z effect size measures.

"Personalized" classification

Because the "Ioannidis" and "GRADE" classifications do not necessarily provide a rating system that perfectly matches the requirements of your umbrella review, the add.evidence() function offers the possibility to use a "Personalized" criteria to stratify the evidence according to 13 criteria. This Personalized criteria proposes to stratify the evidence in 5 ordinal classes: "Class I", "Class II", "Class III", "Class IV" and "Class V". "Class I" is the highest class that could be achieved and "Class V" is the lowest.
The overall class achieved by a factor is equal to the lowest class achieved by all the criteria used to stratify evidence. For example, if users choose to stratify the evidence according to 3 criteria (the p-value of the meta-analysis, the inconsistency, the publication bias), and that the classes achieved by these 3 criteria are respectively "Class I", "Class III" and "Class IV", the overall class reached by the factor will be "Class IV".
To determine the class that should be assigned to a factor, users have to indicate - for each class - a vector/list of threshold values for all the criteria that are used to stratify the evidence. A description of the criteria and their corresponding inputs is provided below:

n_studies: a number of studies included in the meta-analysis. If the number of studies included in the meta-analysis is strictly superior to the threshold value indicated in studies, the class for which this value is indicated can be reached.
total_n: a total number of participants included in the meta-analysis. If the total number of participants included in the meta-analysis is strictly superior to the threshold value indicated in total_n, the class for which this value is indicated can be reached.
n_cases: a number of cases included in the meta-analysis. If the number of cases included in the meta-analysis is strictly superior to the threshold value indicated in cases, the class for which this value is indicated can be reached.
p_value: a p-value of the pooled effect size under the random-effects model. If the p-value of the pooled effect size is strictly inferior to the threshold value indicated in p.value, the class for which this value is indicated can be reached.
I2: an i-squared (I^2) value. If the I^2 value of the meta-analysis is strictly inferior to the threshold value indicated in I2, the class for which this value is indicated can be reached.
imprecision: a SMD value that will be used to calculate the statistical power of the meta-analysis. If the number of participants included in the meta-analyses allows to obtain a statistical power strictly superior to 80% for the SMD value indicated in imprecision, the class for which this value is indicated can be reached.
rob: a percentage of participants included in studies at low risk of bias. Note that the approach to determining whether a study is at low risk of bias is left to the user. If the percentage of participants included in studies at low risk of bias is strictly superior to the threshold value indicated in rob, the class for which this value is indicated can be reached.
amstar: an AMSTAR rating on the methodological quality of the meta-analysis. If the AMSTAR value of the meta-analysis is strictly superior to the threshold value indicated in amstar, the class for which this value is indicated can be reached.
egger_p: a p-value of an Egger's test for publication bias. If the p-value of the Egger's test is strictly superior to the threshold value indicated in egger_p, the class for which this value is indicated can be reached.
esb_p: a p-value of a test for excess of statistical significance bias (ESB). If the p-value of the test is strictly superior to the threshold value indicated in esb_p, the class for which this value is indicated can be reached.
JK_p: the largest p-value obtained in the jackknife meta-analysis (JK). If the largest p-value obtained in the jackknife meta-analysis is strictly inferior to the threshold value indicated in JK_p, the class for which this value is indicated can be reached.
pi: a "notnull" value indicates that users request the 95% prediction interval of the meta-analysis to exclude the null value to achieve the class for which it is indicated.
largest_CI: a "notnull" value indicates that users request the 95% confidence interval of the largest study included in the meta-analysis to exclude the null value to achieve the class for which it is indicated.

Value

Return an object of class “umbrella” with the evidence classes added.

References

Fusar-Poli, P., & Radua, J. (2018). Ten simple rules for conducting umbrella reviews. Evidence-Based Mental Health, 21, 95-100.

Examples

### perform calculations required for an umbrella review
df <- subset(df.SMD, factor == "Surgical")
umb.full <- umbrella(df)

### stratify evidence according to the Ioannidis classification
evid_ioannidis <- add.evidence(umb.full, criteria = "Ioannidis")
summary(evid_ioannidis)

### stratify evidence according to the Personalized classification with
### the number of studies and cases, the inconsistency as criteria.
### - a class I can be reached if the number of studies is > 10, the number of cases is > 500 and
###   the I2 is < 25%.
### - a class II can be reached if the number of studies is > 5, the number of cases is > 400 and
###   the I2 is < 50%.
### - a class III can be reached if the number of cases is > 300 and the I2 is < 75%.
### - a class IV can be reached if the number of cases is > 100.
### - else, if the number of cases is <= 100, a class V is assigned.
evid_perso1 <- add.evidence(umb.full, criteria = "Personalized",
   class_I = c(n_studies = 10, n_cases = 500, I2 = 25),
   class_II = c(n_studies = 5, n_cases = 400, I2 = 50),
   class_III = c(n_cases = 300, I2 = 75),
   class_IV = c(n_cases = 100))
summary(evid_perso1)

metaumbrella documentation built on Nov. 7, 2023, 1:06 a.m.