# frag.study: Assessing Fragility of an Individual Study With a Binary... In fragility: Assessing and Visualizing Fragility of Clinical Results with Binary Outcomes

## Description

Produces results, such as fragility index, fragility quotient, and event status modifications for altering statistical significance, of an individual study with a binary outcome.

## Usage

 1 2 3 4 frag.study(e0, n0, e1, n1, data, all = FALSE, methods, modify0 = "both", modify1 = "both", alpha = 0.05, alternative = "two.sided", OR = 1, RR = 1, RD = 0, allcase = TRUE) 

## Arguments

 e0 a numeric value or the corresponding column name in the argument data, indicating event count in group 0. n0 a numeric value or the corresponding column name in the argument data, indicating sample size in group 0. e1 a numeric value or the corresponding column name in the argument data, indicating event count in group 1. n1 a numeric value or the corresponding column name in the argument data, indicating sample size in group 1. data an optional data frame containing the dataset of the individual study with a binary outcome. If data is specified, the previous arguments, e0, n0, e1, and n1, should be specified as their corresponding column names in data. all a logical value indicating whether all eligible event status modifications will be considered (the default is FALSE). methods a character string or a vector of character strings indicating the method(s) used to calculate the p-value(s) of the association between treatment and outcome. It must consist of "Fisher" (Fisher's exact test), "chisq" (chi-squared test), "OR" (odds ratio), "RR" (relative risk), and/or "RD" (risk difference). The default is the vector consisting of all aforementioned five methods. modify0 a character string indicating the event status modifications in group 0. It should be one of "increase" (increasing event numbers), "decrease" (decreasing event numbers), "both" (the default, modifying event numbers in both directions), and "none" (no modification). modify1 a character string indicating the event status modifications in group 1. It has the same usage with modify0, with the default being "both". alpha a numeric value between 0 and 1, indicating the statistical significance level (the default is 0.05). alternative a character string of either "two.sided" (the default) or "one.sided", indicating the alternative hypothesis. It is only used for "OR", "RR", and "RD" in the argument methods. OR a numeric positive value indicating the value of odds ratio under the null hypothesis (the default is 1). It is used only if the argument methods includes "OR". RR a numeric positive value indicating the value of relative risk under the null hypothesis (the default is 1). It is used only if the argument methods includes "RR". RD a numeric value between -1 indicating the value of risk difference under the null hypothesis (the default is 0). It is used only if the argument methods includes "RD". allcase a logical value indicating whether obtaining all cases of minimal event status modifications for altering statistical significance or non-significance (the default is TRUE).

## Value

An object of class "frag.study" (if the argument all = FALSE) or of both classes "frag.study" and "frag.study.all" (all = TRUE). The object is a list containing the following components:

 data original data in the form of 2\times2 table. methods methods used to calculate the p-value(s) of the association between treatment and outcome. alpha pre-specified statistical significance level. alternative alternative hypothesis. null values of odds ratio, relative risk, and risk difference under the null hypothesis; they are only used if "OR", "RR", and "RD" are included in the argument methods, respectively. modify0 type of event status modifications in group 0. modify1 type of event status modifications in group 1. f0.range range of event status modifications in group 0. f1.range range of event status modifications in group 1. allcase a logical value indicating whether all cases of minimal event status modifications for altering statistical significance are obtained; it is only included in the object of class "frag.study". tot.mods a matrix containing all possible event status modifications with rows representing modifications in group 0 and columns representing those in group 1; it is determined by f0.range and f1.range, and is only included in the object of class "frag.study.all". pval p-value(s) produced by the method(s) included in the argument methods. FI fragility index(es) based on the method(s). FQ fragility quotient(s) based on the method(s). dir direction of the change of statistical significance. mods minimal event status modification(s) that alter statistical significance or non-significance. pvals p-values based on all possible event status modifications (as in tot.mods); it is only included in the object of class "frag.study.all". FI0 fragility index(es) based on the method(s) when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". FQ0 fragility quotient(s) based on the method(s) when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". dir0 direction of the change of statistical significance when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". mods0 minimal event status modification(s) that alter statistical significance or non-significance when event status modifications are restricted to group 0; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". FI1 fragility index(es) based on the method(s) when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". FQ1 fragility quotient(s) based on the method(s) when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". dir1 direction of the change of statistical significance when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none". mods1 minimal event status modification(s) that alter statistical significance or non-significance when event status modifications are restricted to group 1; it is only included in the object of class "frag.study.all" if both arguments modify0 and modify1 are not "none".

Of note, if both arguments modify0 and modify0 are "none" (i.e., no modification), the items following pval in the above list are not produced, because fragility cannot be assessed without event status modifications.

## Note

If the significance or non-significance cannot be altered based on the given range of event status modifications, the fragility index are fragility quotient are defined to be NA (not available).

## References

Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C, Molnar AO, Dattani ND, Burke A, Guyatt G, Thabane L, Walter SD, Pogue J, Devereaux PJ (2014). "The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index." Journal of Clinical Epidemiology, 67(6), 622–8. <doi: 10.1016/j.jclinepi.2013.10.019>

frag.study.alpha for assessing fragility of an individual study at multiple significance levels; frag.studies for assessing fragility of multiple studies; frag.ma for assessing fragility of a meta-analysis; and frag.nma for assessing fragility of a network meta-analysis.
  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 ## Load datasets of trials on antidepressant drugs data(dat.ad) ## Assess fragility of trial 13 out1 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,]) out1 ## Consider all possible event status modifications ## and focus on odds ratio out2 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,], all = TRUE, methods = "OR") out2 ## Visualize the fragility plot(out2) ## Restrict the modifications to treatment group 0 plot(out2, modify1 = FALSE) ## Change the truncation threshold of p-value to 10^-30 plot(out2, modify1 = FALSE, trun = 30) ## Restrict the modifications to treatment group 1 plot(out2, modify0 = FALSE) ## Assess fragility of trial 13 ## by only increasing event counts in treatment group 0 out3 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,], all = TRUE, modify0 = "increase", modify1 = "none") out3 ## The default plot is for the first method (i.e., Fisher's exact test) ## if the argument method is not specified plot(out3) ## Generate plots based on other methods plot(out3, method = "chisq") plot(out3, method = "OR") plot(out3, method = "RR") plot(out3, method = "RD") ## Assess fragility of trial 13 ## by only modifying event status in treatment group 1 out4 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[13,], all = TRUE, modify0 = "none", modify1 = "both") out4 plot(out4) ## Change background colors in the plot plot(out4, col.sig = c(adjustcolor("blue", alpha.f = 0.1), adjustcolor("green", alpha.f = 0.1))) ## Adjust the plot by specifying additional arguments of plot.default() plot(out4, mgp = c(2, 1, 0), main = "Trial 13", font.main = 1) ## An example of non-significance altered to significance out5 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[21,], all = TRUE) out5 plot(out5, method = "Fisher", legend.pvals = 0.001, main = paste("Fragility index =", out5\$FI["Fisher"], "in trial 21")) ## Change the statistical significance level to 0.005 out6 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[21,], all = TRUE, alpha = 0.005) out6 plot(out6) ## Change the null value of odds ratio to 2 out7 <- frag.study(e0 = e0, n0 = n0, e1 = e1, n1 = n1, data = dat.ad[21,], all = TRUE, methods = c("OR"), alpha = 0.05, OR = 2) out7 plot(out7)