corr_tests: Global evidence strength across correlated tests

In NRejections: Metrics for Multiple Testing with Correlated Outcomes

Description

This is the main wrapper function for the user to call. For an arbitrary number of outcome variables, regresses the outcome on an exposure of interest (X) and adjusted covariates (C). Returns the results of the original sample (statistics and inference corresponding to X for each model, along with the observed number of rejections), a 100*(1 - alpha.fam) percent null interval for the number of rejections in samples generated under the global null, the excess hits (the difference between the observed number of rejections and the upper null interval limit), and results of a test of the global null hypothesis at alpha.fam of the global null. The global test can be conducted based on the number of rejections or based on various FWER-control methods (see References).

Usage

corr_tests(
  d,
  X,
  C = NA,
  Ys,
  B = 2000,
  cores,
  alpha = 0.05,
  alpha.fam = 0.05,
  method = "nreject"
)

Arguments

d	Dataframe
X	Single quoted name of covariate of interest
C	Vector of quoted covariate names
Ys	Vector of quoted outcome names
B	Number of resamples to generate
cores	Number of cores to use for parallelization. Defaults to number available.
alpha	Alpha level for individual hypothesis tests
alpha.fam	Alpha level for global test and null interval
method	Which methods to report (ours, Westfall's two methods, Bonferroni, Holm, Romano)

Value

samp.res is a list containing the number of observed rejections (rej), the coefficient estimates of interest for each outcome model (bhats), their t-values (tvals), their uncorrected p-values at level alpha (pvals), and an N X W matrix of residuals for each model (resid).

nrej.bt contains the number of rejections in each bootstrap resample.

tvals.bt is a W X B matrix containing t-values for the resamples.

pvals.bt is a W X B matrix containing p-values for the resamples.

null.int contains the lower and upper limits of a 100*(1 - alpha.fam) percent null interval.

excess.hits is the difference between the observed rejections and the upper limit of the null interval.

global.test is a dataframe containing global test results for each user-specified method, including an indicator for whether the test rejects the global null at alpha.fam (reject), the p-value of the global test where possible (reject), and the critical value of the global test based on the number of rejections (crit).

References

Mathur, M.B., & VanderWeele, T.J. (in preparation). New metrics for multiple testing with correlated outcomes.

Romano, J. P., & Wolf, M. (2007). Control of generalized error rates in multiple testing. The Annals of Statistics, 1378-1408.

Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. Taylor & Francis Group.

Examples

##### Example 1 #####
data(rock)

res = corr_tests( d = rock,
                 X = c("area"),
                 C = NA,
                 Ys = c("perm", "peri", "shape"),
                 method = "nreject" )

# mean rejections in resamples
# should be close to 0.05 * 3 = 0.15
mean( as.numeric(res$nrej.bt) ) 


##### Example 1 #####
cor = make_corr_mat( nX = 10,
nY = 20,
rho.XX = 0.10,
rho.YY = 0.5,
rho.XY = 0.1,
prop.corr = .4 )

d = sim_data( n = 300, cor = cor )

# X1 is the covariate of interest, and all other X variables are adjusted
all.covars = names(d)[ grep( "X", names(d) ) ]
C = all.covars[ !all.covars == "X1" ]

# may take 10 min to run
res = corr_tests( d,
                 X = "X1",
                 C = C,
                 Ys = names(d)[ grep( "Y", names(d) ) ],
                 method = "nreject" )

# look at the main results
res$null.int
res$excess.hits
res$global.test

NRejections documentation built on July 9, 2020, 5:08 p.m.