mcse: Monte Carlo Standard Errors for Tests
In nptest: Nonparametric Bootstrap and Permutation Tests

mcse	R Documentation

Monte Carlo Standard Errors for Tests

Description

This function calculates Monte Carlo standard errors for (non-exact) nonparametric tests. The MCSEs can be used to determine (i) the accuracy of a test for a given number of resamples, or (ii) the number of resamples needed to achieve a test with a given accuracy.

Usage

mcse(R, delta, conf.level = 0.95, sig.level = 0.05,
     alternative = c("two.sided", "one.sided"))

Arguments

`R`	Number of resamples (positive integer).
`delta`	Accuracy of the approximation (number between 0 and 1).
`conf.level`	Confidence level for the approximation (number between 0 and 1).
`sig.level`	Significance level of the test (number between 0 and 1).
`alternative`	Alternative hypothesis (two-sided or one-sided).

Details

Note: either R or delta must be provided.

Let F(x) denote the distribution function for the full permutation distribution, and let G(x) denote the approximation obtained from R resamples. The Monte Carlo standard error is given by

\sigma(x) = \sqrt{ F(x) [1 - F(x)] / R }

which is the standard deviation of G(x).

A symmetric confidence interval for F(x) can be approximated as

G(x) +/- C \sigma(x)

where C is some quantile of the standard normal distribution. Note that the critical value C corresponds to the confidence level (conf.level) of the approximation.

Let \alpha denote the significance level (sig.level) for a one-sided test (\alpha is one-half the significance level for two-sided tests). Define a to be the value of the test statistic such that F(a) = \alpha.

The parameter \delta (delta) quantifies the accuracy of the approximation, such that

|G(a) - \alpha| < \alpha \delta

with a given confidence, which is controlled by the conf.level argument.

Value

`mcse`	Monte Carlo standard error.
`R`	Number of resamples.
`delta`	Accuracy of approximation.
`conf.level`	Confidence level.
`sig.level`	Significance level.
`alternative`	Alternative hypothesis.

Note

This function is only relevant for non-exact tests. For exact tests, F(x) = G(x) so the Monte Carlo standard error is zero.

Author(s)

Nathaniel E. Helwig <helwig@umn.edu>

References

Helwig, N. E. (2019). Statistical nonparametric mapping: Multivariate permutation tests for location, correlation, and regression problems in neuroimaging. WIREs Computational Statistics, 11(2), e1457. doi: 10.1002/wics.1457

Examples


###***###   EXAMPLE 1   ###***###

# get the Monte Carlo standard error and the 
# accuracy (i.e., delta) for given R = 10000
# using the default two-sided alternative hypothesis,
# the default confidence level (conf.level = 0.95),
# and the default significance level (sig.level = 0.05)

mcse(R = 10000)

# se = 0.0016
# delta = 0.1224


###***###   EXAMPLE 2   ###***###

# get the Monte Carlo standard error and the 
# number of resamples (i.e., R) for given delta = 0.01
# using a one-sided alternative hypothesis,
# the default confidence level (conf.level = 0.95),
# and the default significance level (sig.level = 0.05)

mcse(delta = 0.1, alternative = "one.sided")

# se = 0.0026
# R = 7299

nptest documentation built on April 15, 2023, 1:08 a.m.