Permutation test for wavelet-domain scalar-on-function regression

Description

This function assesses statistical significance of a wcr or wnet fit by referring the cross-validation criterion to a permutation distribution.

Usage

1
2
3
4
5
6
7
8
wcr.perm(y, xfuncs, min.scale = 0, nfeatures, ncomp, method = c("pcr", "pls"), 
         covt = NULL, nrep = 1, nsplit = 1, nfold = 5, nperm = 20, 
         perm.method = NULL, family = "gaussian", seed.real = NULL,
         seed.perm = NULL, ...)
wnet.perm(y, xfuncs, min.scale = 0, nfeatures = NULL, alpha = 1, lambda, 
         covt = NULL, nrep = 1, nsplit = 1, nfold = 5, nperm = 20, 
         perm.method = NULL, family = "gaussian", seed.real = NULL,
         seed.perm = NULL, ...)         

Arguments

y, xfuncs, min.scale, nfeatures, method, covt, family, nsplit, nfold

arguments passed to wcr or wnet.

ncomp

number of components; passed to wcr.

alpha, lambda

tuning parameters, passed to wnet.

nrep

number of replicates to average over, when computing the real-data CV criterion.

nperm

number of permutations. The default is suitable for toy applications only.

perm.method

either NULL or one of

  • "responses": permute the response vector y.

  • "y.residuals": permute the residuals upon regressing y ~ covt.

  • "x.residuals": permute the residuals upon regressing xfuncs ~ covt.

See Details.

seed.real

the seed for random data division for real data. If seed.real = NULL, a random seed is used.

seed.perm

the seed for random data division for permuted data. If seed.perm = NULL, a random seed is used.

...

other arguments passed to wcr or wnet.

Details

Permutation tests of this type are discussed, in a classification setting, by Ojala and Garriga (2010). Permuting the responses (perm.method="responses") is appropriate when regressing on functions/images only, with no scalar covariates. For linear regression with covariates, it is preferable to first regress on the covariates, and then permute the residuals (perm.method="y.residuals"). For logistic regression this is not feasible; but, following Potter (2005), one can instead permute the residuals from a regression of the functions/images on the covariates (perm.method="x.residuals"). When perm.method=NULL (the default), "responses" is used if covt is NULL, and "x.residuals" otherwise.

Value

cv

CV value for the real data (averaged over nrep replications).

cv.perm

CV values for the permuted data.

pvalue

p-value for the permutation test.

Author(s)

Lan Huo

References

Ojala, M., and Garriga, G. C. (2010). Permutation tests for studying classifier performance. Journal of Machine Learning Research, 11, 1833–1863.

Potter, D. M. (2005). A permutation test for inference in logistic regression with small- and moderate-sized data sets. Statistics in Medicine, 24, 693–708.

Reiss, P. T., Huo, L., Zhao, Y., Kelly, C., and Ogden, R. T. (2014). Wavelet-domain regression and predictive inference in psychiatric neuroimaging. Available at http://works.bepress.com/phil_reiss/29/

See Also

wcr, wnet

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
n = 200; d = 64

# Create true coefficient function
ftrue = matrix(0,d,d)
ftrue[40:46,34:38] = 1

# Generate random functional predictors, and scalar responses
ii = array(rnorm(n*d^2), dim=c(n,d,d))
iimat = ii; dim(iimat) = c(n,d^2)
yy = iimat %*% as.vector(ftrue) + rnorm(n, sd=.3)
## Not run: 
obj.wnet.perm <- wnet.perm(yy, xfuncs = ii, min.scale = 4, nfeatures = 200, alpha = 1, 
                           nperm = 10)

obj.wcr.perm <- wcr.perm(yy, xfuncs = ii, min.scale = 4, nfeatures = 20, ncomp = 6, 
                         cv1 = TRUE, method = "pls", nperm = 10)

## End(Not run)