perm.class: Monte Carlo Permutation of Model Performance

In cdeterman/OmicsMarkeR: Classification and Feature Selection for 'Omics' Datasets

Description

Applies Monte Carlo permutations to user specified models. The user can either use the results from fs.stability or provide specified model parameters.

Usage

perm.class(fs.model = NULL, X, Y, method, k.folds = 5,
  metric = "Accuracy", nperm = 10, allowParallel = FALSE,
  create.plot = FALSE, verbose = TRUE, ...)

Arguments

fs.model	Object containing results from fs.stability
X	A scaled matrix or dataframe containing numeric values of each feature
Y	A factor vector containing group membership of samples
method	A string of the model to be fit. Available options are "plsda" (Partial Least Squares Discriminant Analysis), "rf" (Random Forest), "gbm" (Gradient Boosting Machine), "svm" (Support Vector Machines), "glmnet" (Elastic-net Generalized Linear Model), and "pam" (Prediction Analysis of Microarrays)
k.folds	How many and what fractions of dataset held-out for prediction (i.e. 3 = 1/3, 10 = 1/10, etc.)
metric	Performance metric to assess. Available options are "Accuracy", "Kappa", and "ROC.AUC".
nperm	Number of permutations, default nperm = 10
allowParallel	Logical argument dictating if parallel processing is allowed via foreach package. Default allowParallel = FALSE
create.plot	Logical argument whether to create a distribution plot of permuation results.
verbose	Logical argument whether output printed automatically in 'pretty' format. Default create.plot = FALSE
...	Extra arguments that the user would like to apply to the models

Value

p.value

Resulting p-value of permuation test

Author(s)

Charles Determan Jr.

References

Guo Y., et. al. (2010) Sample size and statistical power considerations in high-dimensionality data settings: a comparative study of classification algorithms. BMC Bioinformatics 11:447.

Examples

dat.discr <- create.discr.matrix(
    create.corr.matrix(
        create.random.matrix(nvar = 50, 
                             nsamp = 100, 
                             st.dev = 1, 
                             perturb = 0.2)),
    D = 10
)

vars <- dat.discr$discr.mat
groups <- dat.discr$classes

fits <- fs.stability(vars, 
                     groups, 
                     method = c("plsda", "rf"), 
                     f = 10, 
                     k = 3, 
                     k.folds = 10, 
                     verbose = 'none')

perm.class(fits, vars, groups, "rf", k.folds=5,
           metric="Accuracy", nperm=10)

cdeterman/OmicsMarkeR documentation built on May 13, 2019, 2:35 p.m.