Description Usage Arguments Details Value Note Author(s) See Also Examples
Estimation of classification accuracy by multiple classifiers with resampling procedure and comparisons of multiple classifiers.
1 2 3 4 5 6 
formula 
A formula of the form 
data 
Data frame from which variables specified in 
dat 
A matrix or data frame containing the explanatory variables if no formula is given as the principal argument. 
cl 
A factor specifying the class for each observation if no formula principal argument is given. 
method 
A vector of multiple classification methods to be used. Classifiers,
such as 
pars 
A list of resampling scheme such as Leaveoneout
crossvalidation, Crossvalidation, Randomised
validation (holdout) and Bootstrap, and control parameters
for the calculation of accuracy. See 
tr.idx 
User defined index of training samples. Can be generated by

comp 
Comparison method of multiple classifier. If 
... 
Additional parameters to 
subset 
Optional vector, specifying a subset of observations to be used. 
na.action 
Function which indicates what should happen when the data
contains 
The accuracy rates for classification are obtained used techniques such as Random Forest, Support Vector Machine, kNearest Neighbour Classification, Linear Discriminant Analysis and Linear Discriminant Analysis based on sampling methods, including Leaveoneout crossvalidation, Crossvalidation, Randomised validation (holdout) and Bootstrap.
An object of class maccest
, including the components:
method 
Classification method used. 
acc 
Accuracy rate. 
acc.iter 
Accuracy rate of each iteration. 
acc.std 
Standard derivation of accuracy rate. 
mar 
Prediction margin. 
mar.iter 
Prediction margin of each iteration. 
auc 
The area under receiver operating curve (AUC). 
auc.iter 
AUC of each iteration. 
comp 
Multiple comparison method used. 
h.test 
Hypothesis test results of multiple comparison. 
gl.pval 
Global or overall pvalue. 
mc.pval 
Pairwise comparison pvalues. 
sampling 
Sampling scheme used. 
niter 
Number of iteration. 
nreps 
Number of replications in each iteration. 
conf.mat 
Overall confusion matrix. 
acc.boot 
A list of bootrap error such as 
The maccest
can take any classification model if its argument
format is model(formula, data, subset, na.action, ...)
and
their corresponding method predict.model(object, newdata, ...)
can either return the only predicted class label or in a list with
name as class
, such as lda
and pcalda
.
As for the multiple comparisons by ANOVA
, the following
assumptions should be considered:
The samples are randomly and independently selected.
The populations are normally distributed.
The populations all have the same variance.
All the comparisons are based on the results of all iteration.
aam.mcl
is a simplified version which returns acc
(accuracy), auc
(area under ROC curve) and mar
(class
margin).
Wanchang Lin
accest
, aam.mcl
, valipars
,
plot.maccest
trainind
,
boxplot.maccest
,classifier
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19  # Iris data
data(iris)
x < subset(iris, select = Species)
y < iris$Species
method < c("randomForest","svm","pcalda","knn")
pars < valipars(sampling="boot", niter = 3, nreps=5, strat=TRUE)
res < maccest(Species~., data = iris, method=method, pars = pars,
comp="anova")
## or
res < maccest(x, y, method=method, pars=pars, comp="anova")
res
summary(res)
plot(res)
boxplot(res)
oldpar < par(mar = c(5,10,4,2) + 0.1)
plot(res$h.test$tukey,las=1) ## plot the tukey results
par(oldpar)

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.