mcca-package: Diagnostic accuracy methods for classifiers
In mcca: Multi-Category Classification Accuracy

Description Details Functions Installing and using Citation Author(s) References See Also Examples

Six common multi-category classification accuracy evaluation measures are included i.e., Correct Classification Percentage (CCP), Hypervolume Under Manifold (HUM), Integrated Discrimination Improvement (IDI), Net Reclassification Improvement (NRI), Polytomous Discrimination Index (PDI) and R-squared (RSQ). It allows users to fit many popular classification procedures, such as multinomial logistic regression, support vector machine, classification tree, and user computed risk values.

Package:	mcca
Type:	Package
Version:	0.6
Date:	2019-08-05
License:	GPL

`ccp`	Calculate CCP Value
`hum`	Calculate HUM Value
`plot.mcca.hum`	Plot 3D ROC curve
`idi`	Calculate IDI Value
`nri`	Calculate NRI Value
`pdi`	Calculate PDI Value
`rsq`	Calculate RSQ Value
`pm`	Calculate Probability Matrix
`ests`	Estimated Information for Single Model Evaluation Value
`estp`	Estimated Information for Paired Model Evaluation Value

To install this package, make sure you are connected to the internet and issue the following command in the R prompt:

1 2	install.packages("mcca")

To load the package in R:

1 2	library(mcca)

Li J, Gao M, D'Agostino R. Evaluating classification accuracy for modern learning approaches. Statistics in Medicine. 2019;1-27. https://doi.org/10.1002/sim.8103

Ming Gao, Jialiang Li

Maintainer: Ming Gao <gaoming@umich.edu>

Li, J., Gao, M., D’Agostino, R. (2019). Evaluating Classification Accuracy for Modern Learning Approaches. Statistics in Medicine (Tutorials in Biostatistics). 38(13): 2477-2503.

Li, Ming G., D'Agostino. (2019). Evaluating Classification Accuracy for Modern Learning Approaches. Statistics in Medicine.

Li, J. and Fine, J. P. (2008): ROC analysis with multiple tests and multiple classes: methodology and applications in microarray studies. Biostatistics. 9 (3): 566-576.

Li, J., Chow, Y., Wong, W.K., and Wong, T.Y. (2014). Sorting Multiple Classes in Multi-dimensional ROC Analysis: Parametric and Nonparametric Approaches. Biomarkers. 19(1): 1-8.

Li, J., Jiang, B. and Fine, J. P. (2013). Multicategory reclassification statistics for assessing Improvements in diagnostic accuracy. Biostatistics. 14(2): 382-394.

Li, J., Jiang, B., and Fine, J. P. (2013). Letter to Editor: Response. Biostatistics. 14(4): 809-810.

Van Calster B, Vergouwe Y, Looman CWN, Van Belle V, Timmerman D and Steyerberg EW. Assessing the discriminative ability of risk models for more than two outcome categories. European Journal of Epidemiology 2012; 27: 761 C 770.

Li, J., Feng, Q., Fine, J.P., Pencina, M.J., Van Calster, B. (2018). Nonparametric estimation and inference for polytomous discrimination index. Statistical Methods in Medical Research. 27(10): 3092—3103.

CRAN packages HUM for HUM.

CRAN packages nnet, rpart, e1071, MASS employed in this package.

str(iris)
data <- iris[, 1:4]
label <- iris[, 5]
ccp(y = label, d = data, method = "multinom",maxit = 1000,MaxNWts = 2000,trace=FALSE)
ccp(y = label, d = data, method = "multinom")
ccp(y = label, d = data, method = "svm")
ccp(y = label, d = data, method = "svm",kernel="sigmoid",cost=4,scale=TRUE,coef0=0.5)
ccp(y = label, d = data, method = "tree")
p = as.numeric(label)
ccp(y = label, d = p, method = "label")
hum(y = label, d = data,method = "multinom")
hum(y = label, d = data,method = "svm")
hum(y = label, d = data,method = "svm",kernel="linear",cost=4,scale=TRUE)
hum(y = label, d = data, method = "tree")
ests(y = label, d = data,acc="hum",level=0.95,method = "multinom",trace=FALSE)

## $value
## [1] 0.9972

## $sd
## [1] 0.002051529

## $interval
## [1] 0.9935662 1.0000000